Mark/osint-hub

Fork 0

System is a non-functional stub - not a real OSINT investigation tool — Phase 3: Impact & Solutions #83

New issue

Closed

opened 2026-06-08 15:29:26 +00:00 by Mark · 0 comments

Mark commented

2026-06-08 15:29:26 +00:00

Owner

Phase 3: Impact Assessment & Proposed Solutions

Follow-up to Phase 1: Problem. This issue assesses the impact of each identified problem and proposes concrete solutions.

🔴 Critical Impact Items

1. Alembic Migration Divergence

Impact: alembic upgrade head will fail in production. New deployments cannot apply migrations. Blocks ALL other changes.

Solution: Create merge migration reconciling heads d1e2f3a4b5c6 and a2b3c4d5e8f0. Verify with alembic check. Update alembic/env.py if needed.

Estimated effort: 2-4 hours

2. Missing OSINT Data Models

Impact: Social profiles discovered by 10+ gathering modules cannot be stored in normalized format. IP addresses extracted by LLM have no dedicated table. Person-to-domain associations impossible.

Solution:

Create PersonSocialProfile table: id, person_id, case_id, platform, profile_url, username, display_name, is_verified, discovered_at, source_module, raw_data_json
Create IPAddress table: id, case_id, ip, type (ipv4/ipv6), asn, isp, geolocation_json, first_seen, last_seen, tags
Create PersonDomain table: id, person_id, case_id, domain, is_verified, is_primary, source, notes
Update EntityType enum to include social_profile, ip_address
Update RelationshipCreate schema to support new entity types

Estimated effort: 1-2 days

3. API Schemas Disconnected from Normalized Tables

Impact: OrganizationResponse and PersonResponse return flat string fields (e.g., emails: "a@b.com, c@d.com") instead of structured arrays. Frontend must parse these manually.

Solution:

Update OrganizationResponse to include emails: List[EmailSchema], phones: List[PhoneSchema], domains: List[DomainSchema]
Update PersonResponse to include emails: List[EmailSchema], phones: List[PhoneSchema]
Remove flat string fields or keep as deprecated/computed
Update frontend to use structured data

Estimated effort: 1-2 days

🟡 High Impact Items

4. No Module Name Conflict Detection

Impact: Two DashboardSummaryModule classes register under the same name silently. Wrong version may execute depending on import order.

Solution: Add conflict detection in ModuleRegistry.register() — log warning and raise on duplicate. Add command to list unique module names. Fix duplicate by renaming one.

Estimated effort: 2-4 hours

5. Dead Code: `dashboard_summary.py`

Impact: Confusing for developers. Violates single source of truth.

Solution: Remove dashboard_summary.py entirely, or fix to be async with correct EvidenceResult() constructor and unique name.

Estimated effort: 1 hour

6. Fragile String-Based Module Resolution

Impact: cascade_search.py references modules by hardcoded strings. Module renames break the cascade silently.

Solution: Add startup validation checking all cascade_references resolve. Use constants/enum for module names. Add unit test verifying references.

Estimated effort: 4-6 hours

7. No End-to-End Tests

Impact: Cannot validate orchestrator + LLM + modules pipeline works end-to-end.

Solution: Add E2E test with local Ollama or realistic mock that asserts entities created, evidence stored, relationships established, termination criteria fire.

Estimated effort: 2-3 days

8. Multiple Orchestrator Versions

Impact: 5 variants. Unclear which is production. API router uses non-enhanced version.

Solution: Consolidate to 2 orchestrators: (1) LLM-driven with entity queue, (2) Simple module executor. Deprecate/remove v2, v3 after verifying no callers.

Estimated effort: 2-3 days

🟠 Medium Impact Items

9. Missing FK Constraint

Impact: Evidence can reference non-existent parent.

Solution: Add ForeignKeyConstraint on EvidenceItem.parent_evidence_id. Create migration.

Estimated effort: 1-2 hours

10. `asyncio.run()` in Sync Contexts

Impact: Nested event loop errors in async contexts.

Solution: Refactor cascade_search.py and orchestrator.run_investigation() to be fully async.

Estimated effort: 1-2 days

11. EnrichmentPanel Styling Broken

Impact: Component renders but is invisible due to missing CSS classes.

Solution: Include Tailwind CSS or convert classes to project's existing style system.

Estimated effort: 2-4 hours

12. "Coming Soon" Relationship Button

Impact: Feature gap visible to users.

Solution: Implement Add Relationship or hide/remove button.

Estimated effort: 2-4 hours

13. Inconsistent Theming & No State Management

Impact: Visual inconsistency, duplicate API calls.

Solution: Standardize on dark theme. Add React Context or Zustand.

Estimated effort: 1-2 days

Companion to Phase 1: Problem

## Phase 3: Impact Assessment & Proposed Solutions Follow-up to [Phase 1: Problem](https://forgejo.tail018ac4.ts.net/Mark/osint-hub/issues/79). This issue assesses the impact of each identified problem and proposes concrete solutions. --- ### 🔴 Critical Impact Items #### 1. Alembic Migration Divergence **Impact**: `alembic upgrade head` will fail in production. New deployments cannot apply migrations. Blocks ALL other changes. **Solution**: Create merge migration reconciling heads `d1e2f3a4b5c6` and `a2b3c4d5e8f0`. Verify with `alembic check`. Update `alembic/env.py` if needed. **Estimated effort**: 2-4 hours #### 2. Missing OSINT Data Models **Impact**: Social profiles discovered by 10+ gathering modules cannot be stored in normalized format. IP addresses extracted by LLM have no dedicated table. Person-to-domain associations impossible. **Solution**: - Create `PersonSocialProfile` table: `id, person_id, case_id, platform, profile_url, username, display_name, is_verified, discovered_at, source_module, raw_data_json` - Create `IPAddress` table: `id, case_id, ip, type (ipv4/ipv6), asn, isp, geolocation_json, first_seen, last_seen, tags` - Create `PersonDomain` table: `id, person_id, case_id, domain, is_verified, is_primary, source, notes` - Update `EntityType` enum to include `social_profile`, `ip_address` - Update `RelationshipCreate` schema to support new entity types **Estimated effort**: 1-2 days #### 3. API Schemas Disconnected from Normalized Tables **Impact**: `OrganizationResponse` and `PersonResponse` return flat string fields (e.g., `emails: "a@b.com, c@d.com"`) instead of structured arrays. Frontend must parse these manually. **Solution**: - Update `OrganizationResponse` to include `emails: List[EmailSchema]`, `phones: List[PhoneSchema]`, `domains: List[DomainSchema]` - Update `PersonResponse` to include `emails: List[EmailSchema]`, `phones: List[PhoneSchema]` - Remove flat string fields or keep as deprecated/computed - Update frontend to use structured data **Estimated effort**: 1-2 days --- ### 🟡 High Impact Items #### 4. No Module Name Conflict Detection **Impact**: Two `DashboardSummaryModule` classes register under the same name silently. Wrong version may execute depending on import order. **Solution**: Add conflict detection in `ModuleRegistry.register()` — log warning and raise on duplicate. Add command to list unique module names. Fix duplicate by renaming one. **Estimated effort**: 2-4 hours #### 5. Dead Code: `dashboard_summary.py` **Impact**: Confusing for developers. Violates single source of truth. **Solution**: Remove `dashboard_summary.py` entirely, or fix to be async with correct `EvidenceResult()` constructor and unique name. **Estimated effort**: 1 hour #### 6. Fragile String-Based Module Resolution **Impact**: `cascade_search.py` references modules by hardcoded strings. Module renames break the cascade silently. **Solution**: Add startup validation checking all cascade_references resolve. Use constants/enum for module names. Add unit test verifying references. **Estimated effort**: 4-6 hours #### 7. No End-to-End Tests **Impact**: Cannot validate orchestrator + LLM + modules pipeline works end-to-end. **Solution**: Add E2E test with local Ollama or realistic mock that asserts entities created, evidence stored, relationships established, termination criteria fire. **Estimated effort**: 2-3 days #### 8. Multiple Orchestrator Versions **Impact**: 5 variants. Unclear which is production. API router uses non-enhanced version. **Solution**: Consolidate to 2 orchestrators: (1) LLM-driven with entity queue, (2) Simple module executor. Deprecate/remove v2, v3 after verifying no callers. **Estimated effort**: 2-3 days --- ### 🟠 Medium Impact Items #### 9. Missing FK Constraint **Impact**: Evidence can reference non-existent parent. **Solution**: Add `ForeignKeyConstraint` on `EvidenceItem.parent_evidence_id`. Create migration. **Estimated effort**: 1-2 hours #### 10. `asyncio.run()` in Sync Contexts **Impact**: Nested event loop errors in async contexts. **Solution**: Refactor `cascade_search.py` and `orchestrator.run_investigation()` to be fully async. **Estimated effort**: 1-2 days #### 11. EnrichmentPanel Styling Broken **Impact**: Component renders but is invisible due to missing CSS classes. **Solution**: Include Tailwind CSS or convert classes to project's existing style system. **Estimated effort**: 2-4 hours #### 12. "Coming Soon" Relationship Button **Impact**: Feature gap visible to users. **Solution**: Implement Add Relationship or hide/remove button. **Estimated effort**: 2-4 hours #### 13. Inconsistent Theming & No State Management **Impact**: Visual inconsistency, duplicate API calls. **Solution**: Standardize on dark theme. Add React Context or Zustand. **Estimated effort**: 1-2 days --- _Companion to [Phase 1: Problem](https://forgejo.tail018ac4.ts.net/Mark/osint-hub/issues/79)_

Mark

2026-06-08 15:29:42 +00:00

closed this issue
added the
triage-approved
label

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

Mark/osint-hub#83

No description provided.

Rows
Columns

System is a non-functional stub - not a real OSINT investigation tool — Phase 3: Impact & Solutions #83

Phase 3: Impact Assessment & Proposed Solutions

🔴 Critical Impact Items

1. Alembic Migration Divergence

2. Missing OSINT Data Models

3. API Schemas Disconnected from Normalized Tables

🟡 High Impact Items

4. No Module Name Conflict Detection

5. Dead Code: dashboard_summary.py

6. Fragile String-Based Module Resolution

7. No End-to-End Tests

8. Multiple Orchestrator Versions

🟠 Medium Impact Items

9. Missing FK Constraint

10. asyncio.run() in Sync Contexts

11. EnrichmentPanel Styling Broken

12. "Coming Soon" Relationship Button

13. Inconsistent Theming & No State Management

5. Dead Code: `dashboard_summary.py`

10. `asyncio.run()` in Sync Contexts