Mark/osint-hub
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

System is a non-functional stub - not a real OSINT investigation tool — Phase 1: Problem #79

New issue
Closed
opened 2026-05-28 00:36:57 +00:00 by Mark · 3 comments
Mark commented 2026-05-28 00:36:57 +00:00
Owner
Copy link

Phase 1: Problem — Complete Analysis

Original Claim (from #13)

Current system is a proof-of-concept stub, not a real OSINT tool.

Finding: Original Claim No Longer Accurate

The comprehensive codebase audit (2026-06-08) reveals the system has grown significantly beyond its proof-of-concept origins:

Original Issue (#13) Current Status Evidence
Runs only 6 modules instead of 60+ FIXED 78 registered modules, all with working gather()
Data model cannot store basic OSINT data PARTIALLY FIXED Emails/phones normalized. Missing: social profiles, IP addresses, person-domains
No dynamic spawning of investigations FIXED EntityActionQueue + pipeline runner with recursive branching
No recursive sub-investigations for persons FIXED Person queue, playbook, LoopDetector
Findings are generic summaries FIXED Evidence-grounded findings with supporting_evidence_ids

Real Remaining Problems

🔴 Critical (Blocks Production)

  1. Alembic migration divergence — Two unresolved heads, alembic upgrade head will fail
  2. Missing OSINT models — No SocialProfile, IPAddress, PersonDomain tables
  3. API schemas disconnected from DB — Responses flatten normalized data into strings

🟡 High (Degrades Quality)

  1. No module name conflict detection — Silent overwrite on duplicate registration
  2. Dead code: dashboard_summary.py — broken class never executes
  3. Fragile string-based module resolution — No validation in cascade_search.py
  4. No E2E tests — 103 tests pass but all use mocks; 4 tests have stale assertions
  5. 5 orchestrator variants — Unclear which is production

🟠 Medium (Robustness)

  1. Missing FK constraint on EvidenceItem.parent_evidence_id
  2. asyncio.run() in sync contexts risks nested event loops
  3. EnrichmentPanel uses Tailwind classes without Tailwind
  4. "Coming Soon" disabled button
  5. Inconsistent theming, no shared state management

Conclusion

The "stub" characterization is inaccurate. The real problem is unresolved technical debt and missing data models that prevent production deployment despite substantial functional implementation.

See comment 35141 for full audit details.

Decomposed from osint-hub#13

## Phase 1: Problem — Complete Analysis ### Original Claim (from #13) > Current system is a proof-of-concept stub, not a real OSINT tool. ### Finding: Original Claim No Longer Accurate The comprehensive codebase audit (2026-06-08) reveals the system has grown significantly beyond its proof-of-concept origins: | Original Issue (#13) | Current Status | Evidence | |---|---|---| | Runs only 6 modules instead of 60+ | **FIXED** | 78 registered modules, all with working `gather()` | | Data model cannot store basic OSINT data | **PARTIALLY FIXED** | Emails/phones normalized. Missing: social profiles, IP addresses, person-domains | | No dynamic spawning of investigations | **FIXED** | EntityActionQueue + pipeline runner with recursive branching | | No recursive sub-investigations for persons | **FIXED** | Person queue, playbook, LoopDetector | | Findings are generic summaries | **FIXED** | Evidence-grounded findings with supporting_evidence_ids | ### Real Remaining Problems #### 🔴 Critical (Blocks Production) 1. **Alembic migration divergence** — Two unresolved heads, `alembic upgrade head` will fail 2. **Missing OSINT models** — No SocialProfile, IPAddress, PersonDomain tables 3. **API schemas disconnected from DB** — Responses flatten normalized data into strings #### 🟡 High (Degrades Quality) 4. **No module name conflict detection** — Silent overwrite on duplicate registration 5. **Dead code**: `dashboard_summary.py` — broken class never executes 6. **Fragile string-based module resolution** — No validation in cascade_search.py 7. **No E2E tests** — 103 tests pass but all use mocks; 4 tests have stale assertions 8. **5 orchestrator variants** — Unclear which is production #### 🟠 Medium (Robustness) 9. Missing FK constraint on EvidenceItem.parent_evidence_id 10. `asyncio.run()` in sync contexts risks nested event loops 11. EnrichmentPanel uses Tailwind classes without Tailwind 12. "Coming Soon" disabled button 13. Inconsistent theming, no shared state management ### Conclusion The "stub" characterization is inaccurate. The real problem is **unresolved technical debt and missing data models** that prevent production deployment despite substantial functional implementation. See comment 35141 for full audit details. --- _Decomposed from [osint-hub#13](https://forgejo.tail018ac4.ts.net/Mark/osint-hub/issues/13)_
Mark commented 2026-05-28 04:42:04 +00:00
Author
Owner
Copy link

⏸ Needs Decision

Blocking factors:

  • Didn't meet auto-approve criteria — scope=medium, risk=medium, spec=partial

Scope: medium Risk: medium Spec: partial Type: feature
Summary: assessment timed out
Suggestion: retry needed
Rationale: hq timeout

Action: Review each blocking factor above. Remove blocked:needs-decision label to route back to triage.

## ⏸ Needs Decision **Blocking factors:** - **Didn't meet auto-approve criteria** — scope=medium, risk=medium, spec=partial --- **Scope:** medium **Risk:** medium **Spec:** partial **Type:** feature **Summary:** assessment timed out **Suggestion:** retry needed **Rationale:** hq timeout **Action:** Review each blocking factor above. Remove `blocked:needs-decision` label to route back to triage.
Mark commented 2026-05-28 08:28:33 +00:00
Author
Owner
Copy link

⏸ Needs Decision

What needs to be decided?

  • Is this issue still relevant? If yes, remove the blocked:needs-decision label to route back to triage.

Blocking factors:

  • Didn't meet auto-approve criteria — scope=small, risk=low, spec=partial

Scope: small Risk: low Spec: partial Type: other
Summary: States the OSINT Hub is a proof-of-concept stub without specifying what concrete changes are needed.
Suggestion: Close as non-actionable — this is a problem statement with no deliverables, no definition of done, and no actionable work items. The codebase already has extensive module infrastructure; if there are concrete gaps, they need specific issues with measurable outcomes. The actual work specs should be in subsequent phases decomposed from #13.
Rationale: The issue body is a single sentence with zero implementation details. Despite claiming the system is a 'stub,' the actual codebase has 80+ real modules, 39K lines of tests, full orchestrator/playbook/LLM integration, and a functional frontend. Without concrete gaps identified, this cannot be triaged as actionable work. The ROADMAP.md already defines a hardening trajectory from Phase 0-5.

To unblock: Remove the blocked:needs-decision label. The triage will apply the appropriate outcome based on your choice above.

## ⏸ Needs Decision **What needs to be decided?** - **Is this issue still relevant?** If yes, remove the `blocked:needs-decision` label to route back to triage. **Blocking factors:** - **Didn't meet auto-approve criteria** — scope=small, risk=low, spec=partial --- **Scope:** small **Risk:** low **Spec:** partial **Type:** other **Summary:** States the OSINT Hub is a proof-of-concept stub without specifying what concrete changes are needed. **Suggestion:** Close as non-actionable — this is a problem statement with no deliverables, no definition of done, and no actionable work items. The codebase already has extensive module infrastructure; if there are concrete gaps, they need specific issues with measurable outcomes. The actual work specs should be in subsequent phases decomposed from #13. **Rationale:** The issue body is a single sentence with zero implementation details. Despite claiming the system is a 'stub,' the actual codebase has 80+ real modules, 39K lines of tests, full orchestrator/playbook/LLM integration, and a functional frontend. Without concrete gaps identified, this cannot be triaged as actionable work. The ROADMAP.md already defines a hardening trajectory from Phase 0-5. **To unblock:** Remove the `blocked:needs-decision` label. The triage will apply the appropriate outcome based on your choice above.
Mark commented 2026-06-08 15:28:48 +00:00
Author
Owner
Copy link

Phase 1: Problem — Comprehensive Analysis

Original Claim (from #13)

Current system is a proof-of-concept stub, not a real OSINT tool.

Evidence-Based Reassessment

After thorough codebase audit, the original claim no longer reflects the current state. The following original issues from #13 have been resolved:

Original Issue (#13) Status Evidence
Runs only 6 modules instead of 60+ FIXED 78 registered modules, all with working gather() returning EvidenceResult. No stubs, bare pass, or NotImplementedError in any module.
Data model cannot store basic OSINT data PARTIALLY FIXED 36 tables. Emails/phones properly normalized (PersonEmail, PersonPhone, OrganizationEmail, OrganizationPhone). But missing: social profile table, IP address table, person-domain table.
No dynamic spawning of investigations FIXED _auto_queue_extracted_entities() in orchestrator_enhanced.py. Pipeline runner with recursive branch spawning up to max_depth=3. EntityActionQueue with type-specific priorities.
No recursive sub-investigations for persons FIXED Person queue at orchestrator_enhanced.py:943. Person playbook in entity_module_map.py. LoopDetector prevents infinite recursion.
Findings are generic summaries FIXED create_finding() with statement + supporting_evidence_ids. create_evidence_snippet() with grounded quotes. All evidence items have summary, quote_snippet, capture_method.

What Remains: The Real Problems

While the system is NOT a stub, the following production-hardening gaps block it from being a real OSINT tool:

🔴 Critical (Blocks Production Use)

  1. Alembic migration divergence — Two unresolved heads (d1e2f3a4b5c6 and a2b3c4d5e8f0). alembic check fails. alembic upgrade head will fail in production. Must be merged before any deployment.

  2. Missing OSINT data models — No SocialProfile table (10+ modules discover social profiles but store them in a free-text Person.public_profiles field). No IPAddress table. No PersonDomain table. These are fundamental OSINT data types with nowhere to live.

  3. API schemas disconnected from normalized tables — OrganizationResponse and PersonResponse return emails, phones, domains as flat comma-separated strings instead of serving from normalized subtables. The API layer doesn't reflect the database schema.

🟡 High (Degrades Quality Significantly)

  1. No module name conflict detection — Two files define DashboardSummaryModule with name="dashboard_summary". The second import silently overwrites the first. No warning, no error. This will cause data loss when the wrong version runs.

  2. Dead code: dashboard_summary.py — Contains a broken DashboardSummaryModule with synchronous gather() (violates ABC contract) and invalid EvidenceResult() constructor call. Never executes at runtime (overwritten by activity_tracker.py) but signals inconsistent maintenance.

  3. Fragile string-based module resolution — cascade_search.py references modules like "social_profile_finder" and "haveibeenpwned" by string with no validation that the target module exists. A rename breaks the cascade silently.

  4. No end-to-end tests — 103 tests pass, but ALL module tests use mocks. No test exercises the orchestrator against a real LLM with actual module execution. The 4 failing tests have stale assertions that mask real changes.

  5. Multiple orchestrator versions — 5 orchestrator variants (v1, enhanced, v2, v3, v3-llm). Unclear which is the production orchestrator. The API router uses non-enhanced run_investigation. Maintenance burden grows with each version.

🟠 Medium (Quality of Life / Robustness)

  1. Missing FK constraint on EvidenceItem.parent_evidence_id — Comment says "FK to evidence_item.id" but no foreign key constraint is defined. Referential integrity gap.

  2. asyncio.run() in synchronous contexts — cascade_search.py and orchestrator.run_investigation() use asyncio.run() inside synchronous functions. Causes nested event loop errors when called from async contexts.

  3. EnrichmentPanel uses Tailwind/DaisyUI classes without the CSS framework — Invisible broken component styled with classes that don't exist.

  4. Hardcoded admin credentials in frontend — app.jsx:36 defaults to admin@example.local / adminadmin. Issue #34 tracks this separately.

  5. "Coming Soon" button — CaseDetail.jsx:1264, Add Relationship button is disabled with "Coming Soon". Feature gap.

  6. Inconsistent theming — Mix of dark and light background colors across components (#ddd, #f5f5f5 inline styles clash with dark theme).

  7. No shared state management — Components independently fetch the same data, leading to duplicate API calls. No cache layer.

Summary

The system has grown beyond its proof-of-concept origins and now has substantial real functionality:

  • 78 functional OSINT modules
  • Complete LLM-driven investigation orchestration
  • 36 normalized database tables
  • Full frontend with evidence review, relationship graphs, report generation
  • 1030+ lines of investigation services, 1000+ tests

However, it suffers from production-hardening gaps typical of a project that grew fast without dedicated quality/infrastructure work. The original claim of "stub" is inaccurate — the real problem is unresolved technical debt and missing OSINT-specific data models that prevent production deployment.

Recommended Next Steps

  1. Create Phase 3 (Impact/Solution Required) — Prioritize each gap above with severity, impact assessment, and proposed solution
  2. Create Phase 5 (Blockers) — Identify blocking factors for each solution
  3. Merge alembic heads as immediate blocker
  4. Add SocialProfile, IPAddress models as highest-value data model work
  5. Write E2E test with local Ollama LLM to validate the orchestrator pipeline
## Phase 1: Problem — Comprehensive Analysis ### Original Claim (from #13) > Current system is a proof-of-concept stub, not a real OSINT tool. ### Evidence-Based Reassessment After thorough codebase audit, the original claim **no longer reflects the current state**. The following original issues from #13 have been **resolved**: | Original Issue (#13) | Status | Evidence | |---|---|---| | Runs only 6 modules instead of 60+ | **FIXED** | 78 registered modules, all with working `gather()` returning `EvidenceResult`. No stubs, bare `pass`, or `NotImplementedError` in any module. | | Data model cannot store basic OSINT data | **PARTIALLY FIXED** | 36 tables. Emails/phones properly normalized (PersonEmail, PersonPhone, OrganizationEmail, OrganizationPhone). But missing: social profile table, IP address table, person-domain table. | | No dynamic spawning of investigations | **FIXED** | `_auto_queue_extracted_entities()` in orchestrator_enhanced.py. Pipeline runner with recursive branch spawning up to `max_depth=3`. EntityActionQueue with type-specific priorities. | | No recursive sub-investigations for persons | **FIXED** | Person queue at orchestrator_enhanced.py:943. Person playbook in entity_module_map.py. LoopDetector prevents infinite recursion. | | Findings are generic summaries | **FIXED** | `create_finding()` with statement + supporting_evidence_ids. `create_evidence_snippet()` with grounded quotes. All evidence items have summary, quote_snippet, capture_method. | ### What Remains: The Real Problems While the system is NOT a stub, the following **production-hardening gaps** block it from being a real OSINT tool: #### 🔴 Critical (Blocks Production Use) 1. **Alembic migration divergence** — Two unresolved heads (`d1e2f3a4b5c6` and `a2b3c4d5e8f0`). `alembic check` fails. `alembic upgrade head` will fail in production. **Must be merged before any deployment.** 2. **Missing OSINT data models** — No `SocialProfile` table (10+ modules discover social profiles but store them in a free-text `Person.public_profiles` field). No `IPAddress` table. No `PersonDomain` table. These are fundamental OSINT data types with nowhere to live. 3. **API schemas disconnected from normalized tables** — `OrganizationResponse` and `PersonResponse` return `emails`, `phones`, `domains` as flat comma-separated strings instead of serving from normalized subtables. The API layer doesn't reflect the database schema. #### 🟡 High (Degrades Quality Significantly) 4. **No module name conflict detection** — Two files define `DashboardSummaryModule` with `name="dashboard_summary"`. The second import silently overwrites the first. No warning, no error. This will cause data loss when the wrong version runs. 5. **Dead code: `dashboard_summary.py`** — Contains a broken `DashboardSummaryModule` with synchronous `gather()` (violates ABC contract) and invalid `EvidenceResult()` constructor call. Never executes at runtime (overwritten by activity_tracker.py) but signals inconsistent maintenance. 6. **Fragile string-based module resolution** — `cascade_search.py` references modules like `"social_profile_finder"` and `"haveibeenpwned"` by string with no validation that the target module exists. A rename breaks the cascade silently. 7. **No end-to-end tests** — 103 tests pass, but ALL module tests use mocks. No test exercises the orchestrator against a real LLM with actual module execution. The 4 failing tests have stale assertions that mask real changes. 8. **Multiple orchestrator versions** — 5 orchestrator variants (v1, enhanced, v2, v3, v3-llm). Unclear which is the production orchestrator. The API router uses non-enhanced `run_investigation`. Maintenance burden grows with each version. #### 🟠 Medium (Quality of Life / Robustness) 9. **Missing FK constraint on `EvidenceItem.parent_evidence_id`** — Comment says "FK to evidence_item.id" but no foreign key constraint is defined. Referential integrity gap. 10. **`asyncio.run()` in synchronous contexts** — `cascade_search.py` and `orchestrator.run_investigation()` use `asyncio.run()` inside synchronous functions. Causes nested event loop errors when called from async contexts. 11. **EnrichmentPanel uses Tailwind/DaisyUI classes without the CSS framework** — Invisible broken component styled with classes that don't exist. 12. **Hardcoded admin credentials in frontend** — `app.jsx:36` defaults to `admin@example.local` / `adminadmin`. Issue #34 tracks this separately. 13. **"Coming Soon" button** — CaseDetail.jsx:1264, Add Relationship button is disabled with "Coming Soon". Feature gap. 14. **Inconsistent theming** — Mix of dark and light background colors across components (`#ddd`, `#f5f5f5` inline styles clash with dark theme). 15. **No shared state management** — Components independently fetch the same data, leading to duplicate API calls. No cache layer. ### Summary The system has **grown beyond its proof-of-concept origins** and now has substantial real functionality: - 78 functional OSINT modules - Complete LLM-driven investigation orchestration - 36 normalized database tables - Full frontend with evidence review, relationship graphs, report generation - 1030+ lines of investigation services, 1000+ tests However, it suffers from **production-hardening gaps** typical of a project that grew fast without dedicated quality/infrastructure work. The original claim of "stub" is inaccurate — the real problem is **unresolved technical debt and missing OSINT-specific data models** that prevent production deployment. ### Recommended Next Steps 1. **Create Phase 3 (Impact/Solution Required)** — Prioritize each gap above with severity, impact assessment, and proposed solution 2. **Create Phase 5 (Blockers)** — Identify blocking factors for each solution 3. **Merge alembic heads** as immediate blocker 4. **Add SocialProfile, IPAddress models** as highest-value data model work 5. **Write E2E test** with local Ollama LLM to validate the orchestrator pipeline
Mark closed this issue 2026-06-08 15:29:04 +00:00
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
No results found.
Labels
Clear labels   
agentic-osint

Agentic OSINT features

  
audit

   
blocked:commit-ready-failed

   
blocked:gate-fetch

   
blocked:gate-fetch

   
blocked:gate-lint

   
blocked:gate-plan

   
blocked:gate-research

   
blocked:needs-decision

   
blocked:needs-human

   
blocked:no-changes

   
blocked:preflight

Repo has dirty lint; kilo-gsd preflight refuses to dispatch until lint-cleanup lands

  
blocked:verify-failed

   
bug

Something is not working

  
chore

Config/cleanup item

  
deferred

Deferred — will revisit later

  
enhancement

Enhancement request

  
feature-request

   
in-progress

   
lint-baseline

   
lint-cleanup

kilo-gsd preflight lint gate: repo has dirty ruff state, cleanup required before fixer dispatch

  
needs-decompose

Issue needs decompose processing

  
needs-triage

   
performance

   
quickfix

   
security/medium

Medium severity security issue

  
triage-approved

Approved by triage

No labels
agentic-osint
audit
blocked:commit-ready-failed
blocked:gate-fetch
blocked:gate-fetch
blocked:gate-lint
blocked:gate-plan
blocked:gate-research
blocked:needs-decision
blocked:needs-human
blocked:no-changes
blocked:preflight
blocked:verify-failed
bug
chore
deferred
enhancement
feature-request
in-progress
lint-baseline
lint-cleanup
needs-decompose
needs-triage
performance
quickfix
security/medium
triage-approved
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
Mark/osint-hub#79
No description provided.