fix(lint): resolve PLC0415 violations blocking kilo-gsd (refs #97) #98

Closed

Mark wants to merge 0 commits from fix/lint-plc0415 into main

pull from: fix/lint-plc0415

merge into: Mark:main

Mark:main

Conversation 0 Commits - Files changed - +165 -273

Mark commented 2026-08-13 21:58:37 +00:00

Owner

Copy link

Summary

Resolve all ruff lint errors so ruff check . exits 0 against Mark/osint-hub, unblocking kilo-gsd's preflight gate (refs #97). kilo-gsd's preflight truncates ruff output to 1500 chars, so #97 only listed 5 violations — the repo actually had 219 ruff errors across 50 files. This PR clears all of them.

Changes

Pure import relocations plus a small set of non-PLC lint fixes. No behavior changes.

PLC0415 (import at top-level) — hoisted to top: safe stdlib/local imports across modules, routers, services, and tasks (json, re, logging, io, base64, hashlib, ipaddress, subprocess, tempfile, Path, shutil, Decimal, SequenceMatcher, sqlmodel/sqlalchemy sessions, from ..settings import settings, from ..models/..db/..tasks where circular-safe).

PLC0415 — kept lazy with justified # noqa comments (hoisting would change behavior):

• Optional-dependency guarded imports: PIL, pytesseract, pdf2image, pdfplumber, PyPDF2, weasyprint, cv2 — each already wrapped in try/except for graceful degradation; a comment documents the guard.
• Circular-import avoiders: entity_dispatcher.py (orchestrator_enhanced), scheduler.py (queue_manager), tasks.py (queue_manager).
• Test seam: queue_manager.py keeps from app.db import engine lazy because test_process_queue_batch patches app.db.engine; hoisting would bind the real engine at import time and break the test.
• Best-effort enrichment guard in gather.py.
• registry.py: file-level # ruff: noqa: PLC0415 with a docstring note — register_all_modules() deliberately imports each optional module in a guarded try/except so a missing optional dep skips that module instead of crashing registration.

Non-PLC fixes:

• services/execution.py + entity_module_map.py: str, Enum → StrEnum (UP042).
• retry_handler.py + llm/parsing.py: PEP 695 type-parameter syntax (UP047).
• test_module_benchmarks.py: justified # noqa: PT028 (default arg substitutes for parametrize).
• Alembic merge migration: removed unused alembic.op/sqlalchemy imports, typing.Sequence/Union → collections.abc.Sequence + X | Y annotations (UP035/UP007/I001/F401). Revision ids unchanged.
• llm/tools/evidence_extraction.py: fixed a latent broken relative import (from ...models → from ....models; the original lazy import resolved to nonexistent app.services.models and would have failed at call time).

Changed files

• backend/alembic/versions/a25b7f06b9f1_merge_organization_fields_and_priority_.py
• backend/app/change_detector.py
• backend/app/main.py
• backend/app/modules/abuseipdb.py
• backend/app/modules/dark_web_monitor.py
• backend/app/modules/dashboard_summary.py
• backend/app/modules/employee_scraper.py
• backend/app/modules/entity_linker.py
• backend/app/modules/exif_extractor.py
• backend/app/modules/google_dorks.py
• backend/app/modules/local_deep_research.py
• backend/app/modules/ocr_extractor.py
• backend/app/modules/osint_playbook.py
• backend/app/modules/pdf_analyzer.py
• backend/app/modules/pdf_report.py
• backend/app/modules/registry.py
• backend/app/modules/reverse_image_search.py
• backend/app/modules/search_aggregator.py
• backend/app/modules/shodan.py
• backend/app/modules/video_frame_extractor.py
• backend/app/modules/video_metadata.py
• backend/app/modules/virustotal.py
• backend/app/monitoring.py
• backend/app/routers/enrichment.py
• backend/app/routers/evidence.py
• backend/app/routers/gather.py
• backend/app/routers/investigations.py
• backend/app/routers/reports.py
• backend/app/services/browser_tools.py
• backend/app/services/enrichment_pipeline/runner.py
• backend/app/services/execution.py
• backend/app/services/investigations/async_executor.py
• backend/app/services/investigations/entity_dispatcher.py
• backend/app/services/investigations/entity_module_map.py
• backend/app/services/investigations/evidence_dispatcher.py
• backend/app/services/investigations/llm_entity_extraction.py
• backend/app/services/investigations/orchestrator.py
• backend/app/services/investigations/orchestrator_enhanced.py
• backend/app/services/investigations/orchestrator_v3_llm.py
• backend/app/services/investigations/queue_manager.py
• backend/app/services/investigations/retry_handler.py
• backend/app/services/investigations/runbooks.py
• backend/app/services/investigations/scheduler.py
• backend/app/services/llm/parsing.py
• backend/app/services/llm/tools/evidence_extraction.py
• backend/app/services/semantic_retrieval.py
• backend/app/tasks.py
• backend/scripts/scraper_poc.py
• backend/test_entity_investigation.py
• backend/tests/unit/test_module_benchmarks.py

Verification

• ruff check . → exit 0, All checks passed!
• ruff check backend/ → exit 0, All checks passed!
• ruff check backend/app/ → exit 0, All checks passed! (CI lint scope)
• pytest tests/unit → 945 passed, 1 skipped
• Import smoke test: app.main, app.tasks, app.monitoring, all routers, and all 35 touched modules import cleanly — no circular imports introduced.

## Summary Resolve all ruff lint errors so `ruff check .` exits 0 against `Mark/osint-hub`, unblocking kilo-gsd's preflight gate (refs #97). kilo-gsd's preflight truncates ruff output to 1500 chars, so #97 only listed 5 violations — the repo actually had **219 ruff errors across 50 files**. This PR clears all of them. ## Changes Pure import relocations plus a small set of non-PLC lint fixes. No behavior changes. **PLC0415 (import at top-level) — hoisted to top:** safe stdlib/local imports across modules, routers, services, and tasks (json, re, logging, io, base64, hashlib, ipaddress, subprocess, tempfile, Path, shutil, Decimal, SequenceMatcher, sqlmodel/sqlalchemy sessions, `from ..settings import settings`, `from ..models`/`..db`/`..tasks` where circular-safe). **PLC0415 — kept lazy with justified `# noqa` comments** (hoisting would change behavior): - Optional-dependency guarded imports: PIL, pytesseract, pdf2image, pdfplumber, PyPDF2, weasyprint, cv2 — each already wrapped in try/except for graceful degradation; a comment documents the guard. - Circular-import avoiders: `entity_dispatcher.py` (orchestrator_enhanced), `scheduler.py` (queue_manager), `tasks.py` (queue_manager). - Test seam: `queue_manager.py` keeps `from app.db import engine` lazy because `test_process_queue_batch` patches `app.db.engine`; hoisting would bind the real engine at import time and break the test. - Best-effort enrichment guard in `gather.py`. - `registry.py`: file-level `# ruff: noqa: PLC0415` with a docstring note — `register_all_modules()` deliberately imports each optional module in a guarded try/except so a missing optional dep skips that module instead of crashing registration. **Non-PLC fixes:** - `services/execution.py` + `entity_module_map.py`: `str, Enum` → `StrEnum` (UP042). - `retry_handler.py` + `llm/parsing.py`: PEP 695 type-parameter syntax (UP047). - `test_module_benchmarks.py`: justified `# noqa: PT028` (default arg substitutes for parametrize). - Alembic merge migration: removed unused `alembic.op`/`sqlalchemy` imports, `typing.Sequence/Union` → `collections.abc.Sequence` + `X | Y` annotations (UP035/UP007/I001/F401). Revision ids unchanged. - `llm/tools/evidence_extraction.py`: fixed a latent broken relative import (`from ...models` → `from ....models`; the original lazy import resolved to nonexistent `app.services.models` and would have failed at call time). ## Changed files - `backend/alembic/versions/a25b7f06b9f1_merge_organization_fields_and_priority_.py` - `backend/app/change_detector.py` - `backend/app/main.py` - `backend/app/modules/abuseipdb.py` - `backend/app/modules/dark_web_monitor.py` - `backend/app/modules/dashboard_summary.py` - `backend/app/modules/employee_scraper.py` - `backend/app/modules/entity_linker.py` - `backend/app/modules/exif_extractor.py` - `backend/app/modules/google_dorks.py` - `backend/app/modules/local_deep_research.py` - `backend/app/modules/ocr_extractor.py` - `backend/app/modules/osint_playbook.py` - `backend/app/modules/pdf_analyzer.py` - `backend/app/modules/pdf_report.py` - `backend/app/modules/registry.py` - `backend/app/modules/reverse_image_search.py` - `backend/app/modules/search_aggregator.py` - `backend/app/modules/shodan.py` - `backend/app/modules/video_frame_extractor.py` - `backend/app/modules/video_metadata.py` - `backend/app/modules/virustotal.py` - `backend/app/monitoring.py` - `backend/app/routers/enrichment.py` - `backend/app/routers/evidence.py` - `backend/app/routers/gather.py` - `backend/app/routers/investigations.py` - `backend/app/routers/reports.py` - `backend/app/services/browser_tools.py` - `backend/app/services/enrichment_pipeline/runner.py` - `backend/app/services/execution.py` - `backend/app/services/investigations/async_executor.py` - `backend/app/services/investigations/entity_dispatcher.py` - `backend/app/services/investigations/entity_module_map.py` - `backend/app/services/investigations/evidence_dispatcher.py` - `backend/app/services/investigations/llm_entity_extraction.py` - `backend/app/services/investigations/orchestrator.py` - `backend/app/services/investigations/orchestrator_enhanced.py` - `backend/app/services/investigations/orchestrator_v3_llm.py` - `backend/app/services/investigations/queue_manager.py` - `backend/app/services/investigations/retry_handler.py` - `backend/app/services/investigations/runbooks.py` - `backend/app/services/investigations/scheduler.py` - `backend/app/services/llm/parsing.py` - `backend/app/services/llm/tools/evidence_extraction.py` - `backend/app/services/semantic_retrieval.py` - `backend/app/tasks.py` - `backend/scripts/scraper_poc.py` - `backend/test_entity_investigation.py` - `backend/tests/unit/test_module_benchmarks.py` ## Verification - `ruff check .` → **exit 0, All checks passed!** - `ruff check backend/` → **exit 0, All checks passed!** - `ruff check backend/app/` → **exit 0, All checks passed!** (CI lint scope) - `pytest tests/unit` → **945 passed, 1 skipped** - Import smoke test: `app.main`, `app.tasks`, `app.monitoring`, all routers, and all 35 touched modules import cleanly — no circular imports introduced.

Mark added 7 commits 2026-08-13 21:58:38 +00:00

fix(lint): hoist json import in change_detector (closes #97) dda539852b

fix(lint): hoist logging/time imports in main (closes #97) 2047f63407

fix(lint): hoist inline imports in abuseipdb and dark_web_monitor (closes #97) b5812e453d

Merge fix/lint-plc0415-main into fix/lint-plc0415 ad24aef897

Merge fix/lint-plc0415-change-detector into fix/lint-plc0415 708c16c957

Merge fix/lint-plc0415-modules into fix/lint-plc0415 a8d5d89130

fix(lint): resolve remaining PLC0415 + ruff errors so ruff check . exits 0 (refs #97) ... 6a0ee5bc3c

Full-repo cleanup beyond the 5 violations listed in #97 (preflight output was
truncated to 1500 chars; repo actually had 219 ruff errors across 50 files):
- Hoisted safe stdlib/local imports to top-level (no behavior change)
- Left deliberately-guarded lazy imports (optional deps, circular-import
  avoiders, app.db.engine test seam) with justified # noqa: PLC0415
- registry.py: file-level # ruff: noqa: PLC0415 (guarded optional-module
  registration by design)
- Non-PLC fixes: UP042 StrEnum, UP047 PEP695 type params, PT028 noqa,
  alembic merge migration import cleanup
- Fixed latent broken relative import in llm/tools/evidence_extraction.py
  (from ...models -> from ....models; original lazy import would have failed
  at call time)

Verified: ruff check . && ruff check backend/ && ruff check backend/app/ all
exit 0; 945 unit tests pass; app.main/app.tasks/all routers/modules import
cleanly (no circular imports introduced).

Mark referenced this pull request 2026-08-13 21:59:01 +00:00

Fix the lint on osint-hub #97

Mark added the

blocked:preflight

label 2026-08-13 23:02:08 +00:00

Mark referenced this pull request 2026-08-19 20:58:48 +00:00

fix(#101): complete case-deletion dependent-row cleanup under FK enforcement #104

Mark referenced this pull request from a commit 2026-08-24 01:08:00 +00:00

merge: reconcile PLC0415 hoist wave (PR #98) into lint/100-ruff-autofix

Mark referenced this pull request 2026-08-24 01:08:20 +00:00

lint: ruff auto-fix I001 unsorted imports (issue #100 lint debt) #107

Mark closed this pull request 2026-08-26 05:09:27 +00:00

Some checks failed

Hide all checks

CI / Backend Lint & Format (pull_request) Failing after 4s

Details

CI / Golden Fixture Validation (pull_request) Has been skipped

Details

CI / Backend Tests (pull_request) Has been skipped

Details

CI / Backend Security Tests (pull_request) Has been skipped

Details

CI / Frontend Lint (pull_request) Failing after 22s

Details

CI / Frontend Tests (pull_request) Has been skipped

Details

CI / Dependency Security Scan (pull_request) Has been skipped

Details

Performance Regression Tests / Performance Regression Detection (pull_request) Failing after 33s

Details

Stress Tests / Stress & Pipeline Tests (pull_request) Failing after 2m7s

Details

Stress Tests / Full Data Pipeline Validation (pull_request) Has been skipped

Details

CI / Frontend E2E Tests (pull_request) Has been skipped

Details

Performance Regression Tests / Compare with Baseline (pull_request) Has been skipped

Details

CI / All Checks Passed (pull_request) Failing after 48s

Details

Pull request closed

This pull request cannot be reopened because the branch was deleted.

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

agentic-osint

Agentic OSINT features

  

audit

  

blocked:commit-ready-failed

  

blocked:gate-fetch

  

blocked:gate-fetch

  

blocked:gate-lint

  

blocked:gate-plan

  

blocked:gate-research

  

blocked:needs-decision

  

blocked:needs-human

  

blocked:no-changes

  

blocked:preflight

Repo has dirty lint; kilo-gsd preflight refuses to dispatch until lint-cleanup lands

  

blocked:verify-failed

  

bug

Something is not working

  

chore

Config/cleanup item

  

deferred

Deferred — will revisit later

  

enhancement

Enhancement request

  

feature-request

  

in-progress

  

lint-baseline

  

lint-cleanup

kilo-gsd preflight lint gate: repo has dirty ruff state, cleanup required before fixer dispatch

  

needs-decompose

Issue needs decompose processing

  

needs-triage

  

performance

  

quickfix

  

security/medium

Medium severity security issue

  

triage-approved

Approved by triage

No labels

agentic-osint

audit

blocked:commit-ready-failed

blocked:gate-fetch

blocked:gate-fetch

blocked:gate-lint

blocked:gate-plan

blocked:gate-research

blocked:needs-decision

blocked:needs-human

blocked:no-changes

blocked:preflight

blocked:verify-failed

bug

chore

deferred

enhancement

feature-request

in-progress

lint-baseline

lint-cleanup

needs-decompose

needs-triage

performance

quickfix

security/medium

triage-approved

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

Mark/osint-hub!98

No description provided.