73 pre-existing integration/security/performance test failures on main #95

New issue

Closed

opened 2026-08-11 05:08:51 +00:00 by Mark · 1 comment

Mark commented

2026-08-11 05:08:51 +00:00

Owner

Running pytest tests/ -m "not integration and not performance and not security" on pristine main fails 73 tests. Clusters: 13 in integration/test_integration_concurrent, 11 integration/test_integration_module_flow, 10 integration/test_integration_resilience, 9 integration/test_api_entities, 7 security/test_security_injection, 6 integration/test_api_errors, 3 security/test_security_auth, plus parse-evidence creating 0 entities and rate-limit tests getting 403 instead of 429. Many look like the same case-access-control drift (tests never updated after dddb058). These are hidden by the CI || true bug (#94).

Running `pytest tests/ -m "not integration and not performance and not security"` on pristine main fails 73 tests. Clusters: 13 in integration/test_integration_concurrent, 11 integration/test_integration_module_flow, 10 integration/test_integration_resilience, 9 integration/test_api_entities, 7 security/test_security_injection, 6 integration/test_api_errors, 3 security/test_security_auth, plus parse-evidence creating 0 entities and rate-limit tests getting 403 instead of 429. Many look like the same case-access-control drift (tests never updated after dddb058). These are hidden by the CI || true bug (#94).

Mark added the

bug

label

2026-08-11 05:08:51 +00:00

Mark referenced this issue from a commit

2026-08-11 06:37:29 +00:00

fix: alembic multiple-heads merge, CI test gating, parse-evidence domain persistence

Mark referenced this issue

2026-08-11 06:38:35 +00:00

backend CI swallows all test failures with || true #94

Mark commented

2026-08-11 06:39:21 +00:00

Author

Owner

Progress update from 9a07880: the "73 pre-existing failures" were largely caused by missing pytestmark on the integration/security/performance test files, which made them run inside the fast suite and fail (then get swallowed by the CI || true). All 17 files now declare their marker, so the fast unit suite passes 1297/1297 with 487 correctly deselected. The remaining pre-existing failures in the integration/security/performance suites are still real and now surface in CI as non-gating diagnostics. Also fixed two stale fast-suite tests that were failing for schema-drift reasons: test_org_dedup referenced the removed Organization.domains attribute (domains now live on OrganizationDomain child rows), and the parse-evidence mocks returned the pre-refactor flat LLM shape instead of the unified {"entities": [...]} schema.

Progress update from 9a07880: the "73 pre-existing failures" were largely caused by missing pytestmark on the integration/security/performance test files, which made them run inside the fast suite and fail (then get swallowed by the CI || true). All 17 files now declare their marker, so the fast unit suite passes 1297/1297 with 487 correctly deselected. The remaining pre-existing failures in the integration/security/performance suites are still real and now surface in CI as non-gating diagnostics. Also fixed two stale fast-suite tests that were failing for schema-drift reasons: test_org_dedup referenced the removed Organization.domains attribute (domains now live on OrganizationDomain child rows), and the parse-evidence mocks returned the pre-refactor flat LLM shape instead of the unified {"entities": [...]} schema.

Mark added the

triage-approved

label

2026-08-12 03:49:05 +00:00

Mark

2026-08-13 10:32:34 +00:00