backend CI swallows all test failures with || true #94
Labels
No labels
agentic-osint
audit
blocked:commit-ready-failed
blocked:gate-fetch
blocked:gate-fetch
blocked:gate-lint
blocked:gate-plan
blocked:gate-research
blocked:needs-decision
blocked:needs-human
blocked:no-changes
blocked:preflight
blocked:verify-failed
bug
chore
deferred
enhancement
feature-request
in-progress
lint-baseline
lint-cleanup
needs-decompose
needs-triage
performance
quickfix
security/medium
triage-approved
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
Mark/osint-hub#94
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
The backend-test job in .forgejo/workflows/ci.yml runs pytest with
|| trueat the end, so any test failure is swallowed and CI always passes. As ofd9bbe4dthere are 73 pre-existing failures in the integration/security/performance suites (rate-limit tests expecting 429 get 403, parse-evidence creates 0 entities, 404-format tests, etc.) and nobody notices. Remove the|| true(or make it a separate non-gating job) so broken tests actually fail CI.Fixed in
9a07880. Root cause was two-fold: (1) the backend-test job ran the fast suite, fell back to the FULL suite, and appended|| trueso every failure was swallowed; (2) test files under tests/integration/, tests/security/, and tests/performance/ never declared pytestmark, so the-m "not integration and not performance and not security"filter did not exclude them — they ran inside the "fast" suite and failed. Added pytestmark to all 17 unmarked files, made the fast unit suite the gating step, and moved the full suite + security suite to continue-on-error (non-gating) so failures are visible without blocking until #95 is fixed. Fast suite now passes 1297/1297 locally with 487 correctly deselected.fixup verification: issue already fixed on main in
9a078807(verified via API). Gating step in .forgejo/workflows/ci.yml now runspytest tests/ -v --tb=short --cov=app --cov-report=xml --cov-report=term-missing -m "not integration and not performance and not security"WITHOUT|| true, so broken unit tests fail CI. Full suite moved to continue-on-error (non-gating) per the issue's suggested alternative. All 17 unmarked integration/security/performance files got pytestmark so the -m filter excludes them. Closing.