kilo-gsd.py plan-discovery loads wrong verify plan; #23 would have been falsely closed #36

New issue

Closed

opened 2026-06-20 20:49:56 +00:00 by Mark · 0 comments

Mark commented

2026-06-20 20:49:56 +00:00

Owner

kilo-gsd.py plan-discovery picks the wrong plan for issue #23

What

Ran kilo-gsd.py --repo Mark/pc2cad --issue 23 on 2026-06-20. The
pipeline loaded the wrong verification plan and was about to close
issue #23 as "complete" without doing the actual OCP port work.
Killed the run before that could happen.

Sequence (from `/tmp/kilo-gsd-runner-pc2cad-23.log`)

Phase 1/8: Research — 13:37:27 → 13:39:29 (~2 min) ✓
Phase 2/8: Plan — 13:39:29 → 13:41:01 (~1.5 min) ✓
Phase 3/8: Implement — 13:41:01 → 13:41:33 (32 seconds)
Phase 4/8: Lint — ruff check . passed
Phase 5/8: Verify — running commands from
pc2cad-16-organic-strategy.md, NOT a plan for #23

The Kilo run during Implement was doing the right thing (grepping
for _freecad_import, export_step, STEPControl in pc2cad.py —
that's the #23 work). But the implement phase produced no diff
to any tracked file (git diff --stat is empty after the run).

The Verify phase started running commands from
pc2cad-16-organic-strategy.md: checking docs/STRATEGY-ORGANIC.md
exists, running scripts/generate_organic_fixtures.py, executing
spikes/organic_surface_reconstruction/spike_organic_fitter.py —
all the acceptance commands for issue #16, not #23.

If left to complete, the pipeline would have closed #23 as
"complete" because all the #16 commands pass.

Suspected root cause

~/.hermes/scripts/kilo-gsd.py line 461:

plan_files = list(plans_dir.glob(f"*{phase_name.lower()}*")) \
             or list(plans_dir.glob("*"))

The verify phase looks for a plan file containing "verify" in its
name. None match (existing plan files in
~/Projects/pc2cad/.kilo/plans/ don't have "verify" in their
names). Falls back to "any plan file." The order it then iterates
isn't documented; in this run, it picked
pc2cad-16-organic-strategy.md instead of generating a verify
plan for #23 (or picking the issue-named plan
pc2cad-23-...md, which doesn't exist).

There is no .kilo/plans/pc2cad-23-*.md file in the repo, so
even with correct discovery, the pipeline has no plan to use. It
should either:

Generate a verify plan from the issue body, or
Refuse to run if no issue-specific plan exists, or
Have a --plan CLI override to point at the right plan.

Reproduction

# Currently broken: issue #23 has no plan file in .kilo/plans/
ls ~/Projects/pc2cad/.kilo/plans/ | grep -E "23|ocp|export-fc"
# (no output — there is no plan for #23)

# Run the pipeline
~/.hermes/scripts/kilo-gsd.py --repo Mark/pc2cad --issue 23
# (will run, but verify phase will pull a wrong plan)

Impact

False positive closure of #23: if left running, the pipeline
would mark the OCP port issue as complete without porting any
code.
Same risk for any issue without a plan file: the 8 untracked
.kilo/plans/*.md files in pc2cad are the only plans. Issues
#11, #12, #13, #14, #15, #17, #18, #21, #22, #23, #25, #28-35
either have no plan or have plans for the wrong scope.
Silent data corruption: the run doesn't write a wrong plan
to the repo, but the log shows it ran the wrong verification
commands and would have declared success.

Suggested fix (not for me to implement)

Either:

Make kilo-gsd.py fail loudly when no issue-specific plan
exists, with a clear error pointing to where the user should
put the plan, OR
Add a --plan <path> CLI flag to point at the right plan file,
OR
Add a Plan phase that generates a verify plan from the issue
body and stores it before the Verify phase runs.

The 8 untracked .kilo/plans/*.md files in pc2cad are the
only plans on disk. None of them is for issue #23.
Killed run details: parent PID 3492155 (orchestrator), tmux
session kilo-gsd-runner-pc2cad-23 (killed), log files at
/tmp/kilo-gsd-pc2cad-23.log and
/tmp/kilo-gsd-runner-pc2cad-23.log (still on disk for
inspection).

# kilo-gsd.py plan-discovery picks the wrong plan for issue #23 ## What Ran `kilo-gsd.py --repo Mark/pc2cad --issue 23` on 2026-06-20. The pipeline loaded the wrong verification plan and was about to close issue #23 as "complete" without doing the actual OCP port work. Killed the run before that could happen. ## Sequence (from `/tmp/kilo-gsd-runner-pc2cad-23.log`) - `Phase 1/8: Research` — 13:37:27 → 13:39:29 (~2 min) ✓ - `Phase 2/8: Plan` — 13:39:29 → 13:41:01 (~1.5 min) ✓ - `Phase 3/8: Implement` — 13:41:01 → 13:41:33 (**32 seconds**) - `Phase 4/8: Lint` — `ruff check .` passed - `Phase 5/8: Verify` — running commands from `pc2cad-16-organic-strategy.md`, NOT a plan for #23 The Kilo run during Implement was doing the right thing (grepping for `_freecad_import`, `export_step`, `STEPControl` in pc2cad.py — that's the #23 work). But the implement phase produced **no diff** to any tracked file (`git diff --stat` is empty after the run). The Verify phase started running commands from `pc2cad-16-organic-strategy.md`: checking `docs/STRATEGY-ORGANIC.md` exists, running `scripts/generate_organic_fixtures.py`, executing `spikes/organic_surface_reconstruction/spike_organic_fitter.py` — all the acceptance commands for issue #16, not #23. If left to complete, the pipeline would have closed #23 as "complete" because all the #16 commands pass. ## Suspected root cause `~/.hermes/scripts/kilo-gsd.py` line 461: ```python plan_files = list(plans_dir.glob(f"*{phase_name.lower()}*")) \ or list(plans_dir.glob("*")) ``` The verify phase looks for a plan file containing "verify" in its name. None match (existing plan files in `~/Projects/pc2cad/.kilo/plans/` don't have "verify" in their names). Falls back to "any plan file." The order it then iterates isn't documented; in this run, it picked `pc2cad-16-organic-strategy.md` instead of generating a verify plan for #23 (or picking the issue-named plan `pc2cad-23-...md`, which doesn't exist). There is no `.kilo/plans/pc2cad-23-*.md` file in the repo, so even with correct discovery, the pipeline has no plan to use. It should either: 1. Generate a verify plan from the issue body, or 2. Refuse to run if no issue-specific plan exists, or 3. Have a `--plan` CLI override to point at the right plan. ## Reproduction ```bash # Currently broken: issue #23 has no plan file in .kilo/plans/ ls ~/Projects/pc2cad/.kilo/plans/ | grep -E "23|ocp|export-fc" # (no output — there is no plan for #23) # Run the pipeline ~/.hermes/scripts/kilo-gsd.py --repo Mark/pc2cad --issue 23 # (will run, but verify phase will pull a wrong plan) ``` ## Impact - **False positive closure of #23:** if left running, the pipeline would mark the OCP port issue as complete without porting any code. - **Same risk for any issue without a plan file:** the 8 untracked `.kilo/plans/*.md` files in pc2cad are the only plans. Issues #11, #12, #13, #14, #15, #17, #18, #21, #22, #23, #25, #28-35 either have no plan or have plans for the wrong scope. - **Silent data corruption:** the run doesn't write a wrong plan to the repo, but the log shows it ran the wrong verification commands and would have declared success. ## Suggested fix (not for me to implement) Either: - Make `kilo-gsd.py` fail loudly when no issue-specific plan exists, with a clear error pointing to where the user should put the plan, OR - Add a `--plan <path>` CLI flag to point at the right plan file, OR - Add a Plan phase that generates a verify plan from the issue body and stores it before the Verify phase runs. ## Related - The 8 untracked `.kilo/plans/*.md` files in pc2cad are the only plans on disk. None of them is for issue #23. - Killed run details: parent PID 3492155 (orchestrator), tmux session `kilo-gsd-runner-pc2cad-23` (killed), log files at `/tmp/kilo-gsd-pc2cad-23.log` and `/tmp/kilo-gsd-runner-pc2cad-23.log` (still on disk for inspection).