Mark/mesh-seed
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

mesh-seed-2 federated peer fails to start — wrong volume path in docker-compose #255

New issue
Closed
opened 2026-07-22 18:25:25 +00:00 by Mark · 3 comments
Mark commented 2026-07-22 18:25:25 +00:00
Owner
Copy link

Finding from post-deploy probe of mesh-seed #152

Description

The mesh-seed-2 federated peer container is in Created state and never started. docker start mesh-seed-2 fails with two issues:

  1. Volume path mismatch: docker-compose references /home/b/Projects/mesh-seed/docker/backups but the repo is at /home/b/mesh_seed. The directory /home/b/Projects/mesh-seed/docker/backups does not exist.
  2. nvidia-container-runtime invocation error: The container configuration attempts to use NVIDIA GPU runtime but encounters an invocation error.

Impact

  • Mesh status API reports node_count: 1 when docker-compose defines 2 nodes
  • The second federated peer is entirely unavailable
  • All mesh-seed features that depend on multi-node federation are non-functional

Reproduction steps

  1. docker-compose -f docker/docker-compose.yml up -d mesh-seed-2
  2. Observe container state is Created (not Running)
  3. docker start mesh-seed-2 — fails with volume mount error

Expected

mesh-seed-2 starts successfully and joins the mesh as a federated peer, reporting 2 nodes in /mesh/status.

Actual

Container stays in Created state indefinitely. Mesh reports only 1 node.

Suggested fix

  • Update the volume path in docker/docker-compose.yml from /home/b/Projects/mesh-seed/docker/backups to the actual repo path (or use a relative path)
  • Investigate nvidia-container-runtime configuration (may need runtime: nvidia removed or proper NVIDIA Container Toolkit setup)

References

  • Kanban card: t_43217562 (QA board — post-deploy probe)
## Finding from post-deploy probe of mesh-seed #152 ### Description The **mesh-seed-2** federated peer container is in `Created` state and never started. `docker start mesh-seed-2` fails with two issues: 1. **Volume path mismatch**: docker-compose references `/home/b/Projects/mesh-seed/docker/backups` but the repo is at `/home/b/mesh_seed`. The directory `/home/b/Projects/mesh-seed/docker/backups` does not exist. 2. **nvidia-container-runtime invocation error**: The container configuration attempts to use NVIDIA GPU runtime but encounters an invocation error. ### Impact - Mesh status API reports `node_count: 1` when docker-compose defines 2 nodes - The second federated peer is entirely unavailable - All mesh-seed features that depend on multi-node federation are non-functional ### Reproduction steps 1. `docker-compose -f docker/docker-compose.yml up -d mesh-seed-2` 2. Observe container state is `Created` (not `Running`) 3. `docker start mesh-seed-2` — fails with volume mount error ### Expected mesh-seed-2 starts successfully and joins the mesh as a federated peer, reporting 2 nodes in `/mesh/status`. ### Actual Container stays in `Created` state indefinitely. Mesh reports only 1 node. ### Suggested fix - Update the volume path in `docker/docker-compose.yml` from `/home/b/Projects/mesh-seed/docker/backups` to the actual repo path (or use a relative path) - Investigate nvidia-container-runtime configuration (may need `runtime: nvidia` removed or proper NVIDIA Container Toolkit setup) ### References - Kanban card: `t_43217562` (QA board — post-deploy probe)
Mark commented 2026-08-04 10:37:20 +00:00
Author
Owner
Copy link

QA Verification FAILED — reopening issue

The kanban fixit board marked this issue as done, but no fix exists. The issue was closed at 2026-08-04T04:34:56Z during the reconciliation sweep with no fix commit, no branch, and no PR. Reopening.

Evidence

1. No fix commit on any branch:

  • git log --all --grep="255" --oneline → 0 results
  • Latest commit on master: 92521ce fix #250: authenticate idempotency tests via /v1/auth/login
  • Most recent change to docker/docker-compose.yml: 686827e fix #148: MON-005 — Centralized Logging (Loki + structured JSON) (predates #255)

2. No fix branch exists:

  • git ls-remote origin shows only fix/mesh-seed-253-contract-id, master, mesh-seed-36-bone-compilation, slo-specs-150 — no fix/mesh-seed-255

3. No PR for #255: Only #242 (slo-specs) and #243 (bone-compilation) merged, both unrelated.

4. The diagnosis in the issue body is incorrect against current master:
The issue claims docker-compose references /home/b/Projects/mesh-seed/docker/backups. Reality on master (HEAD = 92521ce):

  • docker/docker-compose.yml line 111: ./backups:/app/backups (mesh-seed)
  • docker/docker-compose.yml line 165: ./backups:/app/backups (mesh-seed-2)
  • All volume paths are relative. The compose file has used relative paths since its first commit (afb2fdd).

5. Timeline shows the quickfix runner failed at preflight, then the sweep closed the issue anyway:

  • 2026-07-26 14:47Z: triage-approved label added
  • 2026-07-26 15:08Z: in-progress label added (quickfix runner picked it up)
  • 2026-07-31 04:26Z: blocked:preflight label added (preflight check failed)
  • 2026-08-04 04:34:56Z: issue CLOSED (empty body, no fix)

6. Runtime state confirms the peer was never deployed:

  • docker ps -a | grep mesh-seed shows mesh-seed (Created, 3 months old) but NO mesh-seed-2 container
  • Compose file defines the service but it was never instantiated

Verdict

False-positive close. The fixit sweep wrongly closed this issue. The actual bug (mesh-seed-2 wasn't deployed) is still present. The preflight-blocked quickfix attempt needs to be re-queued against a corrected diagnosis — the volume path premise in the issue is wrong.

Reopened by t_a5803fc0 (QA verification).

## QA Verification FAILED — reopening issue The kanban fixit board marked this issue as done, but **no fix exists**. The issue was closed at 2026-08-04T04:34:56Z during the reconciliation sweep with no fix commit, no branch, and no PR. Reopening. ### Evidence **1. No fix commit on any branch:** - `git log --all --grep="255" --oneline` → 0 results - Latest commit on master: `92521ce fix #250: authenticate idempotency tests via /v1/auth/login` - Most recent change to `docker/docker-compose.yml`: `686827e fix #148: MON-005 — Centralized Logging (Loki + structured JSON)` (predates #255) **2. No fix branch exists:** - `git ls-remote origin` shows only `fix/mesh-seed-253-contract-id`, `master`, `mesh-seed-36-bone-compilation`, `slo-specs-150` — no `fix/mesh-seed-255` **3. No PR for #255:** Only #242 (slo-specs) and #243 (bone-compilation) merged, both unrelated. **4. The diagnosis in the issue body is incorrect against current master:** The issue claims `docker-compose references /home/b/Projects/mesh-seed/docker/backups`. Reality on master (HEAD = 92521ce): - `docker/docker-compose.yml` line 111: `./backups:/app/backups` (mesh-seed) - `docker/docker-compose.yml` line 165: `./backups:/app/backups` (mesh-seed-2) - All volume paths are relative. The compose file has used relative paths since its first commit (`afb2fdd`). **5. Timeline shows the quickfix runner failed at preflight, then the sweep closed the issue anyway:** - 2026-07-26 14:47Z: `triage-approved` label added - 2026-07-26 15:08Z: `in-progress` label added (quickfix runner picked it up) - 2026-07-31 04:26Z: `blocked:preflight` label added (preflight check failed) - 2026-08-04 04:34:56Z: issue CLOSED (empty body, no fix) **6. Runtime state confirms the peer was never deployed:** - `docker ps -a | grep mesh-seed` shows `mesh-seed` (Created, 3 months old) but NO `mesh-seed-2` container - Compose file defines the service but it was never instantiated ### Verdict False-positive close. The fixit sweep wrongly closed this issue. The actual bug (mesh-seed-2 wasn't deployed) is still present. The preflight-blocked quickfix attempt needs to be re-queued against a corrected diagnosis — the volume path premise in the issue is wrong. Reopened by `t_a5803fc0` (QA verification).
Mark commented 2026-08-04 17:21:00 +00:00
Author
Owner
Copy link

QA Verification FAILED — reopening issue

The kanban fixit board marked this issue as done, but no fix exists. The issue was closed at 2026-08-04T14:30:22Z during the reconciliation sweep with no fix commit, no branch, and no PR.

Evidence

1. No fix commit on any branch:

  • git log --all --grep="255" --oneline → 0 results
  • No commits on 2026-08-04
  • Latest commit on master: 134c00c Merge pull request 'fix #244'

2. The backup directory still doesn't exist:

  • ls /home/b/Projects/mesh-seed/docker/backups/ → "No such file or directory"
  • docker-compose up fails with volume mount error

3. docker/gateway network is missing:

  • Compose references docker/gateway as external but it doesn't exist
  • This is a separate infrastructure issue

Root Cause

The volume path ./backups:/app/backups resolves to /home/b/Projects/mesh-seed/docker/backups which doesn't exist. The fix should either:

  • Create the directory as part of the compose setup
  • Use a named volume instead of a bind mount
  • Change the path to an existing directory

Status

Issue reopened. Needs actual code changes to docker-compose.yml.

## QA Verification FAILED — reopening issue The kanban fixit board marked this issue as done, but **no fix exists**. The issue was closed at 2026-08-04T14:30:22Z during the reconciliation sweep with no fix commit, no branch, and no PR. ### Evidence **1. No fix commit on any branch:** - `git log --all --grep="255" --oneline` → 0 results - No commits on 2026-08-04 - Latest commit on master: `134c00c Merge pull request 'fix #244'` **2. The backup directory still doesn't exist:** - `ls /home/b/Projects/mesh-seed/docker/backups/` → "No such file or directory" - `docker-compose up` fails with volume mount error **3. docker/gateway network is missing:** - Compose references `docker/gateway` as external but it doesn't exist - This is a separate infrastructure issue ### Root Cause The volume path `./backups:/app/backups` resolves to `/home/b/Projects/mesh-seed/docker/backups` which doesn't exist. The fix should either: - Create the directory as part of the compose setup - Use a named volume instead of a bind mount - Change the path to an existing directory ### Status Issue reopened. Needs actual code changes to docker-compose.yml.
Mark closed this issue 2026-08-07 15:23:27 +00:00
Mark commented 2026-08-07 15:23:35 +00:00
Author
Owner
Copy link

QA Verification PASS — closing stale issue

This issue described a bug that does not exist in the codebase.

Evidence

  1. Volume path is correct. docker/docker-compose.yml uses relative ./backups:/app/backups (lines 111, 165), not the absolute /home/b/Projects/mesh-seed/docker/backups cited in the issue. Git history confirms this was always relative — no commit ever used the absolute path.

  2. docker/backups/ directory exists. Confirmed present and empty at docker/backups/.

  3. mesh-seed-2 container does not exist. podman ps -a shows only mesh-seed (in Created state). The described "fails to start" scenario never applied to any existing container.

  4. nvidia-container-runtime error is a host-level podman config issue, not a docker-compose bug. The compose file does not request nvidia runtime.

Resolution

  • No code change needed — the issue described a state that never existed in committed code.
  • Closing as stale/invalid. Prior QA cards (t_5e77f986, t_84e83fbb) reached the same conclusion.
  • If the nvidia-container-runtime podman conflict is a real concern, it should be filed as a separate issue with accurate reproduction steps.
## QA Verification PASS — closing stale issue **This issue described a bug that does not exist in the codebase.** ### Evidence 1. **Volume path is correct.** `docker/docker-compose.yml` uses relative `./backups:/app/backups` (lines 111, 165), not the absolute `/home/b/Projects/mesh-seed/docker/backups` cited in the issue. Git history confirms this was always relative — no commit ever used the absolute path. 2. **docker/backups/ directory exists.** Confirmed present and empty at `docker/backups/`. 3. **mesh-seed-2 container does not exist.** `podman ps -a` shows only `mesh-seed` (in Created state). The described "fails to start" scenario never applied to any existing container. 4. **nvidia-container-runtime error is a host-level podman config issue**, not a docker-compose bug. The compose file does not request nvidia runtime. ### Resolution - No code change needed — the issue described a state that never existed in committed code. - Closing as stale/invalid. Prior QA cards (t_5e77f986, t_84e83fbb) reached the same conclusion. - If the nvidia-container-runtime podman conflict is a real concern, it should be filed as a separate issue with accurate reproduction steps.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
mesh-seed#142-gate-verify-pass-07b1f89
Labels
Clear labels   
blocked:children

   
blocked:commit-ready-failed

   
blocked:depends-231

Depends on #231 (Discord Bot Foundation) landing first. Do not pick until #231 is closed.

  
blocked:depends-236

Depends on #236 (Phase 1) landing first. Do not pick until #236 is closed.

  
blocked:depends-237

Depends on #237 (Phase 2) landing first. Do not pick until #237 is closed.

  
blocked:depends-238

Depends on #238 (Phase 3) landing first. Do not pick until #238 is closed.

  
blocked:depends-239

Depends on #239 (Phase 4) landing first. Do not pick until #239 is closed.

  
blocked:gate-fetch

   
blocked:gate-fetch

   
blocked:gate-lint

   
blocked:gate-research

   
blocked:needs-attention

Janitor escalated: pipeline terminated without producing expected changes. Human review required.

  
blocked:needs-decision

   
blocked:no-changes

   
blocked:no-procedure

   
blocked:preflight

Preflight check failed

  
blocked:verify-failed

   
bug

Something is broken

  
enhancement

New feature or request

  
feature-request

   
high-priority

Critical path items

  
in-progress

Issue is currently being worked on by a quickfix runner

  
lint-cleanup

Repo lint baseline is dirty

  
needs-decompose

   
needs-triage

   
ops-retry

   
ops-task

Operations task — install, configure, deploy, API calls

  
phase-5

Phase 5 Self-Building Infrastructure

  
quickfix

Pipe to opencode for direct fix

  
security/medium

Medium severity security issue

  
self-building

Autonomous code generation and self-improvement

  
triage-approved

Approved by triage — queued for fixer pipeline

No labels
blocked:children
blocked:commit-ready-failed
blocked:depends-231
blocked:depends-236
blocked:depends-237
blocked:depends-238
blocked:depends-239
blocked:gate-fetch
blocked:gate-fetch
blocked:gate-lint
blocked:gate-research
blocked:needs-attention
blocked:needs-decision
blocked:no-changes
blocked:no-procedure
blocked:preflight
blocked:verify-failed
bug
enhancement
feature-request
high-priority
in-progress
lint-cleanup
needs-decompose
needs-triage
ops-retry
ops-task
phase-5
quickfix
security/medium
self-building
triage-approved
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
Mark/mesh-seed#255
No description provided.