BUG: Mesh accepts intents for which no workers exist - fails silently with cryptic error #118

Closed
opened 2026-04-10 23:48:31 +00:00 by Mark · 21 comments
Owner

Bug Description

The mesh accepts intent submissions via HTTP API even when no workers are registered to handle that intent type. The intent enters the system, goes through the full routing/bidding process, and only fails at execution time with an unhelpful error message.

Expected Behavior

Option A (Fail Fast): Gateway should reject intent submission with HTTP 400/422 if no workers exist for the intent type.

Option B (Clear Error): If accepted, the error should clearly state "No workers registered for capability 'write_article'" rather than the cryptic "No valid capabilities for worker".

Actual Behavior

  1. Intent submitted successfully via HTTP POST to /intent
  2. Gateway returns 202 Accepted with intent_id
  3. Client polls status endpoint
  4. Status returns "completed" but with error: {"error": "No valid capabilities for worker", "status": "failed"}
  5. No indication to user that the mesh cannot actually execute the work

Reproduction Steps

# 1. Start fresh mesh with Docker Compose
# (docker-compose.test.yml from tests/e2e/docker_trump_essay/)
docker-compose -f docker-compose.test.yml up -d

# 2. Submit intent for which no workers exist
curl -X POST http://localhost:8081/intent \
  -H "Content-Type: application/json" \
  -d '{
    "type": "write_article",
    "payload": {"topic": "test", "word_count": 100},
    "budget": {"max_cost_usd": 0.50, "max_latency_ms": 30000}
  }'

# 3. Returns: {"intent_id": "...", "status": "accepted"}

# 4. Poll for result
curl http://localhost:8081/intent/{intent_id}/status

# 5. Returns: {"status": "completed", "result": {"error": "No valid capabilities for worker"}}

Logs

From test run (2026-04-10):

[16:37:30.705] ✅ Intent submitted
[16:37:30.717] Status: submitted (poll #1)
[16:37:32.728] ✅ Intent completed
{
  "result": {
    "error": "No valid capabilities for worker",
    "status": "failed"
  }
}

Root Cause Analysis

The mesh has multiple layers that should catch this:

  1. Gateway (api_gateway.py) - Validates intent format but doesn't check if any workers can handle it
  2. mesh_node bidding (_bid_on_intent) - Tries to bid but finds no eligible workers
  3. Intent Router - Should route to appropriate workers but fails silently

The error message "No valid capabilities for worker" comes from the capability loader or worker execution layer, but by then the intent has already been accepted and resources wasted.

Impact

  • User Experience: Confusing - intent is "accepted" but fails at completion
  • Resource Waste: Bond escrow, routing overhead, polling all wasted
  • Debugging Difficulty: Error message doesn't indicate root cause (no workers registered)

Possible Solutions

Add validation in api_gateway.py submit_intent():

# Check if any workers can handle this intent type
available_capabilities = get_available_capabilities()
if intent_type not in available_capabilities:
    return jsonify({
        "error": f"No workers registered for intent type '{intent_type}'",
        "available_types": list(available_capabilities.keys())
    }), 422

Solution 2: mesh_node Early Rejection

In mesh_node._bid_on_intent():

  • If no workers available for intent type
  • Emit intent.rejected immediately with clear reason
  • Update ledger with rejection reason

Solution 3: Better Error Messages

At minimum, change error from:

  • ❌ "No valid capabilities for worker"
  • ✅ "No workers registered for capability 'write_article'. Available: [list]"
  • Worker registration/provisioning may also be broken (separate issue)
  • This bug masks whether workers exist but fail vs no workers registered

Environment

  • Docker Compose setup from tests/e2e/docker_trump_essay/
  • Podman 4.9.3
  • Redis 7-alpine
  • Gateway and mesh-node from latest docker images

Severity

Medium-High - Breaks basic functionality, wastes resources, poor UX

## Bug Description The mesh accepts intent submissions via HTTP API even when no workers are registered to handle that intent type. The intent enters the system, goes through the full routing/bidding process, and only fails at execution time with an unhelpful error message. ## Expected Behavior **Option A (Fail Fast):** Gateway should reject intent submission with HTTP 400/422 if no workers exist for the intent type. **Option B (Clear Error):** If accepted, the error should clearly state "No workers registered for capability 'write_article'" rather than the cryptic "No valid capabilities for worker". ## Actual Behavior 1. Intent submitted successfully via HTTP POST to `/intent` 2. Gateway returns 202 Accepted with intent_id 3. Client polls status endpoint 4. Status returns "completed" but with error: `{"error": "No valid capabilities for worker", "status": "failed"}` 5. No indication to user that the mesh cannot actually execute the work ## Reproduction Steps ```bash # 1. Start fresh mesh with Docker Compose # (docker-compose.test.yml from tests/e2e/docker_trump_essay/) docker-compose -f docker-compose.test.yml up -d # 2. Submit intent for which no workers exist curl -X POST http://localhost:8081/intent \ -H "Content-Type: application/json" \ -d '{ "type": "write_article", "payload": {"topic": "test", "word_count": 100}, "budget": {"max_cost_usd": 0.50, "max_latency_ms": 30000} }' # 3. Returns: {"intent_id": "...", "status": "accepted"} # 4. Poll for result curl http://localhost:8081/intent/{intent_id}/status # 5. Returns: {"status": "completed", "result": {"error": "No valid capabilities for worker"}} ``` ## Logs From test run (2026-04-10): ``` [16:37:30.705] ✅ Intent submitted [16:37:30.717] Status: submitted (poll #1) [16:37:32.728] ✅ Intent completed { "result": { "error": "No valid capabilities for worker", "status": "failed" } } ``` ## Root Cause Analysis The mesh has multiple layers that should catch this: 1. **Gateway (`api_gateway.py`)** - Validates intent format but doesn't check if any workers can handle it 2. **mesh_node bidding (`_bid_on_intent`)** - Tries to bid but finds no eligible workers 3. **Intent Router** - Should route to appropriate workers but fails silently The error message "No valid capabilities for worker" comes from the capability loader or worker execution layer, but by then the intent has already been accepted and resources wasted. ## Impact - **User Experience:** Confusing - intent is "accepted" but fails at completion - **Resource Waste:** Bond escrow, routing overhead, polling all wasted - **Debugging Difficulty:** Error message doesn't indicate root cause (no workers registered) ## Possible Solutions ### Solution 1: Gateway Validation (Recommended) Add validation in `api_gateway.py` `submit_intent()`: ```python # Check if any workers can handle this intent type available_capabilities = get_available_capabilities() if intent_type not in available_capabilities: return jsonify({ "error": f"No workers registered for intent type '{intent_type}'", "available_types": list(available_capabilities.keys()) }), 422 ``` ### Solution 2: mesh_node Early Rejection In `mesh_node._bid_on_intent()`: - If no workers available for intent type - Emit `intent.rejected` immediately with clear reason - Update ledger with rejection reason ### Solution 3: Better Error Messages At minimum, change error from: - ❌ `"No valid capabilities for worker"` - ✅ `"No workers registered for capability 'write_article'. Available: [list]"` ## Related Issues - Worker registration/provisioning may also be broken (separate issue) - This bug masks whether workers exist but fail vs no workers registered ## Environment - Docker Compose setup from `tests/e2e/docker_trump_essay/` - Podman 4.9.3 - Redis 7-alpine - Gateway and mesh-node from latest docker images ## Severity **Medium-High** - Breaks basic functionality, wastes resources, poor UX
Author
Owner

🤖 [quickfix-mesh-seed-118] still working — overseer retry

🤖 [`quickfix-mesh-seed-118`] still working — overseer retry
Author
Owner

🤖 [quickfix-mesh-seed-118] still working — overseer retry

🤖 [`quickfix-mesh-seed-118`] still working — overseer retry
Author
Owner

🤖 [quickfix-mesh-seed-118] still working — overseer retry

🤖 [`quickfix-mesh-seed-118`] still working — overseer retry
Author
Owner

🤖 [quickfix-mesh-seed-118] still working — overseer retry

🤖 [`quickfix-mesh-seed-118`] still working — overseer retry
Author
Owner

🤖 [quickfix-mesh-seed-118] still working — overseer retry

🤖 [`quickfix-mesh-seed-118`] still working — overseer retry
Author
Owner

🤖 [quickfix-mesh-seed-118] still working — overseer retry

🤖 [`quickfix-mesh-seed-118`] still working — overseer retry
Author
Owner

🤖 [quickfix-mesh-seed-118] still working — overseer retry

🤖 [`quickfix-mesh-seed-118`] still working — overseer retry
Author
Owner

🤖 [quickfix-mesh-seed-118] still working — overseer retry

🤖 [`quickfix-mesh-seed-118`] still working — overseer retry
Author
Owner

🤖 [quickfix-mesh-seed-118] still working — overseer retry

🤖 [`quickfix-mesh-seed-118`] still working — overseer retry
Author
Owner

🤖 [quickfix-mesh-seed-118] still working — overseer retry

🤖 [`quickfix-mesh-seed-118`] still working — overseer retry
Author
Owner

🤖 [quickfix-mesh-seed-118] still working — overseer retry

🤖 [`quickfix-mesh-seed-118`] still working — overseer retry
Author
Owner

🤖 [quickfix-mesh-seed-118] still working — overseer retry

🤖 [`quickfix-mesh-seed-118`] still working — overseer retry
Author
Owner

🤖 [quickfix-mesh-seed-118] still working — overseer retry

🤖 [`quickfix-mesh-seed-118`] still working — overseer retry
Author
Owner

🤖 [quickfix-mesh-seed-118] still working — overseer retry

🤖 [`quickfix-mesh-seed-118`] still working — overseer retry
Author
Owner

🤖 [quickfix-mesh-seed-118] still working — overseer retry

🤖 [`quickfix-mesh-seed-118`] still working — overseer retry
Author
Owner

🤖 [quickfix-mesh-seed-118] still working — overseer retry

🤖 [`quickfix-mesh-seed-118`] still working — overseer retry
Author
Owner

🤖 [quickfix-mesh-seed-118] still working — overseer retry

🤖 [`quickfix-mesh-seed-118`] still working — overseer retry
Author
Owner

🤖 [quickfix-mesh-seed-118] still working — overseer retry

🤖 [`quickfix-mesh-seed-118`] still working — overseer retry
Author
Owner

🤖 [quickfix-mesh-seed-118] still working — overseer retry

🤖 [`quickfix-mesh-seed-118`] still working — overseer retry
Author
Owner

🤖 [quickfix-mesh-seed-118] still working — overseer retry

🤖 [`quickfix-mesh-seed-118`] still working — overseer retry
Author
Owner

🤖 [quickfix-mesh-seed-118] still working — overseer retry

🤖 [`quickfix-mesh-seed-118`] still working — overseer retry
Mark 2026-05-27 20:12:35 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
Mark/mesh-seed#118
No description provided.