LLM Upgrade: intent_router semantic routing decisions

Mark commented

2026-04-05 06:30:46 +00:00

Owner

Problem

intent_router uses hardcoded routing:

if intent_type in GPU_INTENT_TYPES: ...
# Bone fast-path based on exact type match
# RoutePlanner invoked based on heuristics

Misses semantic routing opportunities:

Understanding intent nuance (fast vs comprehensive)
Worker specialization (expert vs generalist)
Cost-quality tradeoffs (expensive worker for complex intent)

Solution

Add LLM-assisted routing for ambiguous intents:

Semantic routing: When routing is ambiguous:
- Multiple workers match with similar scores
- Intent has nuance (depth, speed, quality preferences)
- User preference hints in payload (budget, deadline, quality_tier)
LLM query: "Given intent '{intent}' with payload '{payload}', which worker is best? Consider: cost ${budget}, deadline {deadline}s, quality tier '{quality_tier}'. Return worker_id with reasoning."
Gate logic: Use LLM routing when:
- RoutePlanner returns multiple candidates with similar confidence
- Intent payload has quality/budget/deadline constraints
- User preference indicators present
- Heuristic routing score difference < threshold
Integration:
- Add _route_semantic() method to intent_router
- Call after RoutePlanner resolves candidates
- Fallback to score-based selection

Files to Modify

runtime/intent_router.py - add _route_semantic(), integrate with existing routing
Update routing flow: bones → RoutePlanner → semantic selection

Acceptance Criteria

Semantic routing considers user preferences (budget, deadline, quality)
Chooses specialist worker for complex intents
Chooses cheap bone for simple intents
Fallback to score-based routing when LLM unavailable
Routing decisions logged with reasoning

## Problem `intent_router` uses hardcoded routing: ```python if intent_type in GPU_INTENT_TYPES: ... # Bone fast-path based on exact type match # RoutePlanner invoked based on heuristics ``` Misses semantic routing opportunities: - Understanding intent nuance (fast vs comprehensive) - Worker specialization (expert vs generalist) - Cost-quality tradeoffs (expensive worker for complex intent) ## Solution Add LLM-assisted routing for ambiguous intents: 1. **Semantic routing**: When routing is ambiguous: - Multiple workers match with similar scores - Intent has nuance (depth, speed, quality preferences) - User preference hints in payload (budget, deadline, quality_tier) 2. **LLM query**: "Given intent '{intent}' with payload '{payload}', which worker is best? Consider: cost ${budget}, deadline {deadline}s, quality tier '{quality_tier}'. Return worker_id with reasoning." 3. **Gate logic**: Use LLM routing when: - RoutePlanner returns multiple candidates with similar confidence - Intent payload has quality/budget/deadline constraints - User preference indicators present - Heuristic routing score difference < threshold 4. **Integration**: - Add `_route_semantic()` method to intent_router - Call after RoutePlanner resolves candidates - Fallback to score-based selection ## Files to Modify - `runtime/intent_router.py` - add `_route_semantic()`, integrate with existing routing - Update routing flow: bones → RoutePlanner → semantic selection ## Acceptance Criteria - [ ] Semantic routing considers user preferences (budget, deadline, quality) - [ ] Chooses specialist worker for complex intents - [ ] Chooses cheap bone for simple intents - [ ] Fallback to score-based routing when LLM unavailable - [ ] Routing decisions logged with reasoning

Mark added the

feature-request

label

2026-05-12 23:26:59 +00:00

Mark added

blocked:needs-decision

and removed

feature-request

labels

2026-05-21 02:40:05 +00:00

Mark removed the

blocked:needs-decision

label

2026-05-28 07:19:14 +00:00

Mark added the

blocked:needs-decision

label

2026-06-07 05:57:18 +00:00

Mark commented

2026-06-07 05:57:18 +00:00

Author

Owner

⏸ Needs Decision

What needs to be decided?

Is this issue still relevant? If yes, remove the blocked:needs-decision label to route back to triage.

Blocking factors:

Didn't meet auto-approve criteria — scope=small, risk=low, spec=partial

Scope: small Risk: low Spec: partial Type: enhancement
Summary: Adds LLM-assisted semantic routing to intent_router for ambiguous routing decisions using user preference hints (budget, deadline, quality_tier).
Suggestion: Implement _route_semantic() as a drop-in layer after RoutePlanner with strict fallback to score-based when LLM unavailable. Gate on confidence gap < threshold or presence of preference hints. However, note tension with #121 (remove LLM calls from infrastructure) — consider routing the ambiguity to a lightweight worker that makes the call instead of embedding the LLM call directly in intent_router.
Rationale: Scope is small: single-file additive change (add _route_semantic to runtime/intent_router.py, gate logic, logging). Risk is low: existing bones→RoutePlanner path untouched, fallback to score-based selection guarantees no regression. Spec alignment is partial: routing decisions belong to infrastructure per AGENTS.md ('The substrate routes'), so the concept is aligned, but adding an LLM call to infrastructure conflicts with active effort #121 to remove LLM calls from infrastructure components. The suggestion mitigates this by proposing a worker-based delegation pattern.

To unblock: Remove the blocked:needs-decision label. The triage will apply the appropriate outcome based on your choice above.

## ⏸ Needs Decision **What needs to be decided?** - **Is this issue still relevant?** If yes, remove the `blocked:needs-decision` label to route back to triage. **Blocking factors:** - **Didn't meet auto-approve criteria** — scope=small, risk=low, spec=partial --- **Scope:** small **Risk:** low **Spec:** partial **Type:** enhancement **Summary:** Adds LLM-assisted semantic routing to intent_router for ambiguous routing decisions using user preference hints (budget, deadline, quality_tier). **Suggestion:** Implement `_route_semantic()` as a drop-in layer after RoutePlanner with strict fallback to score-based when LLM unavailable. Gate on confidence gap < threshold or presence of preference hints. However, note tension with #121 (remove LLM calls from infrastructure) — consider routing the ambiguity to a lightweight worker that makes the call instead of embedding the LLM call directly in intent_router. **Rationale:** Scope is small: single-file additive change (add _route_semantic to runtime/intent_router.py, gate logic, logging). Risk is low: existing bones→RoutePlanner path untouched, fallback to score-based selection guarantees no regression. Spec alignment is partial: routing decisions belong to infrastructure per AGENTS.md ('The substrate routes'), so the concept is aligned, but adding an LLM call to infrastructure conflicts with active effort #121 to remove LLM calls from infrastructure components. The suggestion mitigates this by proposing a worker-based delegation pattern. **To unblock:** Remove the `blocked:needs-decision` label. The triage will apply the appropriate outcome based on your choice above.

Mark added the

blocked:preflight

label

2026-07-31 04:26:46 +00:00

Mark commented

2026-08-11 04:32:36 +00:00

Author

Owner

Approved — removing blocked:needs-decision. Triage failed to auto-approve, but this is low/medium risk, aligned.

Mark

2026-08-11 04:33:17 +00:00