CONV-002: Multi-turn Context Management

Mark commented

2026-06-16 02:48:53 +00:00

Author

Owner

Research Analysis Complete

Systematic investigation of the codebase produced a structured analysis covering:

1. Current Architecture

Intent flow: External Input → API Gateway → intent.discovered (Redis backplane) → mesh_node._on_intent_discovered() → BidManager.bid() → WorkerSpawner.spawn() → EphemeralWorker → PrimitiveLoader.execute() → LLM call → result.committed
Key finding: Intents are completely isolated. Each intent carries only id, type, payload, budget. No thread_id, conversation_id, or parent_intent exists in the flow.
Worker execution: EphemeralWorker runs as subprocess.Popen, receives intent via INTENT_ID/INTENT_PAYLOAD env vars, calls llm_complete(prompt) with a single string (no message array).
Schema gap: intents table has no thread_id, conversation_id, parent_intent_id, turn_number.

2. Missing Components

Component	Current State	Required
Context Retrieval	None	Fetch previous N turns, format for LLM, token budget, relevance filtering
Context Injection	None	Prepend context to prompt before LLM call
Token Budget Management	None	Respect context window (e.g., 2000 tokens for history)
Relevance Filtering	None	Semantic similarity (embedding) or recency-based
Worker API	None	Declare multi-turn support, receive context via env/API, update thread state
Thread Tracking	None	`thread_id` in intent payload, ledger, backplane messages

3. Recommended Implementation (5 phases)

Data Model: Migrate intents table with thread_id, parent_intent_id, turn_number; add IntentRepository.get_thread_context() and get_by_thread()
Intent Submission & Routing: API Gateway accepts optional thread_id; Mesh Node extracts and passes to WorkerSpawner.spawn()
Worker Context Propagation: WorkerSpawner passes THREAD_ID, THREAD_CONTEXT_JSON env vars; EphemeralWorker parses and retrieves
Context Injection & LLM Integration: Format context + intent into prompt; PrimitiveExecution.build_*_prompt() accepts context; LLMRegistry adds llm_chat_complete(messages[]) for conversation format
Worker Declaration & Thread Updates: Capability template adds supports_multi_turn: true; worker emits thread.updated on completion

4. Key Design Decisions

Thread ID source: Server-generated (UUID) if not provided; client can pass for continuity
Context retrieval: Hybrid — last 3 turns + semantic top-K from older history (via embeddings)
Token budget: Configurable per capability in template.yaml; default 2000
Context format: Structured messages array [{"role": "user", "content": "..."}, ...]
Worker opt-in: Capability declares supports_multi_turn in template.yaml

5. Integration Points

Layer	File	Change
Schema	`substrate/ledger/schema.py`	Add thread columns to `intents`
Repository	`substrate/ledger/repositories/intent.py`	Add thread context methods
API Gateway	`docker/api_gateway.py`	Accept `thread_id` in `/intent` POST
Mesh Node	`docker/mesh_node.py`	Track thread context, pass to spawner
Worker Spawner	`docker/worker_spawner.py`	Add `THREAD_ID`, `THREAD_CONTEXT` env vars
Ephemeral Worker	`docker/ephemeral_worker.py`	Read thread env vars, inject context
Primitive Execution	`runtime/primitive_execution.py`	`build_*_prompt()` accepts context
LLM Registry	`substrate/llm_registry.py`	Add `llm_chat_complete(messages)`

6. Dependencies & Risks

CONV-001 (Conversation Interface): Defines HTTP/WebSocket API for thread management
EMBED-004 (Conversational Memory): Embedding-based relevance filtering
#123 (No embedding infrastructure): Blocks semantic relevance; literal matching only currently
Risk: Schema migration breaking existing deployments — mitigate with DEFAULT '' columns and ALTER TABLE in SchemaManager._add_column_if_missing()
Risk: Context exceeds LLM context window — enforce max_tokens in retrieval; truncate oldest first

7. Test Strategy

Unit: IntentRepository.get_thread_context() with mocked ledger
Integration: API Gateway → Mesh Node → Worker Spawner → Ephemeral Worker with thread_id
E2E: Multi-turn conversation via API (Turn 1 → Turn 2 referencing Turn 1 → Turn 3)
Architecture: Verify worker receives context via env vars (not direct ledger access)

Full analysis log: /tmp/kilo-gsd-mesh-seed-132-research.log

Closing this research ticket — implementation can be tracked as follow-up issues.

## Research Analysis Complete Systematic investigation of the codebase produced a structured analysis covering: ### 1. Current Architecture - Intent flow: External Input → API Gateway → `intent.discovered` (Redis backplane) → `mesh_node._on_intent_discovered()` → `BidManager.bid()` → `WorkerSpawner.spawn()` → `EphemeralWorker` → `PrimitiveLoader.execute()` → LLM call → `result.committed` - **Key finding**: Intents are completely isolated. Each intent carries only `id`, `type`, `payload`, `budget`. No `thread_id`, `conversation_id`, or `parent_intent` exists in the flow. - Worker execution: `EphemeralWorker` runs as `subprocess.Popen`, receives intent via `INTENT_ID`/`INTENT_PAYLOAD` env vars, calls `llm_complete(prompt)` with a single string (no message array). - Schema gap: `intents` table has no `thread_id`, `conversation_id`, `parent_intent_id`, `turn_number`. ### 2. Missing Components | Component | Current State | Required | |-----------|---------------|----------| | Context Retrieval | None | Fetch previous N turns, format for LLM, token budget, relevance filtering | | Context Injection | None | Prepend context to prompt before LLM call | | Token Budget Management | None | Respect context window (e.g., 2000 tokens for history) | | Relevance Filtering | None | Semantic similarity (embedding) or recency-based | | Worker API | None | Declare multi-turn support, receive context via env/API, update thread state | | Thread Tracking | None | `thread_id` in intent payload, ledger, backplane messages | ### 3. Recommended Implementation (5 phases) 1. **Data Model**: Migrate `intents` table with `thread_id`, `parent_intent_id`, `turn_number`; add `IntentRepository.get_thread_context()` and `get_by_thread()` 2. **Intent Submission & Routing**: API Gateway accepts optional `thread_id`; Mesh Node extracts and passes to `WorkerSpawner.spawn()` 3. **Worker Context Propagation**: `WorkerSpawner` passes `THREAD_ID`, `THREAD_CONTEXT_JSON` env vars; `EphemeralWorker` parses and retrieves 4. **Context Injection & LLM Integration**: Format context + intent into prompt; `PrimitiveExecution.build_*_prompt()` accepts `context`; `LLMRegistry` adds `llm_chat_complete(messages[])` for conversation format 5. **Worker Declaration & Thread Updates**: Capability template adds `supports_multi_turn: true`; worker emits `thread.updated` on completion ### 4. Key Design Decisions - **Thread ID source**: Server-generated (UUID) if not provided; client can pass for continuity - **Context retrieval**: Hybrid — last 3 turns + semantic top-K from older history (via embeddings) - **Token budget**: Configurable per capability in template.yaml; default 2000 - **Context format**: Structured messages array `[{"role": "user", "content": "..."}, ...]` - **Worker opt-in**: Capability declares `supports_multi_turn` in template.yaml ### 5. Integration Points | Layer | File | Change | |-------|------|--------| | Schema | `substrate/ledger/schema.py` | Add thread columns to `intents` | | Repository | `substrate/ledger/repositories/intent.py` | Add thread context methods | | API Gateway | `docker/api_gateway.py` | Accept `thread_id` in `/intent` POST | | Mesh Node | `docker/mesh_node.py` | Track thread context, pass to spawner | | Worker Spawner | `docker/worker_spawner.py` | Add `THREAD_ID`, `THREAD_CONTEXT` env vars | | Ephemeral Worker | `docker/ephemeral_worker.py` | Read thread env vars, inject context | | Primitive Execution | `runtime/primitive_execution.py` | `build_*_prompt()` accepts context | | LLM Registry | `substrate/llm_registry.py` | Add `llm_chat_complete(messages)` | ### 6. Dependencies & Risks - **CONV-001** (Conversation Interface): Defines HTTP/WebSocket API for thread management - **EMBED-004** (Conversational Memory): Embedding-based relevance filtering - **#123** (No embedding infrastructure): Blocks semantic relevance; literal matching only currently - **Risk**: Schema migration breaking existing deployments — mitigate with `DEFAULT ''` columns and `ALTER TABLE` in `SchemaManager._add_column_if_missing()` - **Risk**: Context exceeds LLM context window — enforce `max_tokens` in retrieval; truncate oldest first ### 7. Test Strategy 1. Unit: `IntentRepository.get_thread_context()` with mocked ledger 2. Integration: API Gateway → Mesh Node → Worker Spawner → Ephemeral Worker with `thread_id` 3. E2E: Multi-turn conversation via API (Turn 1 → Turn 2 referencing Turn 1 → Turn 3) 4. Architecture: Verify worker receives context via env vars (not direct ledger access) Full analysis log: `/tmp/kilo-gsd-mesh-seed-132-research.log` Closing this research ticket — implementation can be tracked as follow-up issues.

2026-04-11 03:12:13 +00:00

Overview

Workers can access and utilize previous turns in a conversation for context-aware responses.

Current State

Workers receive isolated intents with no history.

Missing Components

Context Retrieval

Fetch previous N turns from thread
Format context for LLM consumption
Token budget management (don't exceed context window)
Relevance filtering (not just last N, but most relevant)

Context Injection

def execute(intent, thread_id=None):
    if thread_id:
        context = get_thread_context(thread_id, max_tokens=2000)
        prompt = f"{context}\n\nCurrent request: {intent}"
    else:
        prompt = intent
    return llm.complete(prompt)

Worker API

Workers declare if they support multi-turn
Context passed via environment or API
Worker can update thread state

Acceptance Criteria

Worker receives previous turns as context
Context formatted for LLM prompts
Token limits respected
Responses reference previous context

CONV-001: Conversation Interface Infrastructure
EMBED-004: Conversational Memory

Priority

HIGH - Required for chat-like interactions

## Overview Workers can access and utilize previous turns in a conversation for context-aware responses. ## Current State Workers receive isolated intents with no history. ## Missing Components ### Context Retrieval - [ ] Fetch previous N turns from thread - [ ] Format context for LLM consumption - [ ] Token budget management (don't exceed context window) - [ ] Relevance filtering (not just last N, but most relevant) ### Context Injection ```python def execute(intent, thread_id=None): if thread_id: context = get_thread_context(thread_id, max_tokens=2000) prompt = f"{context}\n\nCurrent request: {intent}" else: prompt = intent return llm.complete(prompt) ``` ### Worker API - [ ] Workers declare if they support multi-turn - [ ] Context passed via environment or API - [ ] Worker can update thread state ## Acceptance Criteria - [ ] Worker receives previous turns as context - [ ] Context formatted for LLM prompts - [ ] Token limits respected - [ ] Responses reference previous context ## Related - CONV-001: Conversation Interface Infrastructure - EMBED-004: Conversational Memory ## Priority HIGH - Required for chat-like interactions

Mark added the

feature-request

label

2026-05-12 23:26:55 +00:00

Mark added

blocked:needs-decision

and removed

labels

2026-05-21 02:15:58 +00:00

Mark removed the

2026-05-28 07:19:11 +00:00

triage-approved

2026-06-07 05:34:00 +00:00

in-progress

2026-06-16 02:30:58 +00:00

Component	Current State	Required
Context Retrieval	None	Fetch previous N turns, format for LLM, token budget, relevance filtering
Context Injection	None	Prepend context to prompt before LLM call
Token Budget Management	None	Respect context window (e.g., 2000 tokens for history)
Relevance Filtering	None	Semantic similarity (embedding) or recency-based
Worker API	None	Declare multi-turn support, receive context via env/API, update thread state
Thread Tracking	None	`thread_id` in intent payload, ledger, backplane messages

Layer	File	Change
Schema	`substrate/ledger/schema.py`	Add thread columns to `intents`
Repository	`substrate/ledger/repositories/intent.py`	Add thread context methods
API Gateway	`docker/api_gateway.py`	Accept `thread_id` in `/intent` POST
Mesh Node	`docker/mesh_node.py`	Track thread context, pass to spawner
Worker Spawner	`docker/worker_spawner.py`	Add `THREAD_ID`, `THREAD_CONTEXT` env vars
Ephemeral Worker	`docker/ephemeral_worker.py`	Read thread env vars, inject context
Primitive Execution	`runtime/primitive_execution.py`	`build_*_prompt()` accepts context
LLM Registry	`substrate/llm_registry.py`	Add `llm_chat_complete(messages)`

Mark

2026-06-16 02:49:03 +00:00

closed this issue
removed the
in-progress

triage-approved
labels

Mark referenced this issue

2026-07-15 11:38:32 +00:00

Decompose: CONV-003: Web Chat Interface #133

Rows
Columns

CONV-002: Multi-turn Context Management #132