[Worker] Prompt tuning system for worker capabilities #58

New issue

Closed

opened 2026-03-25 03:23:27 +00:00 by Mark · 1 comment

Mark commented

2026-03-25 03:23:27 +00:00

Owner

Problem

Workers need dynamic prompt tuning based on:

Intent context and history
Skill effectiveness feedback
Success/failure patterns
User preferences

Current State

Workers have static capability templates
No runtime prompt adjustment
No learning from previous executions
Fixed behavior regardless of context

Proposed System

1. Context-Aware Prompts

execution:
  prompt_template: |
    You are {{profile.role}} handling {{intent.type}}.
    Previous similar intents: {{context.similar_count}}
    Success rate: {{context.success_rate}}
    Adjust approach: {{context.suggestions}}

2. Prompt Tuning Skills

prompt_optimizer: Analyze and improve prompts
context_compressor: Fit more context in token budget
tone_adapter: Match user's communication style

3. Feedback Loop

Worker executes with current prompt
Result evaluated (success, latency, cost)
analyze-project identifies prompt issues
prompt_optimizer suggests improvements
New prompt variant tested (A/B)
Better prompt wins championship

4. Template System

Base templates in templates/workers/
Runtime template selection based on intent
Template composition (mixins for common patterns)
Versioned templates with rollback

Metrics to Track

Prompt token efficiency
Instruction clarity score (from analyze-project)
Success rate by prompt variant
User satisfaction proxy (bond rewards)

agent-tool-builder provides tool design patterns
ai-engineer has prompting expertise
Championship system can select best prompts

## Problem Workers need dynamic prompt tuning based on: - Intent context and history - Skill effectiveness feedback - Success/failure patterns - User preferences ## Current State - Workers have static capability templates - No runtime prompt adjustment - No learning from previous executions - Fixed behavior regardless of context ## Proposed System ### 1. Context-Aware Prompts ```yaml execution: prompt_template: | You are {{profile.role}} handling {{intent.type}}. Previous similar intents: {{context.similar_count}} Success rate: {{context.success_rate}} Adjust approach: {{context.suggestions}} ``` ### 2. Prompt Tuning Skills - `prompt_optimizer`: Analyze and improve prompts - `context_compressor`: Fit more context in token budget - `tone_adapter`: Match user's communication style ### 3. Feedback Loop 1. Worker executes with current prompt 2. Result evaluated (success, latency, cost) 3. `analyze-project` identifies prompt issues 4. `prompt_optimizer` suggests improvements 5. New prompt variant tested (A/B) 6. Better prompt wins championship ### 4. Template System - Base templates in `templates/workers/` - Runtime template selection based on intent - Template composition (mixins for common patterns) - Versioned templates with rollback ## Metrics to Track - Prompt token efficiency - Instruction clarity score (from analyze-project) - Success rate by prompt variant - User satisfaction proxy (bond rewards) ## Related - `agent-tool-builder` provides tool design patterns - `ai-engineer` has prompting expertise - Championship system can select best prompts