Mark/mesh-seed
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

[Phase 6.4] Conversation Memory #73

New issue
Closed
opened 2026-04-02 20:53:37 +00:00 by Mark · 2 comments
Mark commented 2026-04-02 20:53:37 +00:00
Owner
Copy link

Overview

Multi-turn conversation memory for contextual, coherent interactions.

Problem Statement

Current mesh processes intents as isolated, one-shot requests. Cannot:

  • Remember previous messages in conversation
  • Maintain project context across sessions
  • Learn user preferences over time
  • Build on previous work (iterative refinement)
  • Handle multi-step workflows with context

Each intent starts from zero, losing valuable context.

Proposed Solution

6.4.1: Conversation Table

CREATE TABLE conversations (
    conversation_id TEXT PRIMARY KEY,
    project TEXT,
    created_at TIMESTAMP,
    updated_at TIMESTAMP,
    metadata JSON
);

CREATE TABLE conversation_turns (
    turn_id TEXT PRIMARY KEY,
    conversation_id TEXT,
    role TEXT,  -- user, assistant, system
    content TEXT,
    intent_id TEXT,  -- Optional: linked intent
    created_at TIMESTAMP,
    metadata JSON
);

CREATE TABLE project_memory (
    project TEXT PRIMARY KEY,
    context TEXT,  -- Long-term project context
    preferences JSON,  -- User preferences
    created_at TIMESTAMP,
    updated_at TIMESTAMP
);

6.4.2: Conversation Memory Manager

class ConversationMemory:
    """Manage conversation history and project memory."""
    
    def create_conversation(self, project: str = None) -> ConversationID:
        """Start new conversation"""
        
    def add_turn(self, conversation_id: str, role: str, content: str, intent_id: str = None):
        """Add message to conversation"""
        
    def get_turns(self, conversation_id: str, limit: int = 10) -> list[Turn]:
        """Get recent turns for context"""
        
    def get_full_conversation(self, conversation_id: str) -> list[Turn]:
        """Get all turns"""
        
    def get_project_memory(self, project: str) -> ProjectMemory:
        """Get long-term project context"""
        
    def update_project_memory(self, project: str, context: str, preferences: dict):
        """Update project-level memory"""

6.4.3: Context Injection in Intents

# Intent with conversation context
intent_type: refine_article
payload:
  conversation_id: conv_abc123
  content: "Make it more concise"
  
# Router injects context:
intent:
  payload:
    conversation_id: conv_abc123
    content: "Make it more concise"
    context:  # Injected by router
      turns:
        - {role: user, content: "Write article about AI ethics"}
        - {role: assistant, content: "Generated 619-word article..."}
      project_memory:
        preferences: {style: technical, length: short}

6.4.4: Memory Expiration

# Configuration
memory_config:
  conversation_ttl: 7 days  # Delete old conversations
  turn_limit: 100  # Max turns per conversation
  project_memory_ttl: 30 days  # Refresh project memory
  
# Cleanup job
def cleanup_expired_memory():
    """Remove old conversations and turns"""

Deliverables

  • conversations table in ledger
  • conversation_turns table in ledger
  • project_memory table in ledger
  • substrate/conversation_memory.py - Memory manager
  • Context injection in intent router
  • API endpoints: /v1/conversations/*
  • Memory cleanup job
  • Unit tests
  • Integration tests

Dependencies

  • None (standalone)

Estimated Duration

1 week

Success Criteria

  • Conversations tracked with turns
  • Recent turns injected as context
  • Project memory persists across sessions
  • Multi-turn conversations work
  • Memory cleanup prevents unbounded growth

Related

  • Parent: #69 (Phase 6 Meta)
## Overview Multi-turn conversation memory for contextual, coherent interactions. ## Problem Statement Current mesh processes intents as isolated, one-shot requests. Cannot: - Remember previous messages in conversation - Maintain project context across sessions - Learn user preferences over time - Build on previous work (iterative refinement) - Handle multi-step workflows with context Each intent starts from zero, losing valuable context. ## Proposed Solution ### 6.4.1: Conversation Table ```sql CREATE TABLE conversations ( conversation_id TEXT PRIMARY KEY, project TEXT, created_at TIMESTAMP, updated_at TIMESTAMP, metadata JSON ); CREATE TABLE conversation_turns ( turn_id TEXT PRIMARY KEY, conversation_id TEXT, role TEXT, -- user, assistant, system content TEXT, intent_id TEXT, -- Optional: linked intent created_at TIMESTAMP, metadata JSON ); CREATE TABLE project_memory ( project TEXT PRIMARY KEY, context TEXT, -- Long-term project context preferences JSON, -- User preferences created_at TIMESTAMP, updated_at TIMESTAMP ); ``` ### 6.4.2: Conversation Memory Manager ```python class ConversationMemory: """Manage conversation history and project memory.""" def create_conversation(self, project: str = None) -> ConversationID: """Start new conversation""" def add_turn(self, conversation_id: str, role: str, content: str, intent_id: str = None): """Add message to conversation""" def get_turns(self, conversation_id: str, limit: int = 10) -> list[Turn]: """Get recent turns for context""" def get_full_conversation(self, conversation_id: str) -> list[Turn]: """Get all turns""" def get_project_memory(self, project: str) -> ProjectMemory: """Get long-term project context""" def update_project_memory(self, project: str, context: str, preferences: dict): """Update project-level memory""" ``` ### 6.4.3: Context Injection in Intents ```yaml # Intent with conversation context intent_type: refine_article payload: conversation_id: conv_abc123 content: "Make it more concise" # Router injects context: intent: payload: conversation_id: conv_abc123 content: "Make it more concise" context: # Injected by router turns: - {role: user, content: "Write article about AI ethics"} - {role: assistant, content: "Generated 619-word article..."} project_memory: preferences: {style: technical, length: short} ``` ### 6.4.4: Memory Expiration ```python # Configuration memory_config: conversation_ttl: 7 days # Delete old conversations turn_limit: 100 # Max turns per conversation project_memory_ttl: 30 days # Refresh project memory # Cleanup job def cleanup_expired_memory(): """Remove old conversations and turns""" ``` ## Deliverables - [ ] `conversations` table in ledger - [ ] `conversation_turns` table in ledger - [ ] `project_memory` table in ledger - [ ] `substrate/conversation_memory.py` - Memory manager - [ ] Context injection in intent router - [ ] API endpoints: `/v1/conversations/*` - [ ] Memory cleanup job - [ ] Unit tests - [ ] Integration tests ## Dependencies - None (standalone) ## Estimated Duration 1 week ## Success Criteria - [ ] Conversations tracked with turns - [ ] Recent turns injected as context - [ ] Project memory persists across sessions - [ ] Multi-turn conversations work - [ ] Memory cleanup prevents unbounded growth ## Related - Parent: #69 (Phase 6 Meta)
Mark commented 2026-04-07 03:14:10 +00:00
Author
Owner
Copy link

Implementation Plan

Created comprehensive plan at docs/PROJECT_MEMORY_PLAN.md - extends Issue #73 beyond conversation memory into full project/episodic memory system.

Sub-Issues Created

This work has been broken into 8 sub-issues:

Issue Title Description
#84 Database Schema 7 new tables for project memory
#85 ProjectMemoryManager Core orchestrator class
#86 Fact Extraction LLM-based fact extraction + deduplication
#87 Episode Compression Conversation-to-episode compression
#88 Context Assembly Context builder for LLM calls
#89 Router Integration Inject context into intent routing
#90 API Endpoints REST API for all memory operations
#91 Advanced Features Cross-project, cleanup, monitoring

Key Extensions Beyond Original #73

  • Project-centric multi-session context
  • 3-layer memory (working, episodic, semantic)
  • Automatic compression triggers
  • Integrated TODO system with dependencies
  • Artifact linking
  • Cross-project learning

Status

Awaiting implementation of #84 (database schema) to begin.

## Implementation Plan Created comprehensive plan at `docs/PROJECT_MEMORY_PLAN.md` - extends Issue #73 beyond conversation memory into full project/episodic memory system. ## Sub-Issues Created This work has been broken into 8 sub-issues: | Issue | Title | Description | |-------|-------|-------------| | #84 | Database Schema | 7 new tables for project memory | | #85 | ProjectMemoryManager | Core orchestrator class | | #86 | Fact Extraction | LLM-based fact extraction + deduplication | | #87 | Episode Compression | Conversation-to-episode compression | | #88 | Context Assembly | Context builder for LLM calls | | #89 | Router Integration | Inject context into intent routing | | #90 | API Endpoints | REST API for all memory operations | | #91 | Advanced Features | Cross-project, cleanup, monitoring | ## Key Extensions Beyond Original #73 - Project-centric multi-session context - 3-layer memory (working, episodic, semantic) - Automatic compression triggers - Integrated TODO system with dependencies - Artifact linking - Cross-project learning ## Status Awaiting implementation of #84 (database schema) to begin.
Mark commented 2026-04-07 04:20:35 +00:00
Author
Owner
Copy link

New Work: Global Document Library

Created comprehensive plan at docs/DOCUMENT_LIBRARY_PLAN.md - extends the memory system with RAG for project files.

Document Library Sub-Issues Created

Issue Title Focus
#92 Core Schema Database + DocumentIndex class
#93 File Scanner Directory scanning, file discovery
#94 Document Chunker Split docs into overlapping chunks
#95 Document Retriever Semantic search with excerpts
#96 Skills document_indexer, document_search
#97 Integration & Docs Context assembler, API docs

What's Included

  • Project files (RAG for codebase)
  • Mesh artifacts (from artifact_store)
  • Templates (capabilities, tools, skills)
  • Schemas (config, schemas/)
  • Semantic search with excerpts
  • Filename and line number in results
  • Skills for management

Plan Document

Full details: docs/DOCUMENT_LIBRARY_PLAN.md

## New Work: Global Document Library Created comprehensive plan at `docs/DOCUMENT_LIBRARY_PLAN.md` - extends the memory system with RAG for project files. ### Document Library Sub-Issues Created | Issue | Title | Focus | |-------|-------|-------| | #92 | Core Schema | Database + DocumentIndex class | | #93 | File Scanner | Directory scanning, file discovery | | #94 | Document Chunker | Split docs into overlapping chunks | | #95 | Document Retriever | Semantic search with excerpts | | #96 | Skills | document_indexer, document_search | | #97 | Integration & Docs | Context assembler, API docs | ### What's Included - Project files (RAG for codebase) - Mesh artifacts (from artifact_store) - Templates (capabilities, tools, skills) - Schemas (config, schemas/) - Semantic search with excerpts - Filename and line number in results - Skills for management ### Plan Document Full details: `docs/DOCUMENT_LIBRARY_PLAN.md`
Mark closed this issue 2026-04-07 04:26:46 +00:00
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
mesh-seed#142-gate-verify-pass-07b1f89
Labels
Clear labels   
blocked:children

   
blocked:commit-ready-failed

   
blocked:depends-231

Depends on #231 (Discord Bot Foundation) landing first. Do not pick until #231 is closed.

  
blocked:depends-236

Depends on #236 (Phase 1) landing first. Do not pick until #236 is closed.

  
blocked:depends-237

Depends on #237 (Phase 2) landing first. Do not pick until #237 is closed.

  
blocked:depends-238

Depends on #238 (Phase 3) landing first. Do not pick until #238 is closed.

  
blocked:depends-239

Depends on #239 (Phase 4) landing first. Do not pick until #239 is closed.

  
blocked:gate-fetch

   
blocked:gate-fetch

   
blocked:gate-lint

   
blocked:gate-research

   
blocked:needs-attention

Janitor escalated: pipeline terminated without producing expected changes. Human review required.

  
blocked:needs-decision

   
blocked:no-changes

   
blocked:no-procedure

   
blocked:preflight

Preflight check failed

  
blocked:verify-failed

   
bug

Something is broken

  
enhancement

New feature or request

  
feature-request

   
high-priority

Critical path items

  
in-progress

Issue is currently being worked on by a quickfix runner

  
lint-cleanup

Repo lint baseline is dirty

  
needs-decompose

   
needs-triage

   
ops-retry

   
ops-task

Operations task — install, configure, deploy, API calls

  
phase-5

Phase 5 Self-Building Infrastructure

  
quickfix

Pipe to opencode for direct fix

  
security/medium

Medium severity security issue

  
self-building

Autonomous code generation and self-improvement

  
triage-approved

Approved by triage — queued for fixer pipeline

No labels
blocked:children
blocked:commit-ready-failed
blocked:depends-231
blocked:depends-236
blocked:depends-237
blocked:depends-238
blocked:depends-239
blocked:gate-fetch
blocked:gate-fetch
blocked:gate-lint
blocked:gate-research
blocked:needs-attention
blocked:needs-decision
blocked:no-changes
blocked:no-procedure
blocked:preflight
blocked:verify-failed
bug
enhancement
feature-request
high-priority
in-progress
lint-cleanup
needs-decompose
needs-triage
ops-retry
ops-task
phase-5
quickfix
security/medium
self-building
triage-approved
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
Mark/mesh-seed#73
No description provided.