[P2] Streaming LLM Response Display #87

New issue

Closed

opened 2026-06-10 03:32:30 +00:00 by Mark · 0 comments

Mark commented

2026-06-10 03:32:30 +00:00

Owner

Problem

LLM responses (entity extraction, relationship analysis) appear all at once after completion. For transparency:

User does not see LLM thinking process
Long responses create wait time
No visibility into parsing progress

Proposed Solution

Implement streaming LLM responses:

Backend Changes

Modify LLM client to support SSE streaming
Add streaming variant of extraction endpoints
Stream tokens as they are generated

Frontend Changes

StreamingResponse component
- Display LLM output as it streams
- Markdown rendering with syntax highlighting
- Streaming indicator animation
- Support for code blocks, tables, etc.
Update entity extraction UI
- Streaming display during extraction
- Progressive rendering of results

Files to Create/Modify

backend/app/services/llm/client.py - add streaming support
backend/app/routers/evidence.py - add streaming endpoints
frontend/src/components/StreamingResponse.jsx (new)
frontend/src/components/EvidenceReviewPage.jsx - streaming extraction

Acceptance Criteria

LLM responses stream token by token
Markdown renders correctly during streaming
User sees progress during long extractions
Streaming can be cancelled by user

Effort

Medium - 1 week

## Problem LLM responses (entity extraction, relationship analysis) appear all at once after completion. For transparency: - User does not see LLM thinking process - Long responses create wait time - No visibility into parsing progress ## Proposed Solution Implement streaming LLM responses: ### Backend Changes 1. Modify LLM client to support SSE streaming 2. Add streaming variant of extraction endpoints 3. Stream tokens as they are generated ### Frontend Changes 1. StreamingResponse component - Display LLM output as it streams - Markdown rendering with syntax highlighting - Streaming indicator animation - Support for code blocks, tables, etc. 2. Update entity extraction UI - Streaming display during extraction - Progressive rendering of results ### Files to Create/Modify - backend/app/services/llm/client.py - add streaming support - backend/app/routers/evidence.py - add streaming endpoints - frontend/src/components/StreamingResponse.jsx (new) - frontend/src/components/EvidenceReviewPage.jsx - streaming extraction ## Acceptance Criteria - LLM responses stream token by token - Markdown renders correctly during streaming - User sees progress during long extractions - Streaming can be cancelled by user ## Effort Medium - 1 week

Mark

2026-06-10 03:32:30 +00:00

closed this issue
added the
agentic-osint

enhancement

triage-approved
labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

Mark/osint-hub#87

No description provided.

Rows
Columns