FILE-001: File Upload System

Mark commented

2026-04-11 03:13:34 +00:00

Owner

Overview

Support uploading files (PDFs, images, documents) to the mesh for processing.

Current State

Intent payloads are JSON only. No file attachment support.

Missing Components

Upload API

POST /upload
Content-Type: multipart/form-data

file: <binary data>
metadata: {"filename": "report.pdf", "content_type": "application/pdf"}

Storage

File storage backend (local FS, S3, etc.)
File metadata tracking
Size limits and quotas
Virus scanning (optional)

Processing Pipeline

Text extraction from PDFs
Image OCR (optional)
Content summarization
Automatic embedding for RAG

Intent Integration

{
  "type": "analyze_document",
  "payload": {
    "file_id": "uuid-of-uploaded-file",
    "analysis_type": "summary"
  }
}

Security

File type validation
Size limits
Malware scanning
Access control (who can access uploaded files)

Acceptance Criteria

Files can be uploaded via HTTP
Workers can access file content
Large files handled efficiently
Storage quotas enforced

EMBED-003: RAG System (uses uploaded files)
CONV-003: Web Chat Interface (needs upload)

Priority

HIGH - Required for document processing

## Overview Support uploading files (PDFs, images, documents) to the mesh for processing. ## Current State Intent payloads are JSON only. No file attachment support. ## Missing Components ### Upload API ```python POST /upload Content-Type: multipart/form-data file: <binary data> metadata: {"filename": "report.pdf", "content_type": "application/pdf"} ``` ### Storage - [ ] File storage backend (local FS, S3, etc.) - [ ] File metadata tracking - [ ] Size limits and quotas - [ ] Virus scanning (optional) ### Processing Pipeline - [ ] Text extraction from PDFs - [ ] Image OCR (optional) - [ ] Content summarization - [ ] Automatic embedding for RAG ### Intent Integration ```json { "type": "analyze_document", "payload": { "file_id": "uuid-of-uploaded-file", "analysis_type": "summary" } } ``` ## Security - [ ] File type validation - [ ] Size limits - [ ] Malware scanning - [ ] Access control (who can access uploaded files) ## Acceptance Criteria - [ ] Files can be uploaded via HTTP - [ ] Workers can access file content - [ ] Large files handled efficiently - [ ] Storage quotas enforced ## Related - EMBED-003: RAG System (uses uploaded files) - CONV-003: Web Chat Interface (needs upload) ## Priority HIGH - Required for document processing

Mark added the

feature-request

label

2026-05-12 23:26:54 +00:00

Mark added

blocked:needs-decision

and removed

feature-request

labels

2026-05-18 03:14:03 +00:00

Mark commented

2026-05-18 03:14:04 +00:00

Author

Owner

Feature Triage Assessment

Scope: large
Risk: medium
Spec Alignment: misaligned
Summary: Adds a complete file upload system (HTTP endpoint, storage backends, processing pipeline, quotas, security) to the mesh.
Recommendation: Reject as-is. File hosting/storage is an external service concern, not mesh infrastructure. If document processing capabilities are needed, implement them as worker skills that read from external sources (S3 URLs, object store references) when an intent references a file — the mesh routes intents and spawns workers, it should not become a file server.
Rationale: Scope is large — requires 11+ new files (API route, storage abstraction with 2+ backends, metadata model, processing pipeline with 4 stages, security, quotas). Risk is medium due to security sensitivity (file validation, malware scanning, path traversal) and IO-heavy processing. Spec alignment is misaligned — the project's AGENTS.md defines the mesh as infrastructure that routes intents and spawns workers, not a web application that hosts upload endpoints and manages file storage. This feature describes a separate document processing service that happens to emit mesh intents, not mesh infrastructure itself. The mesh's job is routing and spawning; file upload, storage, and processing are worker/application concerns.

Labeled blocked:needs-decision — unblock on Forgejo to dispatch to the fixer pipeline.

## Feature Triage Assessment **Scope:** large **Risk:** medium **Spec Alignment:** misaligned **Summary:** Adds a complete file upload system (HTTP endpoint, storage backends, processing pipeline, quotas, security) to the mesh. **Recommendation:** Reject as-is. File hosting/storage is an external service concern, not mesh infrastructure. If document processing capabilities are needed, implement them as worker skills that read from external sources (S3 URLs, object store references) when an intent references a file — the mesh routes intents and spawns workers, it should not become a file server. **Rationale:** Scope is large — requires 11+ new files (API route, storage abstraction with 2+ backends, metadata model, processing pipeline with 4 stages, security, quotas). Risk is medium due to security sensitivity (file validation, malware scanning, path traversal) and IO-heavy processing. Spec alignment is misaligned — the project's AGENTS.md defines the mesh as infrastructure that routes intents and spawns workers, not a web application that hosts upload endpoints and manages file storage. This feature describes a separate document processing service that happens to emit mesh intents, not mesh infrastructure itself. The mesh's job is routing and spawning; file upload, storage, and processing are worker/application concerns. Labeled `blocked:needs-decision` — unblock on Forgejo to dispatch to the fixer pipeline.

Mark removed the

blocked:needs-decision

label

2026-05-28 07:19:10 +00:00

Mark added the

blocked:needs-decision

label

2026-06-07 05:30:39 +00:00

Mark commented

2026-06-07 05:30:39 +00:00

Author

Owner

⏸ Needs Decision

What needs to be decided?

Should this be split into smaller sub-issues? If yes, remove the blocked:needs-decision label and the triage will create a decomposition issue.
Do you want to proceed despite the risk? If yes, remove the blocked:needs-decision label and the triage will approve it.

Blocking factors:

Large scope — may need decomposition into phased sub-issues
High risk — needs human sign-off before proceeding

Scope: large Risk: high Spec: aligned Type: feature
Summary: Add comprehensive file upload support with storage backends, processing pipeline, and intent integration for document analysis.
Suggestion: Break into sub-issues by phase: Phase 1 — upload API endpoint, local FS storage, file type validation, size limits (core upload, 3-5 files). Phase 2 — processing pipeline as worker skills (text extraction, summarization, embedding) integrated via the existing intent/worker architecture. Phase 3 — S3 backend, OCR, access control, quotas, malware scanning. Each phase is independently shippable and testable.
Rationale: Scope is large because the issue spans multiple new subsystems (storage layer, processing pipeline, API, security) totaling well over 15 files. Risk is high due to security-sensitive operations (file I/O, malware, path traversal), IO-heavy workloads (upload/download), and integration with core intent routing and worker spawning, which are still partially incomplete per AGENTS.md (issues #98, #102). Spec alignment is positive — the system needs document processing and the worker architecture is well-suited to handle it, but the implementation must respect the infrastructure/agent separation.

To unblock: Remove the blocked:needs-decision label. The triage will apply the appropriate outcome based on your choice above.

## ⏸ Needs Decision **What needs to be decided?** - **Should this be split into smaller sub-issues?** If yes, remove the `blocked:needs-decision` label and the triage will create a decomposition issue. - **Do you want to proceed despite the risk?** If yes, remove the `blocked:needs-decision` label and the triage will approve it. **Blocking factors:** - **Large scope** — may need decomposition into phased sub-issues - **High risk** — needs human sign-off before proceeding --- **Scope:** large **Risk:** high **Spec:** aligned **Type:** feature **Summary:** Add comprehensive file upload support with storage backends, processing pipeline, and intent integration for document analysis. **Suggestion:** Break into sub-issues by phase: Phase 1 — upload API endpoint, local FS storage, file type validation, size limits (core upload, 3-5 files). Phase 2 — processing pipeline as worker skills (text extraction, summarization, embedding) integrated via the existing intent/worker architecture. Phase 3 — S3 backend, OCR, access control, quotas, malware scanning. Each phase is independently shippable and testable. **Rationale:** Scope is large because the issue spans multiple new subsystems (storage layer, processing pipeline, API, security) totaling well over 15 files. Risk is high due to security-sensitive operations (file I/O, malware, path traversal), IO-heavy workloads (upload/download), and integration with core intent routing and worker spawning, which are still partially incomplete per AGENTS.md (issues #98, #102). Spec alignment is positive — the system needs document processing and the worker architecture is well-suited to handle it, but the implementation must respect the infrastructure/agent separation. **To unblock:** Remove the `blocked:needs-decision` label. The triage will apply the appropriate outcome based on your choice above.

Mark referenced this issue

2026-07-15 11:38:32 +00:00

Decompose: CONV-003: Web Chat Interface #133

Mark added the

blocked:preflight

label

2026-07-31 04:26:46 +00:00

Mark closed this issue

2026-08-11 03:53:43 +00:00

Mark commented

2026-08-11 03:53:43 +00:00

Author