[CRITICAL] Cache JSON file reads in memory - eliminate full-file I/O on every query #32

New issue

Closed

opened 2026-06-09 00:03:12 +00:00 by Mark · 0 comments

Mark commented

2026-06-09 00:03:12 +00:00

Owner

Location: backend/src/database.ts:286-410

Problem: Every database query reads the entire JSON file from disk via fs.readFileSync + JSON.parse with no in-memory cache. This means 4-5 synchronous file reads per request.

Impact: 50–100x slower data access as dataset grows. For N subscriptions and M API calls, I/O cost scales as O(N×M).

Fix: Load all 4 JSON files into memory at module init. Maintain Map<string, User> (by id) and Map<string, User> (by email, lowercase). Same for subscriptions by id and userId. Update indexes on mutations with a write-through pattern.

Effort: ~4-6 hours

**Location:** `backend/src/database.ts:286-410` **Problem:** Every database query reads the entire JSON file from disk via `fs.readFileSync` + `JSON.parse` with no in-memory cache. This means 4-5 synchronous file reads per request. **Impact:** 50–100x slower data access as dataset grows. For N subscriptions and M API calls, I/O cost scales as O(N×M). **Fix:** Load all 4 JSON files into memory at module init. Maintain `Map<string, User>` (by id) and `Map<string, User>` (by email, lowercase). Same for subscriptions by id and userId. Update indexes on mutations with a write-through pattern. **Effort:** ~4-6 hours