MemoryQueue delayed jobs never fire — job.delay is never cleared, so scheduled backups never run #51

New issue

Closed

opened 2026-08-14 11:13:36 +00:00 by Mark · 0 comments

Mark commented

2026-08-14 11:13:36 +00:00

Owner

Summary

MemoryQueue.add() with delay > 0 stores job.delay, arms a setTimeout that calls processNextJob() after the delay — but never clears job.delay. processNextJob() selects the next job with j.status === 'pending' && !j.delay, so the delayed job is skipped forever, even after its timer fires. Scheduled backups (which are enqueued with a delay via scheduleNextRun) never execute.

Location

backend/src/core/queue/memory-queue.ts:

line 51: delay: options?.delay, — delay stored on the job
lines 57-66: setTimeout(() => { this.delayedJobs.delete(job.id); this.processNextJob(); }, options.delay) — timer fires but does not clear job.delay
line 72: Array.from(this.jobs.values()).find(j => j.status === 'pending' && !j.delay) — filter permanently excludes the delayed job

Reproduction (from kanban QA card t_3b5379e0)

Create a schedule; scheduleNextRun() (backend/src/core/scheduler/schedule-runner.ts:71-80) adds a job with delay = nextRunAt - now and jobId schedule-runner:<id>.
Wait past the delay. The setTimeout fires processNextJob(), but the job still has .delay set, so find() returns nothing → job stays pending forever, status never transitions, no backup artifact is produced.
Confirmed in the deployed stack: no scheduled backup fired until the container was patched to clear job.delay after the timeout (transient container patch, lost on recreate).

Actual vs expected

Expected: once the delay elapses, the job is processed (delay cleared or filter based on scheduledAt).
Actual: delayed jobs stay pending forever; scheduled backups never run.

References

backend/src/core/queue/memory-queue.ts:51,57-66,72
backend/src/core/scheduler/schedule-runner.ts:64,71-80
Kanban card: t_3b5379e0 (QA board, fixit)

## Summary `MemoryQueue.add()` with `delay > 0` stores `job.delay`, arms a `setTimeout` that calls `processNextJob()` after the delay — but never clears `job.delay`. `processNextJob()` selects the next job with `j.status === 'pending' && !j.delay`, so the delayed job is skipped forever, even after its timer fires. Scheduled backups (which are enqueued with a delay via `scheduleNextRun`) never execute. ## Location `backend/src/core/queue/memory-queue.ts`: - line 51: `delay: options?.delay,` — delay stored on the job - lines 57-66: `setTimeout(() => { this.delayedJobs.delete(job.id); this.processNextJob(); }, options.delay)` — timer fires but does not clear `job.delay` - line 72: `Array.from(this.jobs.values()).find(j => j.status === 'pending' && !j.delay)` — filter permanently excludes the delayed job ## Reproduction (from kanban QA card t_3b5379e0) 1. Create a schedule; `scheduleNextRun()` (`backend/src/core/scheduler/schedule-runner.ts:71-80`) adds a job with `delay = nextRunAt - now` and jobId `schedule-runner:<id>`. 2. Wait past the delay. The `setTimeout` fires `processNextJob()`, but the job still has `.delay` set, so `find()` returns nothing → job stays `pending` forever, status never transitions, no backup artifact is produced. 3. Confirmed in the deployed stack: no scheduled backup fired until the container was patched to clear `job.delay` after the timeout (transient container patch, lost on recreate). ## Actual vs expected - **Expected**: once the delay elapses, the job is processed (delay cleared or filter based on `scheduledAt`). - **Actual**: delayed jobs stay `pending` forever; scheduled backups never run. ## References - `backend/src/core/queue/memory-queue.ts:51,57-66,72` - `backend/src/core/scheduler/schedule-runner.ts:64,71-80` - Kanban card: t_3b5379e0 (QA board, fixit)