Mark/Kimi_Agent_Brother_10_Digitizing_App
1
0
Fork 0
Code Issues 14 Pull requests Projects Releases Packages Wiki Activity Actions

Decompose: Implement real auto-digitize engine #3

New issue
Closed
opened 2026-06-22 14:58:57 +00:00 by Mark · 2 comments
Mark commented 2026-06-22 14:58:57 +00:00
Owner
Copy link

Auto-Digitize Engine — Research & Engineering Spec

Status: Awaiting research spike. This issue is too large for a single fix; it is
a multi-month engineering project spanning image processing, geometry, and PES
binary format work. Decomposed into 5 sub-issues — one per pipeline stage — so each
stage can be researched, planned, and shipped independently.

Context

The Workspace page already has the data model for an auto-digitize engine:

  • src/components/workspace/types.ts defines StitchObject, ColorRegion, StitchType, and the WorkspaceState shape.
  • src/components/workspace/useWorkspace.ts defines autoDigitizeSettings (with stitchPreference, density, minStitchLength) and the SET_AUTO_DIGITIZE_SETTING / SET_STITCH_OBJECTS actions.
  • src/components/workspace/LeftToolbar.tsx registers the auto-digitize tool (Wand2 icon, A shortcut) and the manual-stitch tool.
  • src/components/workspace/ExportDialog.tsx and AppHeader.tsx consume stitchObjects to drive the export flow.

The stub in useWorkspace.ts produces fake ColorRegion[] after a timer fires. Real
digitizing has to convert the input raster into a hierarchy of stitch objects that a
Brother Entrepreneur PR-10 10-needle machine can actually sew.

Why this can't be a single ticket: the pipeline is image processing → geometry →
binary serialization. Each stage has independent libraries, failure modes, and
acceptance criteria. Conflating them into one ticket produces a partial implementation
that ships nothing usable.

Target

Given an uploaded raster image (PNG/JPG, ≤ 4MP for v1), produce a StitchObject[]
that:

  1. Faithfully represents the input (silhouette accuracy within 2 px at native res).
  2. Uses the right stitch type per region (satin for narrow, tatami fill for large,
    running stitch for thin detail).
  3. Stays within Brother PR-10 hoop constraints (default 100×100 mm, configurable).
  4. Limits color count to ≤ 10 (the PR-10 has 10 needles — anything beyond requires
    manual re-thread planning that v1 doesn't attempt).
  5. Maps each detected color to the closest Brother thread palette entry.

Non-Goals (v1)

  • Vector input (SVG, AI) — separate ticket.
  • Cross-stitch / hardanger / specialty stitches — out of scope.
  • Photorealistic stitch shading / thread blending — out of scope.
  • Backend persistence — covered by #4.

Pipeline Stages

The work splits cleanly into 5 stages, ordered by data dependency. Each is a
separate sub-issue with its own research spike, design, and implementation.

P1: Image Ingestion & Color Quantization

Goal: Convert a raster input into a ColorRegion[] of axis-aligned-ish polygons,
each tagged with the dominant quantized color.

Inputs: File from upload (PNG, JPG, WebP), max 4MP (downscale if larger).

Approach candidates:

  • k-means in RGB or Lab color space (k = 6–10), then connected-component labeling.
  • Flood-fill from a quantized palette, then merge adjacent same-color regions.
  • Median-cut quantization, then region growing.

Outputs: ColorRegion[] where each region has { id, polygon: Point[], color: {src: string, paletteIndex: number | null} }. Polygon should be a simple contour
(usable by Phase 3/4 stitch generators).

Acceptance:

  • 95% pixel coverage of source image with ≤ 10 regions.
  • Regions have < 100 vertices each (after Douglas-Peucker simplification).
  • Test fixtures: solid color rect, two-color rect, gradient image, line art.

Files: new src/lib/digitize/regions.ts, integration in
src/components/workspace/useWorkspace.ts to replace the timer stub.

P2: Region Classification

Goal: For each ColorRegion from P1, assign a StitchType ('satin' | 'tatami-fill' | 'running' | 'skip').

Heuristics:

  • Satin: bounding-box aspect ratio > 3:1 AND area below satin-area-cap (e.g. 50 mm²).
    Used for borders, text, narrow decorative shapes.
  • Tatami fill: area above tatami-min-area threshold (e.g. > 4 mm²) AND aspect ratio
    not extreme. Used for solid color fills.
  • Running stitch: longest dimension > 20× shortest AND area very small. Used for
    outlines and fine detail.
  • Skip: region too small to stitch (area < 1 mm²). Excluded from output.

Tunables: expose satinAspectThreshold, tatamiMinAreaMm2, runningAspectRatio
in autoDigitizeSettings with sensible defaults.

Acceptance:

  • Test fixtures where expected classification is hand-verified.
  • Deterministic output for the same input + settings.

Files: new src/lib/digitize/classify.ts, types in
src/components/workspace/types.ts.

P3: Satin Column Stitch Generation

Goal: For each region tagged satin in P2, generate the parallel stitch strokes
that make up a satin column. This is the workhorse of embroidery digitizing — most
logos and text are satin.

Algorithm outline:

  1. Compute the medial axis / skeleton of the region.
  2. For each skeleton edge, walk perpendicular at the configured density to generate
    parallel stitch pairs.
  3. Handle column ends: tie-off stitches, jump stitches, or trims per Brother PES
    conventions.
  4. Respect PR-10 max stitch length (8 mm default) and density (typically 0.4 mm).

Output: StitchObject[] of type satin, each containing a list of
{x, y} needle-down points representing the full stitch run.

Acceptance:

  • Visual test: render stitch runs on a Fabric.js overlay over the source image.
  • Density tunability honored.
  • No stitch longer than 8 mm without an intermediate tie-off.

Files: new src/lib/digitize/satin.ts.

P4: Tatami Fill Stitch Generation

Goal: For each region tagged tatami-fill in P2, generate parallel stitch rows
covering the region interior with optimal direction.

Algorithm outline:

  1. Pick a primary direction (auto: longest bounding-box axis, with user override
    via autoDigitizeSettings.fillDirection).
  2. Generate rows of stitches perpendicular to the direction, spaced at the configured
    density.
  3. Clamp each row to the region polygon (ray-cast clip).
  4. Alternate row direction on adjacent rows (zigzag) to minimize jump stitches.
  5. Underlay: optionally generate a perpendicular underlay row first (improves
    coverage for thin fabrics — Brother PR-10 default is on).

Output: StitchObject[] of type fill.

Acceptance:

  • Region coverage > 98% (within density × polygon clip).
  • Underlay toggle respected.
  • No single stitch longer than 8 mm.

Files: new src/lib/digitize/fill.ts.

P5: Thread Palette Mapping & Engine Integration

Goal: Map each detected region's color to the closest Brother thread catalog
entry, then wire the full P1→P2→P3/P4 pipeline into the auto-digitize tool's action
handler in the Workspace.

Sub-steps:

  1. Source the Brother PR-10 thread catalog (79 entries in the official "Robison-Anton"
    / Brother "Polyneon" 40wt color card used by the machine). Embed as
    src/lib/digitize/palette.ts (Lab values for ΔE-2000 distance).
  2. Map each region's quantized color to the nearest catalog entry by ΔE-2000.
  3. Update src/components/workspace/useWorkspace.ts to invoke the P1→P2→P3/P4
    pipeline when the auto-digitize tool fires, replacing the timer stub.
  4. Update src/components/workspace/RightPanel.tsx to show progress / preview
    before committing stitch objects to state.
  5. Wire the resulting stitchObjects through to the existing ExportDialog (which
    will be made real in #2).

Acceptance:

  • Auto-digitize button on the Workspace produces a real stitchObjects[] for a
    test image.
  • Progress feedback in the RightPanel during processing.
  • Color swatches in the result match detected regions within reasonable tolerance.

Files: src/lib/digitize/palette.ts, modifications to useWorkspace.ts and
RightPanel.tsx.

Dependencies

  • Blocks #2 (PES export): the export pipeline consumes stitchObjects whose
    semantics are defined by P3/P4. Export can be implemented in parallel for the
    binary serialization side, but full end-to-end testing of #2 needs P1–P4 done.
  • Blocks #1 (test suite): vitest setup will be used to verify the deterministic
    outputs of P1/P2/P3/P4. P1–P4 themselves ship without tests, but write the code
    to be testable (pure functions where possible).

Research Required

This issue is large enough that a research spike is warranted before scheduling
P1–P5. The spike should:

  • Compare image-to-stitch libraries (e.g. Ink/Stitch for Python — reference impl,
    not a runtime dep; embroidery-ts; custom).
  • Lock the algorithm choices (k-means vs flood-fill for P1; skeleton algorithm for
    P3; direction optimization for P4).
  • Validate the Brother PR-10 constraints (hoop size, max stitch length, needle
    count, thread palette source).
  • Estimate the polygon simplification tolerance that balances output size vs
    visual fidelity.

A blocked:needs-decision should appear on P1 if the spike isn't complete.

# Auto-Digitize Engine — Research & Engineering Spec > **Status:** Awaiting research spike. This issue is too large for a single fix; it is > a multi-month engineering project spanning image processing, geometry, and PES > binary format work. Decomposed into 5 sub-issues — one per pipeline stage — so each > stage can be researched, planned, and shipped independently. ## Context The Workspace page already has the data model for an auto-digitize engine: - `src/components/workspace/types.ts` defines `StitchObject`, `ColorRegion`, `StitchType`, and the `WorkspaceState` shape. - `src/components/workspace/useWorkspace.ts` defines `autoDigitizeSettings` (with `stitchPreference`, density, `minStitchLength`) and the `SET_AUTO_DIGITIZE_SETTING` / `SET_STITCH_OBJECTS` actions. - `src/components/workspace/LeftToolbar.tsx` registers the `auto-digitize` tool (Wand2 icon, `A` shortcut) and the `manual-stitch` tool. - `src/components/workspace/ExportDialog.tsx` and `AppHeader.tsx` consume `stitchObjects` to drive the export flow. The stub in `useWorkspace.ts` produces fake `ColorRegion[]` after a timer fires. Real digitizing has to convert the input raster into a hierarchy of stitch objects that a Brother Entrepreneur PR-10 10-needle machine can actually sew. **Why this can't be a single ticket:** the pipeline is image processing → geometry → binary serialization. Each stage has independent libraries, failure modes, and acceptance criteria. Conflating them into one ticket produces a partial implementation that ships nothing usable. ## Target Given an uploaded raster image (PNG/JPG, ≤ 4MP for v1), produce a `StitchObject[]` that: 1. Faithfully represents the input (silhouette accuracy within 2 px at native res). 2. Uses the right stitch type per region (satin for narrow, tatami fill for large, running stitch for thin detail). 3. Stays within Brother PR-10 hoop constraints (default 100×100 mm, configurable). 4. Limits color count to ≤ 10 (the PR-10 has 10 needles — anything beyond requires manual re-thread planning that v1 doesn't attempt). 5. Maps each detected color to the closest Brother thread palette entry. ## Non-Goals (v1) - Vector input (SVG, AI) — separate ticket. - Cross-stitch / hardanger / specialty stitches — out of scope. - Photorealistic stitch shading / thread blending — out of scope. - Backend persistence — covered by #4. ## Pipeline Stages The work splits cleanly into 5 stages, ordered by data dependency. Each is a separate sub-issue with its own research spike, design, and implementation. ### P1: Image Ingestion & Color Quantization **Goal:** Convert a raster input into a `ColorRegion[]` of axis-aligned-ish polygons, each tagged with the dominant quantized color. **Inputs:** `File` from upload (PNG, JPG, WebP), max 4MP (downscale if larger). **Approach candidates:** - k-means in RGB or Lab color space (k = 6–10), then connected-component labeling. - Flood-fill from a quantized palette, then merge adjacent same-color regions. - Median-cut quantization, then region growing. **Outputs:** `ColorRegion[]` where each region has `{ id, polygon: Point[], color: {src: string, paletteIndex: number | null} }`. Polygon should be a simple contour (usable by Phase 3/4 stitch generators). **Acceptance:** - 95% pixel coverage of source image with ≤ 10 regions. - Regions have < 100 vertices each (after Douglas-Peucker simplification). - Test fixtures: solid color rect, two-color rect, gradient image, line art. **Files:** new `src/lib/digitize/regions.ts`, integration in `src/components/workspace/useWorkspace.ts` to replace the timer stub. --- ### P2: Region Classification **Goal:** For each `ColorRegion` from P1, assign a `StitchType` (`'satin' | 'tatami-fill' | 'running' | 'skip'`). **Heuristics:** - **Satin:** bounding-box aspect ratio > 3:1 AND area below satin-area-cap (e.g. 50 mm²). Used for borders, text, narrow decorative shapes. - **Tatami fill:** area above tatami-min-area threshold (e.g. > 4 mm²) AND aspect ratio not extreme. Used for solid color fills. - **Running stitch:** longest dimension > 20× shortest AND area very small. Used for outlines and fine detail. - **Skip:** region too small to stitch (area < 1 mm²). Excluded from output. **Tunables:** expose `satinAspectThreshold`, `tatamiMinAreaMm2`, `runningAspectRatio` in `autoDigitizeSettings` with sensible defaults. **Acceptance:** - Test fixtures where expected classification is hand-verified. - Deterministic output for the same input + settings. **Files:** new `src/lib/digitize/classify.ts`, types in `src/components/workspace/types.ts`. --- ### P3: Satin Column Stitch Generation **Goal:** For each region tagged `satin` in P2, generate the parallel stitch strokes that make up a satin column. This is the workhorse of embroidery digitizing — most logos and text are satin. **Algorithm outline:** 1. Compute the medial axis / skeleton of the region. 2. For each skeleton edge, walk perpendicular at the configured density to generate parallel stitch pairs. 3. Handle column ends: tie-off stitches, jump stitches, or trims per Brother PES conventions. 4. Respect PR-10 max stitch length (8 mm default) and density (typically 0.4 mm). **Output:** `StitchObject[]` of type `satin`, each containing a list of `{x, y}` needle-down points representing the full stitch run. **Acceptance:** - Visual test: render stitch runs on a Fabric.js overlay over the source image. - Density tunability honored. - No stitch longer than 8 mm without an intermediate tie-off. **Files:** new `src/lib/digitize/satin.ts`. --- ### P4: Tatami Fill Stitch Generation **Goal:** For each region tagged `tatami-fill` in P2, generate parallel stitch rows covering the region interior with optimal direction. **Algorithm outline:** 1. Pick a primary direction (auto: longest bounding-box axis, with user override via `autoDigitizeSettings.fillDirection`). 2. Generate rows of stitches perpendicular to the direction, spaced at the configured density. 3. Clamp each row to the region polygon (ray-cast clip). 4. Alternate row direction on adjacent rows (zigzag) to minimize jump stitches. 5. Underlay: optionally generate a perpendicular underlay row first (improves coverage for thin fabrics — Brother PR-10 default is on). **Output:** `StitchObject[]` of type `fill`. **Acceptance:** - Region coverage > 98% (within density × polygon clip). - Underlay toggle respected. - No single stitch longer than 8 mm. **Files:** new `src/lib/digitize/fill.ts`. --- ### P5: Thread Palette Mapping & Engine Integration **Goal:** Map each detected region's color to the closest Brother thread catalog entry, then wire the full P1→P2→P3/P4 pipeline into the auto-digitize tool's action handler in the Workspace. **Sub-steps:** 1. Source the Brother PR-10 thread catalog (79 entries in the official "Robison-Anton" / Brother "Polyneon" 40wt color card used by the machine). Embed as `src/lib/digitize/palette.ts` (Lab values for ΔE-2000 distance). 2. Map each region's quantized color to the nearest catalog entry by ΔE-2000. 3. Update `src/components/workspace/useWorkspace.ts` to invoke the P1→P2→P3/P4 pipeline when the auto-digitize tool fires, replacing the timer stub. 4. Update `src/components/workspace/RightPanel.tsx` to show progress / preview before committing stitch objects to state. 5. Wire the resulting `stitchObjects` through to the existing ExportDialog (which will be made real in #2). **Acceptance:** - Auto-digitize button on the Workspace produces a real `stitchObjects[]` for a test image. - Progress feedback in the RightPanel during processing. - Color swatches in the result match detected regions within reasonable tolerance. **Files:** `src/lib/digitize/palette.ts`, modifications to `useWorkspace.ts` and `RightPanel.tsx`. --- ## Dependencies - **Blocks #2 (PES export):** the export pipeline consumes `stitchObjects` whose semantics are defined by P3/P4. Export can be implemented in parallel for the *binary serialization* side, but full end-to-end testing of #2 needs P1–P4 done. - **Blocks #1 (test suite):** vitest setup will be used to verify the deterministic outputs of P1/P2/P3/P4. P1–P4 themselves ship without tests, but write the code to be testable (pure functions where possible). ## Research Required This issue is large enough that a research spike is warranted *before* scheduling P1–P5. The spike should: - Compare image-to-stitch libraries (e.g. Ink/Stitch for Python — reference impl, not a runtime dep; `embroidery-ts`; custom). - Lock the algorithm choices (k-means vs flood-fill for P1; skeleton algorithm for P3; direction optimization for P4). - Validate the Brother PR-10 constraints (hoop size, max stitch length, needle count, thread palette source). - Estimate the polygon simplification tolerance that balances output size vs visual fidelity. A `blocked:needs-decision` should appear on P1 if the spike isn't complete.
Mark commented 2026-06-22 15:00:48 +00:00
Author
Owner
Copy link

Auto-triage: needs decompose

Marked triage-approved + needs-decompose. decompose-executor will pick this up and break it into implementable sub-issues.

## Auto-triage: needs decompose Marked `triage-approved` + `needs-decompose`. decompose-executor will pick this up and break it into implementable sub-issues.
Mark changed title from Implement real auto-digitize engine to Decompose: Implement real auto-digitize engine 2026-06-22 15:28:02 +00:00
Mark commented 2026-06-22 15:28:19 +00:00
Author
Owner
Copy link

This issue has been decomposed into the following sub-issues:

# Title Phase Scope
#6 Decompose: Implement real auto-digitize engine — Phase 1: Im Goal: Convert a raster input into a ColorRegion[] of axis-aligned-ish poly
#7 Decompose: Implement real auto-digitize engine — Phase 2: Re Goal: For each ColorRegion from P1, assign a StitchType (`'satin'
#8 Decompose: Implement real auto-digitize engine — Phase 3: Sa Goal: For each region tagged satin in P2, generate the parallel stitch str
#9 Decompose: Implement real auto-digitize engine — Phase 4: Ta Goal: For each region tagged tatami-fill in P2, generate parallel stitch r
#10 Decompose: Implement real auto-digitize engine — Phase 5: Th Goal: Map each detected region's color to the closest Brother thread catalog

Each sub-issue has been labeled with triage-approved and is ready for queue routing.

This issue has been decomposed into the following sub-issues: | # | Title | Phase Scope | |---|-------|-------------| | #6 | Decompose: Implement real auto-digitize engine — Phase 1: Im | **Goal:** Convert a raster input into a `ColorRegion[]` of axis-aligned-ish poly | | #7 | Decompose: Implement real auto-digitize engine — Phase 2: Re | **Goal:** For each `ColorRegion` from P1, assign a `StitchType` (`'satin' | 'tat | | #8 | Decompose: Implement real auto-digitize engine — Phase 3: Sa | **Goal:** For each region tagged `satin` in P2, generate the parallel stitch str | | #9 | Decompose: Implement real auto-digitize engine — Phase 4: Ta | **Goal:** For each region tagged `tatami-fill` in P2, generate parallel stitch r | | #10 | Decompose: Implement real auto-digitize engine — Phase 5: Th | **Goal:** Map each detected region's color to the closest Brother thread catalog | Each sub-issue has been labeled with `triage-approved` and is ready for queue routing.
Mark closed this issue 2026-06-22 15:28:19 +00:00
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
No results found.
Labels
Clear labels   
blocked:commit-ready-failed

   
blocked:gate-fetch

   
blocked:gate-lint

   
blocked:gate-plan

   
blocked:gate-research

   
blocked:needs-attention

Worker halted; needs human/janitor inspection

  
blocked:needs-decision

   
blocked:needs-human

   
blocked:no-changes

   
blocked:verify-failed

   
bug

   
deferred

   
feature-request

   
in-progress

   
needs-decompose

   
needs-triage

   
quickfix

   
security/medium

   
triage-approved
No labels
blocked:commit-ready-failed
blocked:gate-fetch
blocked:gate-lint
blocked:gate-plan
blocked:gate-research
blocked:needs-attention
blocked:needs-decision
blocked:needs-human
blocked:no-changes
blocked:verify-failed
bug
deferred
feature-request
in-progress
needs-decompose
needs-triage
quickfix
security/medium
triage-approved
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
Mark/Kimi_Agent_Brother_10_Digitizing_App#3
No description provided.