Add post-recording diarization support via cloud STT with speaker labels #18

New issue

Closed

opened 2026-05-23 01:21:47 +00:00 by Mark · 2 comments

Mark commented

2026-05-23 01:21:47 +00:00

Owner

Parent Issue

#6 — No speaker/subject assignment during live recording
#11 — Implementation plan

Description

Per the speaker diarization research (#10), the recommended path is Google Cloud STT batch Recognize with diarization enabled. After a recording completes, the audio can be sent to Google Cloud STT to get speaker-labeled segments, which are then written as SessionTag entries.

Scope

Add a "Diarize" button on TranscriptDetailScreen (post-recording action)
Implement GoogleDiarizationClient that sends recorded audio to Cloud STT batch Recognize with enableSpeakerDiarization=true
Parse the response: extract speakerTag per word, aggregate into speaker time-ranges
For each detected speaker:
1. Create a Speaker entity (named "Speaker 1", "Speaker 2", etc. if not matched to existing)
2. Write SessionTag(tagType=TYPE_SPEAKER, referenceId=speaker.id, startOffsetMs, endOffsetMs) entries
Show a progress indicator during diarization
After completion, update the transcript display with speaker labels
Fall back gracefully if no network or API key not configured

Acceptance Criteria

"Diarize" button appears on TranscriptDetailScreen for completed recordings
Clicking it sends audio to Google Cloud STT with diarization enabled
Progress indicator shows during processing
Detected speakers are created as Speaker entities
SessionTag entries with TYPE_SPEAKER are written with correct time offsets
Transcript display updates to show speaker labels after diarization
Graceful error handling for missing API key or network issues

Dependencies

Google Cloud API key configured in BackendConfig
SessionTag time-range infrastructure from #12

## Parent Issue #6 — No speaker/subject assignment during live recording #11 — Implementation plan ## Description Per the speaker diarization research (#10), the recommended path is Google Cloud STT batch Recognize with diarization enabled. After a recording completes, the audio can be sent to Google Cloud STT to get speaker-labeled segments, which are then written as SessionTag entries. ## Scope - Add a "Diarize" button on TranscriptDetailScreen (post-recording action) - Implement `GoogleDiarizationClient` that sends recorded audio to Cloud STT batch Recognize with `enableSpeakerDiarization=true` - Parse the response: extract `speakerTag` per word, aggregate into speaker time-ranges - For each detected speaker: 1. Create a `Speaker` entity (named "Speaker 1", "Speaker 2", etc. if not matched to existing) 2. Write `SessionTag(tagType=TYPE_SPEAKER, referenceId=speaker.id, startOffsetMs, endOffsetMs)` entries - Show a progress indicator during diarization - After completion, update the transcript display with speaker labels - Fall back gracefully if no network or API key not configured ## Acceptance Criteria - [ ] "Diarize" button appears on TranscriptDetailScreen for completed recordings - [ ] Clicking it sends audio to Google Cloud STT with diarization enabled - [ ] Progress indicator shows during processing - [ ] Detected speakers are created as Speaker entities - [ ] SessionTag entries with TYPE_SPEAKER are written with correct time offsets - [ ] Transcript display updates to show speaker labels after diarization - [ ] Graceful error handling for missing API key or network issues ## Dependencies - Google Cloud API key configured in BackendConfig - SessionTag time-range infrastructure from #12

Mark referenced this issue

2026-05-23 01:27:25 +00:00

No speaker/subject assignment during live recording #6

Mark added the

blocked:needs-decision

label

2026-05-23 16:43:03 +00:00

Mark commented

2026-05-23 16:43:03 +00:00

Author

Owner

⏸ Needs Decision

Blocking factors:

Didn't meet auto-approve criteria — scope=medium, risk=low, spec=aligned

Scope: medium Risk: low Spec: aligned Type: feature
Summary: Adds a 'Diarize' button on the transcript detail screen that sends recorded audio to Google Cloud STT for speaker diarization, creating Speaker entities and SessionTag entries with time ranges.
Suggestion: Create GoogleDiarizationClient using existing OkHttp to call Google Cloud STT speech:recognize with enableSpeakerDiarization=true. Audio file path may need to be stored on RecordingSession or derived from a convention. Parse the response grouping consecutive words by speakerTag into time ranges, create Speaker entities named 'Speaker N', and write SessionTag entries via existing repositories. Extend the ViewModel with a DiarizationState (Idle/Processing/Done/Error) and add an IconButton to TranscriptDetailScreen's top bar that triggers the coroutine. Reuse the cloud API key from BackendSettings or add a dedicated Google Cloud field.
Rationale: Medium scope because it adds a new service module and UI wiring but leverages existing Room entities (Speaker, SessionTag), DAOs, repositories, and OkHttp — no schema changes needed since SessionTag with TYPE_SPEAKER and time-offset infrastructure already exists from #12. Low risk because it's purely additive with no changes to recording/transcription paths, and graceful error handling is specified. Fully aligned with the project's roadmap (parent issues #6, #10, #11). The only open question is how the recorded audio file is referenced post-recording — the service currently doesn't persist the audio path, which may add minor scope to store it on the session.

Action: Review each blocking factor above. Remove blocked:needs-decision label to route back to triage.

## ⏸ Needs Decision **Blocking factors:** - **Didn't meet auto-approve criteria** — scope=medium, risk=low, spec=aligned --- **Scope:** medium **Risk:** low **Spec:** aligned **Type:** feature **Summary:** Adds a 'Diarize' button on the transcript detail screen that sends recorded audio to Google Cloud STT for speaker diarization, creating Speaker entities and SessionTag entries with time ranges. **Suggestion:** Create GoogleDiarizationClient using existing OkHttp to call Google Cloud STT speech:recognize with enableSpeakerDiarization=true. Audio file path may need to be stored on RecordingSession or derived from a convention. Parse the response grouping consecutive words by speakerTag into time ranges, create Speaker entities named 'Speaker N', and write SessionTag entries via existing repositories. Extend the ViewModel with a DiarizationState (Idle/Processing/Done/Error) and add an IconButton to TranscriptDetailScreen's top bar that triggers the coroutine. Reuse the cloud API key from BackendSettings or add a dedicated Google Cloud field. **Rationale:** Medium scope because it adds a new service module and UI wiring but leverages existing Room entities (Speaker, SessionTag), DAOs, repositories, and OkHttp — no schema changes needed since SessionTag with TYPE_SPEAKER and time-offset infrastructure already exists from #12. Low risk because it's purely additive with no changes to recording/transcription paths, and graceful error handling is specified. Fully aligned with the project's roadmap (parent issues #6, #10, #11). The only open question is how the recorded audio file is referenced post-recording — the service currently doesn't persist the audio path, which may add minor scope to store it on the session. **Action:** Review each blocking factor above. Remove `blocked:needs-decision` label to route back to triage.

Mark added

triage-approved

and removed

blocked:needs-decision

labels

2026-05-23 18:04:00 +00:00

Mark commented

2026-05-25 17:14:08 +00:00

Author

Owner

Fixed in commit. Closing.

Mark closed this issue