Skip to main content
Blog

Spool It to SQLite First: A Durable Audio Queue for Local Transcription

When local inference falls behind a live microphone, the backlog has to live somewhere. Put it in an array and a slow model becomes a memory leak; put it in a table and it becomes a row count you can restart into.

· Dev3lop Team

Durable audio spool architecture: a flushed segment inserted into a SQLite audio_jobs table as a BLOB, a single-flight pump ordered by id, VAD and Whisper, with cards for what lives in RAM, what lives on disk, and what happens when inference falls behind

Here is the bug that only shows up in a real meeting.

Your local transcription works beautifully for ten minutes. Then something else on the machine wants the GPU — a build, a video call, a model the user just loaded in another app — and Whisper starts taking six seconds to decode four seconds of audio. The microphone does not care. It keeps producing clips at real time. Your pendingClips array starts growing, and it is holding raw Float32Array buffers: 64 KB for every second of 16 kHz mono audio.

Twenty minutes later the tab is holding tens of megabytes of undecoded audio, the UI is stuttering because the garbage collector is working through a graveyard of large typed arrays, and when the browser eventually reclaims the tab, every one of those clips is gone. Not delayed. Gone. The user watched a recording indicator the whole time.

This is part three of a series on the local speech stack, and the fix is the oldest one in data engineering: write it down before you think about it.

Disk first, model second

The hero diagram is the entire architecture. A clip flushed by the segmenter goes straight into a SQLite table as a BLOB, along with its session, its model id and its timestamps. Only then does a pump pick it up.

CREATE TABLE IF NOT EXISTS audio_jobs (
  id INTEGER PRIMARY KEY AUTOINCREMENT, sessionId TEXT, modelId TEXT,
  speakersOff INTEGER, ts INTEGER, endTs INTEGER, audio BLOB
);
CREATE INDEX IF NOT EXISTS idx_audio_model ON audio_jobs(modelId, id);

The pump is single-flight and strictly ordered: SELECT * FROM audio_jobs WHERE modelId = ? ORDER BY id LIMIT 1. One clip in the pipeline at a time, oldest first. That is not a performance compromise — it is a correctness requirement, because a transcription model is a single loaded set of weights and two concurrent calls against it just fight.

The consequences are worth being explicit about, because they are the whole point:

  • RAM holds one clip. Whatever the backlog is, the heap does not grow with it.
  • A slow model produces a number, not a crash. Queue depth is visible above the workspace.
  • Failed clips stay on disk. Starting the queue again retries them in order; reloading the page and loading the same Whisper model resumes them.
  • The spool is a queue, not a recording. Queued audio is deleted the moment its transcript commits. This is not a design detail, it is the privacy posture — there is no full-meeting audio file anywhere on disk to leak.

And one guard rail in the other direction. Enqueue refuses to let more than eight writes be in flight at once and throws "Local storage cannot keep up with capture." If the storage layer genuinely cannot absorb the microphone, capture stops with an error the user can see. A queue that silently drops work is worse than one that stops.

The commit is one transaction, and that matters more than it sounds

Atomic commit for a transcription job: a single transaction inserting the utterance and deleting the queued audio, with five crash scenarios and what each one costs, including the honest limit where the initial insert itself fails

Transcribing a clip produces two facts that must move together: the transcript row now exists, and the queued audio is no longer needed. Doing those as separate statements gives you two ways to be wrong. Crash between them and you either have a transcript with its audio still queued — which will be transcribed again on the next start, duplicating the line — or audio deleted with no transcript, which is silent data loss.

So they are one transaction. BEGIN, insert the utterance, delete the job, COMMIT, with ROLLBACK on any throw. The invariant that buys: a transcript row and its queued audio can never both exist, and can never both vanish.

Read the five outcomes in that diagram as a durability spec rather than a list of disasters. Four of them are fine. Power cut after the insert into audio_jobs? The clip is queued, and the next start replays it in id order. Power cut during decoding? Nothing was deleted, so the same clip is claimed again. A throw inside the transaction rolls back to a job that is still intact. A crash after COMMIT is simply the correct end state arriving early.

The fifth is the honest limit, and every durability story needs one. If the initial INSERT never lands — storage full, OPFS handle lost — that clip is genuinely gone, and capture stops with an error rather than pretending otherwise. Audio still sitting inside the recorder’s current unflushed segment isn’t crash-durable either, and speaker centroids are not persisted, so a recovered session may rediscover speaker labels rather than reusing the old ones. Say where the edge is; don’t imply there isn’t one.

The number that makes the argument

Chart comparing audio held in RAM over ten minutes of capture at half real-time transcription speed: an in-memory array growing linearly to 19.2 MB while a SQLite spool stays flat at 0.9 megabytes, with a marker where background LLM work pauses at three pending clips

This is the whole case in one chart. Ten minutes of capture while transcription runs at half real time — a realistic bad afternoon, not a pathological one. The backlog grows at 30 seconds of audio per minute either way; the queue depth is identical in both designs. What differs is where those bytes sit.

In the array, five minutes of undecoded 16 kHz mono audio is 19.2 MB of live heap that a crash turns into nothing. In the spool, RAM holds the one clip being decoded — about 0.9 MB at the 14-second maximum — and the rest is rows you can restart into.

The dashed line is the other half of the design: at three pending clips, background language-model work pauses itself. Transcription outranks interpretation, always, because a summary of a meeting you failed to record is worth nothing. That gate and its four siblings are covered in the scheduling post.

None of this is novel. It is back-pressure handling and durable queueing applied to a workload most front-end code never thinks of as a pipeline. The only thing that makes it feel unusual is that the whole thing is running inside a browser tab — which is also why SQLite has to live in a worker, since the OPFS synchronous access handles it needs are worker-only. One tab owns the database; a second one gets a clear error instead of two writers quietly corrupting each other.

If you take one thing from this

Any time a producer you don’t control feeds a consumer whose speed you don’t control, you have a queue whether you designed one or not. The only question is whether it lives somewhere you chose. A microphone is the purest version of that problem: it produces at exactly real time, forever, and it will not slow down because your model is having a bad minute.

Speaker diarization flow: a finished clip embedded by a wespeaker resnet34 model into a 256-dimensional L2-normalized vector, compared by cosine similarity against every live centroid, with the best match adapting its centroid by exponential moving average

Next in the series: the clip has been transcribed, and now something has to decide who said it — with no enrollment, no voiceprints, and a speaker count that isn’t known when the meeting starts. That’s online diarization.

Series: the whole stack · the speech gate · this post · speaker diarization · local LLM scheduling

← All posts