On-Device Meeting Transcription: The Whole Local Speech Stack

Every meeting recorder on the market has the same architecture drawn on the same whiteboard: capture audio in the browser, stream it to a GPU somebody else owns, stream text back. It works. It also means the most sensitive twenty minutes of your quarter — the compensation conversation, the acquisition that isn’t announced, the client who is about to leave — live in a queue on a machine you can’t see, governed by a retention policy you didn’t write.

We built the other one. Not as a thought experiment: as a working meeting workspace that records, transcribes, separates speakers, and summarizes, with the network cable unplugged. Everything below is from the source, not from a pitch deck.

This post is the map. The four that follow it are the hard parts in detail: the two-stage speech gate, the durable audio queue, online speaker diarization, and scheduling a local LLM under a live microphone.

Six stages, no round trip

The hero diagram above is the whole thing. Audio enters at stage one and never leaves the machine; what comes out the other end is a transcript, speaker labels, and model-written findings that cite the transcript.

1. Capture. getUserMedia for the microphone, getDisplayMedia for tab or system audio, or a specific input device — a virtual loopback like BlackHole, if you want true system-wide capture on macOS. Multiple sources are mixed into one signal in the audio graph, at unity gain for a single source (attenuating a lone microphone quietly wrecks the speaker embeddings downstream).
2. Segmenter. An AudioWorkletProcessor batches the render thread’s 128-frame quanta into 1024-sample blocks — about 64 ms at 16 kHz — computes RMS energy per block, and posts them to the main thread, which decides where one utterance ends and the next begins.
3. Speech gate. Each finished clip is screened by Silero VAD in a worker before a transcription model ever sees it.
4. Whisper. @huggingface/transformers running the ASR pipeline in a dedicated worker, on WebGPU where it exists and WASM where it doesn’t.
5. SQLite on OPFS. Both a durable spool for queued audio and the transcript store, in a worker, with transcripts committed atomically.
6. Local LLM. Ollama on 127.0.0.1:11434, reached through a same-origin proxy, producing terms, context, summaries and chat.

The interesting engineering is not in any single stage. It is in the boundaries between them — which thread each stage runs on, and what happens when one of them falls behind.

Here is the split that makes the whole thing feasible. The audio render thread has a hard real-time budget measured in milliseconds — block there and you drop a buffer, permanently. So it does arithmetic and nothing else: frame the samples, sum the squares, post the block. The main thread runs a silence timer and the UI; it decides when a clip ends and never decides what it says. Every model lives in its own dedicated worker, which is also where SQLite has to live, because the OPFS synchronous access handles that make SQLite fast are a worker-only API. And two things run outside the browser entirely: the Ollama daemon, and a small loopback Node service that owns the calendar database.

That layout is not aesthetic. It is what lets a 7B model think about your meeting while the same laptop is still recording it — a scheduling problem ugly enough to get its own post.

The privacy claim, stated precisely

“Local-first” is a marketing word now. So here is the version you can audit.

The left column is what the default gives you. Raw audio segments live in a local queue and are deleted the moment their transcript commits — the spool is a queue, not a recording. Transcript rows, speaker embeddings, model findings, contacts, projects and chat history are all in one browser-local SQLite database. Calendar events sit in a separate SQLite file on disk, owned by a loopback service with a private per-process token, so the model can’t write to it directly even if it decides it wants to.

The right column is the honest part. There are exactly four doors, and each one is a switch somebody has to flip:

• Browser speech recognition starts immediately when it’s supported, because it gives instant live words while Whisper’s model downloads. That service can process audio online — it’s a vendor feature, not ours — and it goes idle whenever local Whisper is running.
• The first model download. Weights come from a CDN once and are then served from the browser cache. After that, offline works.
• Web research is an explicit opt-in, and the rule there is narrow: queries leave the device, recordings never do. Model redaction of a query is best-effort, which is why the confidential case is a manual query.
• BYOK audio upload exists for people who want a hosted transcription API for one clip. Server-held key, explicit checkbox, explicit click, never triggered by Wi-Fi or a chat message.

One rule is enforced in code rather than documented in a README: the background intelligence pipeline refuses any model endpoint that isn’t the local proxy or loopback. A misconfigured VITE_OLLAMA_BASE_URL pointing at a remote host doesn’t quietly start shipping transcripts somewhere; it fails. This is the same instinct behind consent-management design that treats collection as a decision rather than a default, and the same reason data locality is a performance argument as well as a privacy one.

Why local is now a boring technical choice

Three things changed, and none of them were “models got smaller.”

The browser grew up. AudioWorklet gives you a real-time audio thread. WebGPU gives Whisper actual acceleration. OPFS gives SQLite synchronous file access, which means a real database rather than a pile of IndexedDB promises. Workers make every model a separate execution context with no ceremony.

Unified memory made mid-size models practical. A laptop with 32–64 GB can hold a 7B instruct model resident and a 1.5B model beside it, which is exactly the shape this workload wants: a small fast model for terms and a larger one for context.

The economics inverted. Cloud transcription bills per minute of audio, forever, for a workload that is embarrassingly parallel and perfectly local. The cost of shipping data to compute instead of compute to data is the same argument data engineering has been having for a decade — it just arrived in the audio stack late.

What the rest of the series covers

That figure is the subject of the next post, and it’s the best illustration of what building this actually feels like. The naive version of stage three is “run VAD before ASR.” The real version is two gates answering two different questions at two different costs, plus a deliberate decision about which way each one fails — because a speech gate that errors and silently drops audio is far worse than one that lets noise through.

The remaining posts follow the same shape. Spooling clips to SQLite before transcribing them turns a slow model from a memory leak into a row count. Online diarization labels speakers with 256-dimensional embeddings and no enrollment step at all. And scheduling local inference is where the whole thing either stays usable or turns your laptop into a space heater.

If you are building something in this space — internal tools, a product, or a compliance requirement that rules the cloud out — we do this work. The architecture is not exotic anymore. It is just a set of choices most teams haven’t had a reason to make yet.