Local-first AI at work

Last updated: ยท Reviewed quarterly

Local-first AI at work means three things working together: speech-to-text models that run on your own machine instead of a vendor's API, a plain-file record of what happened that any tool can read, and a connective layer like MCP that lets your agents reach that record without a cloud round-trip. Apple Silicon is fast enough, open ASR models are accurate enough, and open protocols exist to wire it together, so local by default is now a realistic architecture choice rather than a workaround.

Why local-first, not just "privacy-friendly"

Privacy is a property of the pipeline, not a setting

A tool can call itself privacy-friendly while still shipping your audio to a third-party transcription API before it ever reaches you. The distinction that matters is architectural: does the sensitive input ever leave the device, or does it merely get deleted afterward on someone else's server. Local-first means the first is structurally impossible, not merely promised in a privacy policy you didn't write and can't audit.

Compliance gets simpler when there's no processor to name

Under GDPR and similar regimes, every third party that touches personal data (a name, a voice, a decision made in a meeting) is a processor you have to name, contract with, and account for. Audio processed only on-device and never transmitted to a transcription API doesn't create that relationship in the first place, because there's no transmission for a processor to receive. That's a narrower, checkable claim than "GDPR compliant," and it's the one worth verifying with your own counsel rather than taking on a vendor's word.

Ownership: your record shouldn't depend on a subscription

If your meeting history lives in a SaaS notetaker, it lives there on the vendor's terms for as long as you keep paying and the vendor keeps existing. Neither is guaranteed indefinitely. The alternative is a record that's yours by construction: files on your own disk, in formats you can open without the vendor's app, covered in depth in plain files beat platform lock-in.

Longevity: software should outlast its subscription

A decade is a reasonable planning horizon for something as durable as "what was decided in every meeting I've had." Cloud accounts get suspended, companies get acquired, free tiers get cut. A local record, in a boring format a text editor can already open, has no dependency on any of those events. This is the same bet plain-text config files and log formats have paid off on for decades.

The technical building blocks

Local speech-to-text has closed the gap on cloud APIs

Two open automatic speech recognition model families now make on-device transcription practical for real meetings, not just short clean clips:

  • Whisper, open-sourced by OpenAI in September 2022, covers 99+ languages and remains the broadest-coverage option for mixed-language or less common languages.
  • Parakeet-TDT, from NVIDIA, is tuned for speed: recent versions top the Hugging Face Open ASR Leaderboard on throughput while matching or beating Whisper's largest model on English accuracy, at the cost of a narrower language list.

Both run through whisper.cpp and comparable inference engines built to use a machine's own hardware, not a remote GPU cluster, which is what turns "transcribe on your laptop" from a toy demo into something that keeps up with a live conversation. See languages and models for the full coverage lists.

Apple Silicon made on-device inference the fast path

The unified memory architecture and Neural Engine in Apple Silicon Macs, launched from 2020, mean a meeting-length transcription job runs at a fraction of the time and power draw it would have taken on a comparable Intel Mac, hardware-accelerated rather than software-emulated. That single hardware shift is a large part of why local transcription went from a niche hobbyist workaround to something a mainstream desktop app can offer as the default.

Plain files: the format that doesn't need the app to survive

Once audio is transcribed, the resulting text is the durable artifact worth keeping, and the format matters more than most teams realize until they try to leave a tool. Line-delimited JSON and plain JSON are old, boring, and exactly what you want from something meant to last: readable by grep, cat, jq, or any text editor, appendable without special tooling, and resilient to partial reads if something crashes mid-write.

MCP: the connective tissue between your files and your agents

The Model Context Protocol, introduced by Anthropic in November 2024 as an open standard, gives AI applications a common way to connect to external data sources and tools instead of every vendor inventing its own integration. In practice a local meeting archive can expose itself once, as an MCP server, and any MCP-capable client can read from it without a bespoke plugin for each one. The separate Agent Client Protocol, developed by Zed Industries, does the complementary job of letting an agent run inside a host application with access to that application's own context. For the practical patterns this unlocks, see feeding meeting context to AI agents.

What a local-first stack looks like

  1. Capture and transcribe on-device. Audio in, text out, nothing transmitted in between. Earkeep records mic and system audio continuously, transcribes locally, and never writes the audio itself to disk. See private by design.
  2. Store as plain files, not a database only the app can read. Earkeep writes one running JSONL transcript per day, with the meetings you keep marked inside it, so the record survives independent of the app. See private by design for the file formats too.
  3. Connect agents through an open protocol, not a walled-garden assistant. Earkeep runs ACP agents against that local history, plus an MCP server for external clients. See agents and MCP.

None of these layers requires the others to come from the same vendor, which is the point: a local-first stack is built from interoperable pieces, not a single company's closed loop.

Getting started this week

You don't need to overhaul your whole toolchain to start. A reasonable first step: pick one recurring meeting type, 1:1s, sprint reviews, whatever repeats weekly, and run it through an on-device transcription tool for two weeks, keeping the plain-file output somewhere you'd already back up. Then point one agent you already use, in-app or via MCP, at that small archive and ask it something concrete, like drafting a follow-up from the actual transcript instead of your memory of the call. The habit generalizes once the mechanics are proven on a small, low-stakes slice of your week.

Related resources

Try it yourself

Earkeep is a small, fast desktop app that captures every meeting, transcribes it locally, and keeps it as plain files your agents can read. Download it free for 14 days, no account, no card required.