Earkeep
LLM Wiki from meeting transcripts: Karpathy's pattern for personal knowledge
Last updated: ยท Reviewed quarterly
In September 2024 Andrej Karpathy published a gist describing a way to organize personal knowledge for an AI agent: raw sources in one folder, an agent-maintained wiki in another, and a schema file telling the agent how to keep the second in step with the first. It is a pattern rather than a product, and meeting transcripts turn out to be close to its ideal input.
Three layers
The structure is deliberately plain — two folders and a file.
| Layer | Holds | Who writes it |
|---|---|---|
raw/ | Immutable sources. Articles, exports, PDFs — and here, one transcript file per day. | The capture tool. Never edited by hand or by the agent. |
wiki/ | One page per concept: structured, cross-linked, rewritten as understanding improves. | The agent, from raw/, under your review. |
AGENTS.md | The schema: page conventions, how to name things, how to resolve contradictions, what not to record. | You. This is the file that determines whether the wiki is any good. |
The layer that carries the pattern is the middle one. A wiki page is not a summary of a source — it is a page about a concept, assembled from every source that touched it. "Pricing model" is one page fed by eleven meetings across seven months, rewritten each time something changed. That is the difference between a folder of meeting summaries and a knowledge base, and it is the thing no amount of retrieval over raw transcripts produces on its own.
The third layer is where most implementations succeed or fail. An AGENTS.md that just says "keep the wiki updated" produces sprawl. A useful one is specific: one page per concept and per project, titles that state claims, decisions recorded with their reasoning and the date, contradictions kept with both dates rather than silently overwritten, and every claim linked back to the transcript it came from. That last rule is the one that keeps the wiki checkable rather than merely plausible.
Meetings as the raw layer
Karpathy's gist assumes you are ingesting things you read. Meeting transcripts have three properties that make them unusually good raw material, and one that makes them demanding.
- They are generated continuously without effort. Most knowledge bases die because feeding them is work. A transcript arrives whether or not you had the energy.
- They contain what documents omit. The rejected alternative, the caveat, the constraint mentioned in passing — none of which reaches the written summary, and all of which is what you want in six months.
- They are dated and sequential by construction. Which gives the wiki a reliable timeline: when a position changed, and what was said just before it did.
- They are extremely noisy. Filler, cross-talk, digressions, the ten minutes about the weekend. Compression ratios are brutal — and that is an argument for the compile-ahead pattern rather than against it, because retrieval over raw transcripts returns exactly this noise.
The three operations
Ingest. New transcripts arrive; the agent reads them, decides which concepts they touch, and updates those pages — adding a decision, revising a claim that changed, creating a page for something that appeared for the first time. Run it daily or weekly. Weekly is often better: a week's transcripts together let the agent see that the same concern came up three times, which a per-meeting ingest cannot.
Query. Ask questions of the wiki rather than of the transcripts. "What have we decided about enterprise pricing and why?" is answerable from one page that was assembled for exactly that purpose. The retrieval is over a corpus that is small, clean, and already synthesized — which is why the answers are better even when the underlying model is the same.
Lint. The operation people skip, and the one that keeps a wiki from rotting. The agent walks the whole thing looking for duplicate pages, contradictions between pages, claims with no source link, stale decisions whose stated expiry conditions have been met, and orphan pages nothing links to. Monthly is enough. Treat its output as a report to review rather than as edits to accept.
Compile ahead, or retrieve at query time
Standard RAG embeds your documents, and at question time retrieves the most similar chunks and hands them to the model. It works, and against meeting transcripts it has specific weaknesses that are worth naming precisely.
| RAG over transcripts | LLM Wiki | |
|---|---|---|
| Corpus | Raw speech — noise, filler, repetition | Synthesized pages, one concept each |
| Across time | Returns chunks; can't tell you a position changed | The page holds the history and the supersession |
| Improvement | None — every query starts from the same corpus | Accretes; each ingest makes the next answer better |
| Inspectable | An embedding store you cannot read | Markdown files you can open and correct |
| Cost shape | Cheap to set up, paid per query | Expensive per ingest, cheap per query |
| Failure mode | Retrieves the wrong chunk; you see it and retry | Compiles a wrong claim; it persists until linted |
The last row is the honest cost, and it deserves more attention than the pattern's advocates usually give it. A retrieval error is transient and visible. A compilation error is durable: if the agent misreads a meeting and writes a confident wrong sentence into a page, that sentence is what you and every future query will read. This is exactly why the source-link rule and the lint pass are not optional decoration — they are what makes a compiled claim checkable against the transcript that produced it.
Note also that the two are not exclusive. Karpathy's own framing keeps retrieval in the picture: query runs over wiki/ rather than over raw/. The wiki improves the corpus; it does not replace search. See local RAG: querying your meeting history with AI for the retrieval half.
Building it on transcripts you already have
The pattern needs three things: a durable raw layer, plain files, and an agent with filesystem access. Earkeep supplies the first two by design. It records mic and system audio continuously while it runs and transcribes locally on your device — no bot to invite, no start/stop to remember — and writes one .jsonl file per day into a folder you choose. Meetings are defined afterwards by selecting a span on a timeline, or automatically from a connected calendar; audio is processed only in memory and never written to disk. That folder is raw/, unedited and append-only, which is precisely what the pattern asks for.
For the agent, Earkeep exposes an MCP server, so a local coding agent can read the transcripts directly and write the wiki alongside them — see agents and MCP and feeding meeting context to AI agents. Run the whole thing against a local model and no part of it involves a third party at all, which for a knowledge base assembled from your employer's meetings is not a small consideration: local-first AI at work.
A reasonable first version is two weeks of transcripts, an AGENTS.md of about twenty lines, and one ingest run — then read every page it produced. The pages will be wrong in instructive ways, and fixing the schema in response is the actual work of adopting this pattern.
Where this fits
The manual counterpart — Zettelkasten, PARA, notes you write yourself — is in building a personal knowledge base from meeting notes, and the end-to-end version is in from meeting to knowledge base. The durability argument for the raw layer is in plain files beat platform lock-in.
Give your agent something worth compiling
Download Earkeep free for macOS and start the raw layer today — the wiki can be built from it later.
14-day full trial, no account, no card required.