AI-enhanced PKM and agent memory: what to automate, and what not to

Obsidian and Logseq both have large plugin ecosystems, and both move fast enough that any list of features is stale within a release or two. Sorting the plugins into classes is more durable than naming them, because the classes have been stable for two years even as the individual projects have changed considerably.

• Chat over the vault. The plugin indexes your notes as embeddings, retrieves the closest passages when you ask something, and answers with citations back into your own files. Obsidian Copilot is the best-known example. This is ordinary retrieval-augmented generation pointed at a folder you own, and it can be run against a local model or a hosted one. The mechanics are the subject of local RAG.
• Semantic neighbours. The plugin embeds every note and shows the nearest ones in a pane while you write, without you asking a question. Smart Connections is the long-running example. Its value is surfacing the note you forgot you wrote; its failure mode is a pane full of notes that merely share a topic, which is not the same thing as being relevant.
• Generation in the editor. A prompt, and output inserted at the cursor. This is the least interesting class for knowledge management and the most likely to fill a vault with text nobody has read.
• Maintenance. Tagging, linking, summarising, splitting, deduplicating, renaming. This is where the leverage is, and also where the failures are quietest, because nothing looks broken afterwards.

Logseq's ecosystem covers the same four classes, and because Logseq is block-based rather than page-based, retrieval there operates on blocks, which suits a journal of daily notes better than a page-level index does. Most plugins in both tools can be pointed at an OpenAI-compatible endpoint, so they will talk to a local model served by Ollama as happily as to a hosted API. Check the plugin directory for what a given plugin does today rather than trusting an article, including this one.

Automatic tagging is the most requested maintenance job and the one most likely to disappoint in a way you do not notice for months. It genuinely helps in three situations: high-volume material you did not write yourself (clipped articles, transcripts, exports), routing rather than meaning (which project a note belongs to, whether a note contains a decision), and backfilling a vocabulary you have already designed across a few thousand untagged notes.

The failure mode is specific. A model asked to tag a note will always return tags, and the tags will always look right, because they accurately describe the note's surface. But a tag is not worth anything for being descriptive. It is worth something for partitioning: for splitting the base into a part you want and a part you do not. Model-generated vocabularies fail that test in four recurring ways. They drift into synonyms, so #ml, #machine-learning and #ai all end up in use and none of them retrieves the whole set. They produce universals, and a tag applied to sixty percent of your notes carries no information. They duplicate full-text search, tagging a note about pricing with #pricing when the word pricing is already in the note. And they never abstain, because a model will not tell you that a note needs no tag at all.

Four rules make the dependable half worth having. Give the model a closed vocabulary by pasting your tag list into the prompt and requiring new tags to arrive as proposals rather than as edits. Tag for retrieval intent rather than subject, because #decision, #open-question and #commitment earn their place where #marketing does not. Audit by counts rather than spot checks: any tag on more than a quarter of the base is noise, any tag with a single note is a typo or a synonym, and both lists are one command away in a folder of plain text. And keep the model out of the delete path, because a wrong tag costs one bad query while a removed right one costs a note you will never find again.

Progressive summarisation is Tiago Forte's distillation technique, the D in his CODE workflow (capture, organise, distill, express). You save a passage, and on each later encounter with the note you compress it one layer further: highlight the passages that matter, bold the best of the highlights, and eventually write a few lines of your own at the top. The layers accumulate over time rather than in one sitting. The method sits inside the wider system described in PARA and CODE.

A model changes the economics completely: it can produce the executive-summary layer for every note in the base, at ingest, in the time it takes to read one of them. For anyone with three hundred unread clippings that is an obvious trade, and worth taking for part of the job. But two things break if the model does all of it.

The first is that the method is lazy on purpose. Compression happens only when you return to a note, so the effort lands exclusively on the notes you revisited, and the layers double as a record of which ones those were. Summarise everything at ingest and that signal disappears: every note is equally distilled, and the base can no longer tell you what you use. The second is that a summary is only meaningful relative to a question. A model summarising a research paper writes what the paper is about; what you want recorded is what it changes about the thing you were going to do next, and the model does not know what you were going to do.

The split that works is mechanical layer to the model, judgement layer to you. Let it pull every highlighted passage into one block, write a neutral abstract, list the named entities and dates, and enumerate the open questions the source raises. Then write one sentence yourself: what this changes, or which note it contradicts. That sentence is worth more than three hundred words of generated prose, and it is the only part of the note that could not have been reconstructed from the source.

Agent designers have converged on a four-part vocabulary borrowed from cognitive psychology: working, episodic, semantic and procedural memory. It is worth adopting not as neuroscience but as a checklist, because it forces the two questions a knowledge base usually leaves implicit, namely what gets kept and how each kind gets written.

The payoff is that the four layers want different write policies, and most knowledge bases fail by applying one policy to all of them. Episodic material is append-only and must never be rewritten, because its entire value is being an unaltered record of what was said; the moment an agent tidies a transcript you can no longer use it as evidence. Semantic material is the opposite: a page about a concept should be replaced when your understanding changes rather than versioned into a thread of amendments, which is the same discipline evergreen notes have always asked for, described in Zettelkasten and evergreen notes. Procedural material changes rarely and should change by hand, since an error in the file that tells the agent how to maintain the base is applied to everything the agent touches afterwards. Working memory is thrown away every session, so the only design question there is what gets loaded into it.

Andrej Karpathy's LLM Wiki pattern, published as a gist in September 2024, implements three of these four directly: an immutable raw/ folder, an agent-maintained wiki/ folder, and an AGENTS.md telling the agent how to keep the second in step with the first. That correspondence is covered in detail in the LLM Wiki. You do not need an agent for the structure to be worth adopting: separating an immutable dated layer from a rewritten conceptual layer is good practice in a base only a human will ever touch.

• Compilation instead of retrieval. Classic RAG answers by fetching passages at query time from a corpus nobody has cleaned up, so every query pays for the noise in the raw material. The alternative is to pay once at ingest: an agent reads the new source, updates the concept pages it affects, and queries then run over a corpus that has already been synthesised. The cost is that compilation is a write, and a write can be wrong in ways retrieval cannot, which is why the sensible shape keeps both, with the raw layer as a fallback when a compiled page looks suspect.
• Models small enough to run locally. Each generation of open-weight models raises the quality obtainable on a laptop, embedding models have been cheap enough to run on CPU for a while, and Ollama and llama.cpp have reduced the setup to a couple of commands. This matters more for a knowledge base than for most applications, because a base assembled from your employer's meetings is precisely the material you do not want to send anywhere, an argument set out in local-first AI at work.
• Multimodal notes. Whiteboard photographs, diagrams, screenshots and recorded audio can now be turned into searchable text, so material that used to be dead weight in a vault joins the corpus. Treat the transcription or the OCR output as a raw-layer artefact rather than as a note: keep it, index it, and do not mistake it for something you wrote.
• The agent as curator rather than author. The maintenance jobs nobody does by hand (finding orphan pages, spotting duplicates, flagging pages that contradict a newer source, listing tags used once) are exactly what a scheduled agent is good for. A nightly job that reports problems is more useful than one that fixes them silently, because the report is reviewable and the silent fix is not.

All of this assumes the agent can read the material. For Markdown in a folder that is trivial: give a coding agent filesystem access and the vault is just files, which is a large part of why plain text keeps outlasting the applications built on top of it. The awkward case is a source that is not already a folder you keep, such as a mail archive, an issue tracker, or a year of meetings living inside some other application.

The Model Context Protocol, published by Anthropic in November 2024, is the usual answer to that: a source runs one MCP server, and any MCP-capable client can query it, instead of every client needing its own integration. Earkeep is one example of a corpus wired up that way: a local-first meeting transcriber that keeps every transcript as plain files on your own device and exposes them over an MCP server, so an agent can read a year of meeting history without a bespoke integration being written for it. The same shape applies to any other source you want an agent to remember from.

The test to apply to any such bridge is what remains if the server stops. If the answer is a proprietary database, you have acquired an integration rather than a memory. If the answer is a folder of files you can still open and grep, the protocol is a convenience rather than a dependency, which is the only arrangement worth building a decade of notes on.