Your Vault Needs a Compiler and an Audit Trail

Ben Holmes moves the cost of organization from capture to compilation: people record rough observations, and agents later add metadata, links, and focused views. That inversion works only when the rough notes remain canonical and every generated page remains traceable and disposable.

Ben Holmes, Warp, LLM Knowledge Bases: a practical guide, published by AI Engineer. YouTube source I3bpdgFJCUY.

The morning paper is the least interesting part

Ben Holmes's most appealing image is a wiki that refreshes while he sleeps. The next morning, his notes have been linked, classified, and reorganized around whatever subject he cares about. "It's like the daily paper," he says, "but it's your own."

A personal morning paper sounds delightful. It also raises the question newspapers spend their whole existence trying to answer: why should the reader trust what made the page?

Holmes's real contribution is not Hubble, Warp, Oz, or the graph he generates near the end. It is a clean division of labor. The human notices things and records them with as little friction as possible. Agents perform repetitive transformations: add metadata, recover links, search for related notes, and compile focused views. The raw folder is source code; the wiki and graph are builds.

Capture can be permissive. Synthesis cannot.

That distinction is the difference between a useful knowledge pipeline and a vault that launders rough thoughts into confident prose.

Move organization downstream

Holmes starts with an ordinary directory of Markdown. Any viewer can open it: Warp, Obsidian, or something else. The sequence then moves through enrichment, backlinks and tags, focused wikis, and finally visualization. The order matters. A graph is not the knowledge base. It is the last derivative of several earlier transformations.

Voice dictation makes the intake side deliberately loose. Holmes says it can reach "like 200 words per minute," an approximate claim rather than a benchmark (03:08-03:31). He presents Handy as an on-device option and describes Voice Ink, his preferred hotkey-and-mobile option, as charging roughly a $20 lifetime fee (03:34-04:15). A long note about the founding of Walt Disney began as one uninterrupted spoken reaction after an Acquired episode. Paragraphs and polish came later.

"Don't worry if you're being a little bit scrappy, a little bit rambly," he advises. His generated wikis and visualizations need enough source material to synthesize; volume alone does not guarantee meaningful patterns.

This reverses the tax imposed by many second-brain systems. They ask a person to choose folders, tags, links, and a durable title at the exact moment an idea is most fragile. Holmes postpones that work. The trade is sensible, provided the raw note preserves what kind of thing each sentence is. A verbatim quote, a half-remembered claim, and the author's own speculation should not become indistinguishable merely because all three arrived through dictation.

The enrichment pass is the real product

Holmes demonstrates an enriched note about Ferrari. The agent adds four different things: topic tags, a source URL, an enrichment timestamp, and backlinks. Those operations are easy to blur together, but they solve different problems.

The tags classify. The URL attempts to establish provenance. The timestamp acts as cross-run memory, telling a later agent that some agent already processed the file. The backlinks connect the note to nearby material through local keyword searches. Web tools such as Exa can help recover the likely source; file operations handle the local corpus.

The best line in the talk arrives when Holmes explains his taxonomy. Allowed tags live in a reference folder, and the agent is told to be reluctant to invent new ones because "Claude loves to get creative." That is not a throwaway joke. Automation compounds entropy as efficiently as it compounds order. Without a closed vocabulary, one recurring customer problem can become flaky-tests, test-flakiness, unstable-e2e, and six other categories that never meet again.

The timestamp is useful but too weak for a durable pipeline. Holmes alternates between finding notes that are "not enriched" and finding notes that are "not tagged." Those are not the same state. A note may have tags but no verified source. It may have been edited after enrichment. The prompt or schema may have changed. A prior run may have failed halfway through.

A found URL is not automatically the source. Keyword overlap is not automatically a meaningful relation. Typed relations belong on claims and evidence links, not in the job-status record. The safe move is to let the agent propose clerical structure while keeping uncertainty visible.

A wiki is a query-shaped projection

Holmes's notes on The Left Hand of Darkness show that backlinks may already be enough. Separate chapter reactions become a "Wikipedia rabbit hole of your own thoughts" without a generated encyclopedia sitting between reader and note.

When he does build wikis, he credits an Andrej Karpathy gist and supplies a focus: take a raw directory and compile it around "whatever focus area that we care about." The resulting pages can organize people, concepts, sources, and organizations. Holmes demonstrates a Bible-in-a-Year view for characters whose names he forgets, and an AI wiki that happens to surface jazz musician Adam Neely because Neely's commentary on AI and music entered the corpus.

The focus prompt is not a neutral lens. It determines what the wiki includes, how it groups entities, and which contradictions disappear into summary. A customer wiki may be excellent for locating prior meetings and source links. It becomes dangerous when an agent merges a customer's exact wording, Holmes's interpretation, and a web result into one smooth paragraph without claim-level lineage.

Generated pages are best treated as query results that happen to be readable. Rebuild them. Throw them away. Never make them the only surviving record of what was observed.

Background execution changes the threat model

The automation is a sync-mutate-sync loop. Markdown moves into a sandbox. An agent finds pending notes, performs enrichment or regenerates a wiki, writes diffs, and syncs the vault back. Holmes uses the Obsidian headless CLI and suggests Git as another transport. Local scheduled jobs are possible, but a laptop must remain open. Oz moves the work to the cloud and can start from a schedule or triggers such as Slack and iMessage.

Moving slow jobs out of the foreground is genuinely useful. It also creates a larger system than the demo's smooth morning experience suggests. The laptop and cloud worker can edit the same file. A failed run can leave half an enrichment. A new model or prompt can rewrite metadata even when no source changed. Meeting notes may contain customer details that were safe during local dictation but should never enter a remote sandbox.

Start locally and write generated pages to a separate directory. Once that is reliable, run enrichment as reviewable Git patches rather than silent rewrites. Move a job to the cloud only after confidential paths are excluded and rollback has been tested. A fresh build proves that the job ran, not that its conclusions are true.

The graph is an interface, not evidence

Near the end, Holmes asks an agent to produce an HTML and Tailwind graph of the notebook rather than installing a dedicated graph product. Nodes and edges reveal clusters, orphan notes, and capture habits. Holmes suggests they can expose "gaps in your thinking."

The graph can show gaps in the recorded and generated structure. It cannot show what the market believes, what the reader failed to notice, or whether an agent linked two notes for a bad reason. A beautiful dense cluster may be nothing more than repeated keywords and duplicated source material. Visualization should come last and be trusted least.

Why this matters for Diffie

Anand's vault already contains pieces of the system Holmes describes. Source notes under Sources/X carry URLs, original text, summaries, tags, and backlinks. Daily notes already contain machine-delimited automation sections and Diffie activity. The missing layer is not another general wiki. It is an evidence model that turns scattered product and market observations into falsifiable decisions.

The folders should reflect that separation. Voice notes, call transcripts, X posts, account research, and telemetry receipts remain canonical. Agents normalize them into evidence records and propose typed links. Claims state something that can be supported or contradicted. Experiments specify a segment, message or product change, success metric, sample, and end condition. Weekly reviews and account dossiers are generated views and can be rebuilt.

For the first version, each evidence record needs only a source and locator, observation date, account or speaker, exact wording, and one status: observed, inferred, or unresolved. Add persona, workflow, pain, and trigger labels only when a recurring decision requires them. Claims, not raw notes, should link supporting and contradicting records. New taxonomy terms can wait until existing labels repeatedly fail retrieval.

The distinction between telemetry and customer evidence is especially important for Diffie. Repeated failed anonymous browser runs are product telemetry, not evidence that frontend engineers need Diffie. Record enough to reproduce the failure: tenant class, target environment, failure stage, and run IDs. Then seek customer evidence about who experiences the problem and what it costs them. If a frontend engineer later describes the same failure as costly, and account research confirms the relevant workflow, the system can propose a scoped ICP claim. That claim can produce an outbound message using the engineer's language and a reproducible product evaluation.

Use one weekly decision loop:

1. Choose one disputed claim, such as: "Frontend teams running Playwright in CI lose release time to flaky browser tests."
2. List the strongest supporting record, the strongest counterexample, and whether the claim includes direct customer language. If it has only telemetry or web research, the next action is an interview, not an ICP conclusion.
3. Select one test and hold the other variables steady: either test the segment and message with outbound, or test Diffie against a reproducible customer scenario.
4. Before running it, write the metric, threshold, deadline, and what each result would cause you to do.
5. At review, continue the claim, narrow its segment, investigate a product failure, collect missing evidence, or pause it.

Interpret the result conservatively. Customer confirmation plus a failed product evaluation points to product work. A successful evaluation plus outbound silence leaves the message or cohort unresolved; it does not disprove the pain. Change one variable in the next test, and retain the result as counterevidence rather than smoothing it into the weekly summary. This is where Holmes's analogy breaks: a background agent can compile text, but it cannot decide what silence means. That judgment remains founder work.

Automate clerical confidence, preserve human doubt

Holmes has the right inversion: capture first, compile later. His fixed tag list and enrichment timestamp are more important than the graph because they turn a vague assistant into a repeatable process.

The next step is to distinguish canonical evidence from generated interpretation, give relations meaning, version the enrichment state, and make every clean summary traceable to the messy thing that produced it. The vault should wake up organized. It should never wake up certain on Anand's behalf.