I Replaced the Vector Database with a Folder of Markdown Files

Part four of The Corporate Second Brain, a six-part series. The full map is at the end.

When a company decides to build an AI knowledge base, the first thing a vendor sells them is a vector database. It is presented as the obvious, grown-up choice. Embeddings. Semantic search. The whole apparatus.

For a lot of companies, it is the wrong first move. I have started telling teams to build the first version out of a folder of plain markdown files and see if they ever actually outgrow it. Most do not as fast as they think.

I am not the only one saying this out loud. Andrej Karpathy, who helped build the AI everyone is now buying, published his own personal knowledge base earlier this year. Roughly a hundred sources. Around four hundred thousand words. No vector database. No embeddings. Just markdown files an AI reads and maintains, navigated the same way a developer navigates a codebase.

What the folder approach actually is

The idea is almost offensively simple.

You keep the raw sources in a folder, and the AI is never allowed to edit them. They are the record. On top of that, the AI writes plain summaries, one page per topic, with links between related pages. There is one index file that lists everything, and one log file that records what came in and when. When you want an answer, the AI reads the index, follows the links to the right pages, and reads them. That is the whole system. No specialty database. The search is the same kind of plain text search a programmer has used for forty years.

The unlock is that the AI does the reading once, up front, and writes down what it learned in files a human can open and correct. It is not re-deriving everything from scratch on every question. It is maintaining a wiki, and the wiki is just files.

Why plain text keeps winning

You can read it. Open any file and see exactly what the AI believes and where it got it. When it is wrong, you fix a sentence in a document. Try that with a table of numbers inside a vector database.

The audit trail is free. Version control already tracks every change to every file. Who wrote it, when, and what it replaced. In a regulated shop, that history is worth more than the fanciest retrieval.

Permissions come from the folders. This is the thread from earlier in the series. The files carry their own access through the operating system you already run. There is no second copy of the rules to drift out of sync.

It is cheap and it is portable. A folder of text is a folder of text. It runs anywhere, it costs almost nothing to store, and no vendor owns your knowledge.

The objection I always get

Someone always says it, and it is a fair point. "Plain text search only finds the words you typed. A vector database finds meaning. If I search for 'time off' it will never find the doc titled 'PTO policy.' You need embeddings for that."

True, and smaller than it sounds. Two things close most of that gap. First, the AI is doing the searching, not a search box. Ask it about time off and it already knows to also look for PTO, leave, vacation, and holiday, because knowing that words mean the same thing is exactly what a language model is good at. It runs a handful of plain searches and reads what comes back. Second, the summary pages carry their own vocabulary. When the AI writes the PTO page, it names the synonyms in the text, so the next plain search finds it anyway. You are not relying on the raw filename matching the raw question. You are relying on a model that reads. The case where this genuinely breaks is real, and it is the next section, not the common case the objection imagines.

The honest part: when a folder is not enough

I am not going to oversell this, because the internet is already full of people who will.

A folder of files is the right call for a focused, mostly-stable body of knowledge, on the order of dozens to a few hundred documents, where the topics are clear enough that a person could roughly organize them. That describes a surprising number of companies, and almost every team, department, or product inside a larger one.

It is the wrong call when you have millions of documents nobody can pre-organize, or when the core need is fuzzy similarity search across a giant, messy pile, "find me anything that feels related to this," at a scale no human index could keep up with. That is exactly what vector databases are good at, and when you genuinely have that problem, use one. Karpathy himself adds a bit more machinery once his own base grows past a few hundred pages. The line is real. It just sits much further out than the vendors want you to believe.

And I will not tell you the folder is more accurate than a vector system for high-stakes work. That is not established, and anyone claiming it is getting ahead of the evidence. The case for files is simpler than that. It is legibility, auditability, and control, at a scale most companies actually live at.

What to do this week

1. Count your real corpus. Not everything. The documents that would answer eighty percent of the questions your team actually asks. If that number is in the hundreds, you probably do not need a vector database yet.
2. Stand up a folder. One directory, plain markdown, tracked in version control. Drop in your top twenty documents. Point an AI assistant at it and ask it real questions.
3. Notice what breaks. If plain text search is finding what you need, you have your answer. If it genuinely is not, now you know precisely why you need something heavier, instead of buying it on faith.
4. Delay the database until it earns its place. Make the fancy tool prove it is solving a problem you actually have.

The reflex to reach for the most sophisticated tool first is the expensive habit here. Start with the folder. It is legible, it is yours, and for more companies than the market admits, it is enough.

Next up, what all of this plumbing is actually for, which is not answering questions. It is closing the gap between what the company knows and what leadership does about it on Monday.

---

The Corporate Second Brain, a six-part series

1. Your Company Already Has a Second Brain
2. RBAC Is the Hard Part of AI
3. Why Ingestion Is the Whole Ballgame
4. I Replaced the Vector Database with a Folder (this post)
5. From What We Know to What You Should Do
6. Build a Corporate Second Brain in a Weekend