I built myself a knowledge base, then refused to let it speak for me

I spent a few days feeding decades of my own writing into a system.

Blog posts, my master's thesis, self-interviews, theater pieces, reflections written with AI. Over 200,000 words, cleaned into structured material and turned into something you can talk to. It lives at saomin.tw/me—throw a question at it, and it answers in "my" voice.

Like tossing a stone into a valley and listening for the echo.

Once it was done, I did something that sounds contradictory.

I built this base, then decided not to let it speak for me the easy way.

The original approach chopped the voice into fragments

Technically, the original setup worked like this:

Cut each article into small chunks, turn them into strings of numbers (vectors), and store them in a database. You ask a question, the system finds the chunks "most similar" to it, pulls them out, stuffs them into a prompt, and asks Claude to assemble an answer from those fragments.

This method has a name—RAG—and it's an industry standard. It solves a real problem: there's too much text to fit into a single conversation, so you only feed in the relevant parts.

But I knew early on it had two flaws, and I'd written them down in my notes.

The first is technical: the model that judges "how similar" things are isn't great with Chinese. Every similarity score crowds into a narrow band, which means it can barely tell which chunk is actually more relevant. I thought retrieval was the thing holding the "don't make things up" line. In fact, the prompt was holding that line—it was me telling it, "if you never wrote it, say you never wrote it."

The second is far more fatal:What it pulls back are fragments, and a voice doesn't live in fragments.

The tone of a single sentence only holds up because of the sentences around it. Dig it out of the whole piece and the tone is damaged. The value of my twenty years of writing was never in some quotable line being searchable—it's in the rhythm of the whole piece: when to pause, when to leave a thought unfinished, when to suddenly turn. Fragments can't carry that.

An outsider's method forced me to figure out what I actually wanted

The shift was sparked by something I read.

Andrej Karpathy on using an LLM as a personal knowledge base: throw your messy raw material at the LLM, let it compile a clean wiki, and from then on you query the wiki. Some people in Taiwan are doing something similar with Obsidian plus Claude Code as a second brain.

Their key insight is very practical: at "small scale"—he gives an example of roughly 400,000 words—you actuallydon't need complex RAG. The LLM reads the index and reads whole articles directly, and that's enough.

That line went straight for the heart of my architecture. My engineering of chopping fragments and computing vectors might be completely unnecessary at this scale.

I nearly copied it wholesale. But I stopped to think, and realized one thing couldn't be copied.

Karpathy's approach treats the LLM as ametabolic engine for knowledge. It chews through the raw material, summarizes it, organizes it into a wiki—and a summary can replace the original, because once you've summarized the facts, the facts are still there.

But me.saomin isn't a knowledge base. It'sa voice, a container。

For a voice,a summary kills it. Summarize one of my articles down to three bullet points—every fact correct—but the "me" is gone. Tone is the entire reason it exists. A summarized "me" is a résumé, not a person.

I've written about this distinction before—a mirror only gives back what you already have; an adversary gives you what you don't. This time the version is:An index can help you find the path, but it can't walk it for you.

So I made a split:

His infrastructure, I'll take.Markdown, letting the LLM maintain the index, no RAG at small scale, version control. His other two moves, I refuse.First, no letting summaries replace originals—the map is only for navigation, and generation is always fed the whole article. Second, no synthetic data, no fine-tuning—that would average the voice out and also invite fabrication, violating my iron rule of "factual honesty."

Then I did something unsexy first: an inventory

Before changing the architecture, I decided to first figure out what I actually had on hand. Changing architecture on a hunch is the mistake creative people make most easily.

I wrote a small tool to give all my material a checkup. Most of it ran on mechanical statistics; I only asked Claude to write one synthesis, a page for each topic.

The checkup results slapped me in the face on a few counts:

First, I'd always said I had over 400,000 words. Wrong—it's 230,000. Those extra 200,000 were the pre-filter candidate pool, which I'd double-counted. A basic fact about "how much I've said," and I'd nearly doubled it in my memory.

Second, my voice is severely lopsided in its diet. Ninety-seven percent is the "outward, finished, expository" tone—blog posts, company writing, theater pieces, the thesis. The private, the in-the-moment, the emotional come to only about four percent, a little over 5,000 words. In other words, the "me" in this base is a me perpetually talking to the outside world, rarely the me behind closed doors.

Third, there's a fifteen-year gap in the middle. The writing clusters in the early blog years of 2004 to 2010, and the recent stretch around 2026. The middle—2011 to 2025—is almost empty.

These three things, RAG would never have told me. It just quietly pulls out the fragments it can reach when you ask a question; you'd never know how vast the stretch it can't reach actually is.

A map can do something RAG structurally cannot

After switching to a map, I saw something I couldn't see before.

One page of topic organization surfaced a contradiction in my own stance—one I hadn't even been aware of—spanning twenty years:

When I wrote my master's thesis, I believed "truth is a myth"—anti-foundationalism, nothing is simply there to begin with. But in recent years, reflecting, I've written that "real understanding is irreducible"—a nearly opposite stance, close to essentialism.

And the knife I've used to cut both of these—semiotics—hasn't changed in twenty years. Back then I used it to cut "history"; now I use it to cut "Claude."

The same knife, a different thing on the cutting board, opposite conclusions.

This kind of synthetic judgment across twenty years is something fragment retrieval structurally cannot do. It can only return a few chunks containing the keyword at a time; it has no concept of "the whole," so it will never see a contradiction that spans twenty years and hides inside a shift of tone.

A map sees it. Because a map is for seeing the whole picture, not for finding sentences.

Is this progress?—I want to keep an honest ledger

It would be easy to spin these few days of work as "my digital double got more like me."

That's a lie. I want to keep the ledger in two columns.

The column already cashed in is real progress. A binary database I couldn't read became a markdown map I can see in Obsidian, version-control, and self-diagnose. My understanding of myself went from "I've probably got a lot of words" to "230,000 words, 97% outward, a fifteen-year gap." What I should fill in next went from a guess to a data-backed call.

But the column not yet cashed in matters more.

For the person throwing a question at saomin.tw/me,nothing changed at this step. Because the generation path is still the old fragment RAG. Right now the map is only an analysis layer; it hasn't been wired into generation. "Will reading the map and pulling whole articles be more faithful than fragments"—that's still a hypothesis, and I haven't run the A/B test.

And the biggest gap is still there: the me talking to the outside world will keep talking to the outside world. That private, emotional layer of voice hasn't been filled in. The echo is still systematically skewed toward the "proper."

So this round is progress in understanding and progress in infrastructure—not progress in the quality of the echo.

I now have a clearer sense of what I have and what I lack; I also have a base that can be version-controlled and self-diagnosed. But this thing's ability to speak to the outside world didn't improve at this step.

In other words:This makes the promise solid; it doesn't yet deliver on it. It's progress—but the "walked a stretch in the right direction" kind, not the "results got better" kind.

Worth doing. But I won't spin it as "the echo is already more like me."

What I want isn't a system that organizes me

Back to the contradiction at the start—I built the base, then refused to let it speak for me the easy way.

The easy way is to let the LLM chew through me, summarize me into clean entries, and from then on query the entries. Fast and cheap.

I don't want that.

Because a summarized me is a me that's been tidied up, stripped of context, and made queryable on demand. And the value of my twenty years of writing lies precisely not in being "searchable" but in being "deformable"—I wrote about the same thing differently at forty than at twenty, and that difference is me. Tidying it up erases that difference.

So I let the map serve only as navigation. It can tell Claude "go look in article number such-and-such," but it can't speak for those articles. When it's time to speak, always feed the whole article—along with its long-windedness, its pauses, the places where it doesn't finish a thought.

The map is for finding the path.

The walking still has to be done by the original text itself.

Further reading: - How do I address Claude? - Is AI a mirror, or another person? - There's English inside the bones of Chinese