How to use Karpathy's loop method behind LLM Wiki and AutoResearch

Two Andrej Karpathy projects blew up within months of each other, and almost nobody connected them. One builds a personal knowledge base that gets smarter every time you feed it a document. The other runs machine learning experiments on its own, all night, without a human touching the keyboard. On the surface they look like completely different tools solving completely different problems.

They're not. Both run on the exact same underlying method — and once you see it, you'll notice it's the actual answer to "prompting is dead," not some clever marketing line.

What are LLM Wiki and AutoResearch, exactly?

LLM Wiki started as a tweet, followed two days later by a GitHub gist called llm-wiki.md — an idea, not a product or code, but a pattern for building a personal knowledge base. You paste the document into an agent like Claude Code, Codex, or Cursor, and the agent builds out the specifics with you. Karpathy's own framing, quoted widely since, is that "Obsidian is the IDE; the LLM is the programmer; the wiki is the codebase."

You can read the original idea file directly on Karpathy's GitHub gist.

AutoResearch is the other half of the story. Released in March 2026, it's roughly 630 lines across three files, where the agent may edit only train.py, prepare.py scores results and stays off-limits, and program.md sets exploration constraints. The repo went viral almost immediately — released as an open-source repository under MIT license, shipping about 630 lines of code across three core files, going viral within days and sitting near 90,000 GitHub stars.

You can go straight to the source on karpathy/autoresearch on GitHub — the README walks through the exact setup.

Why do these two "different" projects run on the same method?

Strip away the surface-level differences and both projects follow an identical loop: propose a change, check the change against something objective, keep it or throw it away, repeat. That's it. No back-and-forth chat, no you re-reading output and typing a follow-up. You define the goal once and the system iterates toward it on its own.

One outlet described the shift plainly: a prompt is one instruction, after which you decide the next step, while a loop is a goal the model pursues until it arrives — the model plans, acts, checks its own result, then repeats, and you define the objective once while the loop handles iteration.

For AutoResearch, that loop looks almost mechanical. In March 2026, Andrej Karpathy released a GitHub repository called AutoResearch — three files, about 630 lines of code, and the agent's job inside it is narrow on purpose. prepare.py handles constants, data prep, and runtime utilities that the agent may not modify, train.py contains the model, optimizer, and training loop that the agent does modify, and keeping the scope to a single file keeps diffs reviewable.

For LLM Wiki, the loop is less about code and more about knowledge. When you add a new source, the LLM doesn't just index it — it reads it, extracts the key information, integrates it into the existing wiki, updates entity pages, revises topic summaries, and flags contradictions. That's the same propose-check-keep cycle, just applied to markdown pages instead of a training script.

What actually makes the loop work — and why does it need a verifier?

This is the part almost every "just add an agent loop" tutorial skips, and it's the entire reason the method works at all. Without a strict, mechanical check on each attempt, an autonomous loop is worthless. As one breakdown of the pattern put it bluntly: a verifier grades each attempt, and that check can be a passing test, a moving metric, or a build — without a verifier, the agent simply agrees with itself on repeat.

In AutoResearch, the verifier is almost comically simple — a single number. The agent edits the training script, runs a short job, and checks whether the loss went down. No source-checking, no fact-checking, no subjective judgment call. Karpathy built in constraints specifically to keep that check honest and fast: training always runs for a fixed time budget of exactly 5 minutes regardless of platform, which means you can expect roughly 12 experiments an hour and around 100 experiments while you sleep. And there's a hard boundary the agent can't cross: prepare.py contains constants, data prep, and runtime utilities the agent must not modify, while train.py is the single file it's allowed to touch.

That fixed time budget isn't arbitrary — it's what makes results comparable across runs. There are two upsides to this design: it makes experiments directly comparable regardless of what the agent changes (model size, batch size, architecture, etc.), and it means the loop will find the most optimal model for your platform within that time budget.

For LLM Wiki, the "verifier" is softer — structure and consistency rather than a single loss number. That's exactly why upfront schema design matters so much. As one community implementation notes, an LLM wiki is a knowledge system where the LLM maintains structured wiki pages instead of re-searching raw documents on every question, with new sources compiled into durable markdown pages while cross-references update over time. Somebody — you — has to decide what an entry looks like and how entries link before any of that can happen automatically.

How is this different from a normal RAG chatbot?

If you've used ChatGPT file uploads or NotebookLM, you've already used RAG (retrieval-augmented generation), and it has a specific weakness Karpathy's pattern was designed to fix. Most people's experience with LLMs and documents looks like RAG: you upload files, the LLM retrieves relevant chunks at query time and generates an answer — this works, but the LLM is rediscovering knowledge from scratch on every question, and there's no accumulation.

The wiki pattern flips that. Instead of re-deriving relationships between documents every single time you ask a question, the system builds a durable structure once and keeps refining it. A widely-shared explanation of the idea uses a compiler analogy that sticks: you don't execute source code every time you want to run a program — you compile it once into a binary and run that, and Karpathy says to treat knowledge the same way, where your PDFs and notes are the source code and the wiki is the binary.

Can you build this for your own domain without copying the exact repo?

Yes — and this is the part worth actually doing instead of just reading about. You don't need Karpathy's specific implementation to get the benefit; you need the pattern. A minimal setup looks like this:

1. Pick your agent. Any coding-capable LLM agent works. The original gist is designed to be copy-pasted to your own LLM agent (OpenAI Codex, Claude Code, OpenCode/Pi, or similar), which will then build out the specifics in collaboration with you. Claude Code and OpenAI Codex are the two most common starting points right now.

2. Separate raw sources from compiled knowledge. One faithful community rebuild lays out the folder structure clearly: raw/ holds immutable source material while wiki/ holds compiled knowledge pages maintained by the LLM, including an index.md as the global table of contents and a log.md as an append-only operation log. Keep those layers separate — your source files should never get edited by the agent.

3. Write the schema before you feed it a single document. This is the step everyone skips and immediately regrets. Decide what an entry page looks like, what metadata it needs, and how pages should cross-link before you ingest anything. That upfront discipline is what makes every ingestion afterward feel automatic instead of chaotic.

4. For research-style loops, define a single measurable check. If you're adapting the AutoResearch side of the pattern for your own project — code, content, data pipelines, whatever — pick one number or pass/fail condition the agent can check after every attempt. If you can't state your verifier in one sentence, the loop isn't ready yet.

5. Set a hard boundary on what the agent can touch. AutoResearch limits the agent to a single file. Whatever you're building, give your agent the same kind of fence. It keeps runs reviewable and stops small mistakes from compounding into a mess you have to untangle by hand.

If you want a working reference instead of starting from a blank file, lucasastorian/llmwiki is an open-source implementation that connects to Claude via MCP and handles document ingestion, OCR, and nightly synthesis automatically.

Is this actually a new idea, or just AutoML with better PR?

Worth addressing head-on, because it came up almost immediately after AutoResearch went viral. Critics pointed out that automated experiment loops aren't new — some said Karpathy had done little more than rediscover part of a process known as AutoML that researchers at Google, Microsoft, and other labs have used for years, which similarly uses an optimization loop and series of experiments to tune model architecture. The counterpoint, and the actual distinction worth caring about, is what kind of agent is driving the loop: unlike AutoML, this uses an AI agent that can read AI research papers and develop hypotheses for which improvement to make, whereas AutoML systems tend to depend on random variations or evolutionary algorithms to decide what to try.

That distinction is the whole point. The loop mechanics aren't the innovation — verifiers and optimization loops have existed for decades. What's new is putting a reasoning agent inside that loop instead of a random search algorithm, so the "propose" step is informed rather than blind.

What should you build first?

Don't try to replicate both projects at once. Pick whichever pain point actually annoys you. If you're drowning in PDFs, research notes, or scattered docs that never talk to each other, start with a personal wiki — it's the lower-stakes, higher-payoff project for a weekend. If you've got a repeatable technical task with a clear success metric — a script that either passes tests or doesn't, a model that either improves a benchmark or doesn't — build the verifier loop instead.

Either way, the lesson underneath both projects is the same: stop asking the AI a question and reading the answer. Start defining what "correct" looks like, then let the system chase that definition until it gets there. That's the actual shift here — not a new tool, a new relationship between you and the loop.