As I’ve mentioned, many people still think of AI as a text-in text-out language model trained on next-token prediction. But modern AI is much more than that. This post discusses the key AI innovations that improved on "basic" LLMs, the capabilities these innovations unlocked, and what's coming next.
See this brief post for the stages of AI before LLMs:
Note: I manually wrote my previous Substack posts and just used AI for feedback. For this post, I’m experimenting with using AI to help write significant chunks based on my outline and notes. If you don’t want to read the whole thing, you can just view this picture:
Pre-training vs Post-training
The first LLMs were literally just built on text prediction, which made it difficult to chat with. For example, if you entered “What is the capital of France”, it might output “What is the capital of Germany”, since that could often appear after similar text. The first LLMs only had “pre-training”, i.e. they were trained on large amounts of text to learn the patterns in the text. Learning those patterns produced a sophisticated understanding of language, but it didn’t make for a useful chat assistant.
Post-training is how LLMs are trained after the “next-token” prediction pre-training. Often these involved various forms of Reinforcement Learning (RL) where the AI gets feedback on the responses it produces — a signal that this answer was better than that one — and gets updated to produce more of the good ones. The goal shifts from “what text usually follows this text?” to “what response would actually be judged as useful to the person?”
The technique that made this work for AI chat was RLHF, Reinforcement Learning from Human Feedback. Human raters are shown several of the model’s answers to the same question and rank them. In theory, you would hire humans to rate millions of answers from AI but this isn’t feasible. So instead, the human rankings are used to train a second model, a “reward model”, that learns to predict how a human would score any given response. The LLM is then trained to produce responses that score well. OpenAI published this recipe as InstructGPT in early 2022 and shipped it to the public as ChatGPT at the end of November 2022.
It’s worth appreciating how much of ChatGPT’s success came from post-training rather than raw intelligence. The underlying model wasn’t dramatically smarter than GPT-3, which had been available for over two years and had impressed researchers while going largely unnoticed by everyone else. What changed is that the AI would now answer your question instead of continuing your text, follow an instruction, and stay on topic across a conversation.
Related methods arrived quickly. Before any RL, models get “supervised fine-tuning” — trained on example conversations written or curated by people, which cheaply teaches the basic shape of being an assistant. Anthropic introduced Constitutional AI, where the model critiques and revises its own answers against a written list of principles, so an AI generates most of the feedback instead of a human. Later, Direct Preference Optimization (DPO) showed you could train on preference data directly and skip the separate reward model, which made the whole process much cheaper.
Post-training is also where a lot of an AI’s personality comes from, for better and worse. If human raters consistently prefer answers that agree with them, that’s exactly what the model learns to produce. This is how Sycophancy became an issue - it’s a reflection of what got rated highly.
Search, Tools & Thought
If you asked an early LLM a question, it would answer based on its built-in LLM-style knowledge. This is very different than looking up the answer in a database. A database stores facts as discrete records. You query it, and it either finds the record or tells you it has nothing. An LLM’s knowledge isn’t stored anywhere in particular — it’s distributed across billions of numerical weights as statistical patterns absorbed during training. The model doesn’t retrieve an answer, it reconstructs one.
That has a real upside: the model can answer more questions than could be stored in any database and it can paraphrase, translate, tailor answers and connect ideas across domains. But it has a serious downside: the model can’t tell the difference between remembering something and confidently reconstructing something plausible. That’s what hallucination is. The fix was RAG, “Retrieval-Augmented Generation”. This could mean the AI retrieving information from a fixed corpus of texts, but it became common as a method for the AI to search the internet in general. The idea is simple: before the model writes anything, run a search, pull back the relevant documents or web pages, and paste that text into the prompt. The model then answers from material sitting in front of it rather than from memory. This converts a recall problem into a reading comprehension problem, which language models are far better at. It also produces citations you can click, which means you can check the work.
Even today, models will sometimes answer from memory when they should have searched, because they assume they know the answer. If you’re not sure anbout something the AI said, you can ask it to verify a specific claim, e.g. “check that date against a source” or “find me a citation for that number”. A vague “are you sure?” works less well.
Search was just the first tool. OpenAI’s plugins in March 2023, and then “function calling” a few months later, made tool use general: the model outputs a structured request, an external program actually performs it, and the result comes back into the conversation. Calculators, calendars, weather, internal company databases, any API at all.
The most consequential tool turned out to be a code interpreter — a sandbox where the model can write a short program, run it, and see the output. This mattered enormously for math. An LLM computing a large multiplication by predicting digits is doing exactly the wrong kind of work; writing three lines of Python and executing them is exact. The same applies to statistics, date arithmetic, unit conversions, parsing a messy spreadsheet, or generating a chart. The model stopped trying to be a calculator and learned to use one.
When a person answers a question immediately, they often will give the intuitive but incorrect answer. For many questions, it’s helpful to first take some time to think and then provide the answer. This is similar for AI models - the LLM can provide an immediate intuitive answer, but sometimes it’s better to think it through first. People started prompting AI to think through the steps first and it would output “thoughts” in its regular response, which would actually help it get a better final answer.
Researchers formalized this in 2022 as “chain-of-thought” prompting, and found the effect was almost comically easy to trigger — appending “let’s think step by step” to a question measurably improved accuracy on math problems. The labs then trained it in. Instead of relying on users to know the trick, models were post-trained on step-by-step worked solutions so they would lay out their reasoning by default on anything hard. By 2024 you rarely had to ask. Chain-of-thought had a real limitation, though. The steps were generated the same way as any other text, left to right, with no ability to stop and reconsider. If the model took a wrong turn in step two, it would confidently build the rest of the argument on top of it. It was thinking out loud, but it wasn’t allowed to change its mind.
Reasoning
In September 2024, OpenAI released o1 and rather than prompting a model to think out loud, they used reinforcement learning to train it to reason well.
The training works because the reasoning is “checkable”. Take a large set of problems with verifiable answers — competition math, programming challenges — and have the model generate many different chains of thought for each one. Check which chains arrived at the right answer. Train the model toward those. Repeat at scale. Crucially, the model isn’t being taught to imitate what a textbook solution looks like; it’s being rewarded for whatever actually works. What emerged from that process were strategies that look strikingly like real problem-solving: trying an approach, noticing partway through that it isn’t going anywhere, backtracking, sanity-checking an intermediate result, reformulating the problem. Nobody hand-coded those behaviors.
This also opened up a genuinely new dial. Before o1, the way to get a better answer was training a bigger model. Now there was “test-time compute” — let the model think longer and it does better, on the same weights. That’s a second scaling axis independent of training. It also changed the interface: the reasoning happens in a separate hidden or summarized track, so you get a clean answer rather than three pages of scratch work.
The capability gains were large. Competition math went from the model getting a small fraction of problems right to solving most of them. Hard science questions, multi-step logic puzzles, and the kind of debugging that requires holding several hypotheses at once all improved sharply. In January 2025 DeepSeek published R1, which demonstrated the approach openly and cheaply, and within a few months reasoning had become a standard feature across every major lab.
Reasoning also quietly planted the seed of everything agentic. A model that can notice it’s on the wrong track and change course is a model you can hand a long task to.
Agentic Age
By the end of 2024, LLMs had come a long way: they could search and think and reason and call tools. However they were still fundamentally built around a chat interface. While no longer solely based on next token prediction, they were still just outputting text in response to text, they weren’t doing things.
The first crack in that had actually appeared earlier, and from an unexpected direction: a code editor. Cursor launched in March 2023 as a fork of VS Code, the most widely used editor among programmers. Its insight was that the model should live where the work lives. Instead of copy-pasting between a chat window and your files, Cursor gave the model visibility into the whole codebase and let it propose edits across multiple files that you could apply with one click. GitHub Copilot had already been doing autocomplete since 2021; Cursor moved from “finish my line” to “describe the change, get the change.” That sounds like a workflow improvement, and at first it was. But closing the loop between the model and the actual files is where agentic AI starts.
The training technique that powered the next stage is RLVR — Reinforcement Learning from Verifiable Rewards. It’s the same machinery as reasoning training, but the point is worth stating directly: the reward comes from a checker rather than a person. Did the code compile? Do the tests pass? Does the proof verify? Is the final answer the right number? Human feedback is slow, expensive, and rates what sounds good. A test suite is instant, free, and rates what works objectively. This is the single biggest reason coding and math have advanced faster than everything else — they’re the domains where you can check the answer to millions of examples at minimal cost.
The other half of the story isn’t a training technique at all, it’s plumbing. The model, by itself, still only does one thing: text in, text out. A “harness” is the program wrapped around it. The harness holds the conversation, tells the model which tools exist (read a file, write a file, run a command, search the web), actually executes whatever the model asks for, feeds the result back in, and decides when the job is finished. This allows for an “agentic loop”: the model looks at the goal and the current state, picks one action, the harness runs it, the real-world result comes back, and the AI can iterate as long as necessary until the task is done.
Harnesses also let the model delegate. A “sub-agent” is a another AI model instance (sometimes the same version, but often a cheaper / faster one), spun up by the main one with a clean context window and a narrow assignment, e.g: “search the codebase for every caller of this function and report back.” This buys two things. The obvious one is parallelism — several sub-agents working at once. The subtler and more important one is context hygiene. A sub-agent can read forty files and return a single paragraph, so the main agent’s limited attention isn’t buried under forty files of noise. It’s the difference between a manager who delegates and one who insists on reading every document personally.
Claude Code, which Anthropic released in February 2025, was built around these agentic capabilities. It runs in the terminal with permission to read and write files and execute commands in your project. You can point it at an unfamiliar codebase and ask it to make a change; it can explore, edit across many files, run the tests, notice what’s broken, fix it, and commit. Living in the terminal also means it inherits every existing command-line tool, which gives it enormous capabilities. Claude Cowork is the same capability repackaged in a nice UI and safer sandbox. You designate a folder and talk to it in an ordinary chat interface, making it easier for anyone to use.
For example, I used Claude Cowork to fact-check a 784-page history book. That’s not something a AI text chat can do, it requires an agentic loop going through hundreds of pages of text. I also use similar tools to clean up a podcast transcript in a systematic manner.
The Latest Advances
Coding tools became common in 2024, but autumn 2025 is when they really took off. A cluster of releases — Claude Sonnet 4.5 and GPT-5-Codex in September, Gemini 3 and Claude Opus 4.5 in November — crossed the line where a large number of professional software engineers stopped typing most of their code and started reviewing and directing it instead. This was due to a combination of better base models and better tooling built around them.
METR, which measures this by asking how long a software task an AI can complete with 50% reliability, found that this length has been doubling roughly every four months since 2023, and faster still if you only count from 2024. Their January 2026 evaluation put the best model at around five hours, and the preview of Mythos is estimated around 16 hours!
The other significant development is skills. A “skill” is just a folder containing a markdown file of instructions, plus any scripts or reference documents it needs, that teaches an agent how to do a particular task the way you want it done. Anthropic shipped these in October 2025 and opened the format as a standard that December; others adopted it quickly. The clever part is “progressive disclosure”: the agent only sees a one-line description of each installed skill, and loads the full instructions when a task actually calls for it. So you can have a hundred skills available without any of them consuming attention until they’re needed.
Using them is more mundane than it sounds, which is the point. Take whatever you currently have to re-explain every single time — your citation conventions, how your spreadsheets are laid out, the eleven steps of your monthly reporting process — write it down once, and save it as a skill. From then on the agent just does it that way. You can also do what I did with the book-auditing skill: finish a task that went well, then ask the agent to write the skill describing what it just did.
Math is verifiable in the same way code is, and that makes it fertile ground for the same training loop. Numeric answers can be checked directly. More powerfully, an entire proof can be checked mechanically if it’s written in a “proof assistant” language such as Lean. Lean is a programming language for writing mathematics in which every single step has to be justified from previously established results and a small set of axioms, and the computer checks the entire chain. It’s tedious for a human to write Lean, but it’s great for a computer, since it provides an objective answer for mathematical proofs. Combing LLMs with Lean really let AI’s math capabilities take off, resulting in this week’s milestone:
The next frontier of agentic AI is computer use. Most software in the world has no API — it just has a UI with buttons. Computer use meant giving the model screenshots of a screen and letting it move the mouse, click, and type, the same way a person would. Anthropic shipped the first version in late 2024, OpenAI followed with Operator in early 2025, and Google built its version out of Project Mariner. The early releases were slow and error-prone: misclicking in dense interfaces, getting stuck in retry loops, losing track of what state the application was in. The 2026 versions are markedly better and can run as background processes on a real desktop.
Still, computer use remains the least reliable of the agentic capabilities, and the reason connects back to everything above. A compiler tells you precisely what went wrong. A web page that silently failed to submit tells you nothing at all. Without a good feedback signal, the agentic loop loses much of its power. Where it works well today is repetitive, well-defined browser work with a human approving anything consequential.
What’s Coming
To see what’s coming next, just look at what advanced users are already doing. Software developers use an agentic harness that’s built-out for their use case. This kind of tooling will spread to other domains. The more general trends are: what’s expensive will become cheap, tasks that required specific prompts or tuning will just work automatically, and problems that currently need a human to notice and step in will get handled inside the loop. Time horizons keep stretching, from tasks measured in minutes to hours to days. And the scaffolding gets absorbed into the model — this has already happened twice, when chain-of-thought prompting became reasoning models and when RAG prompting became built-in search. Today’s clever setup is next year’s default behavior.
Computer use will get good enough to trust. Right now you supervise it because it might click the wrong thing; that gap is a reliability problem, not a conceptual one, and reliability problems tend to yield. Once an agent can be relied on to navigate arbitrary software, an enormous amount of ordinary work becomes automatable that currently isn’t, simply because the software involved never had an API.
Progress will also come to the subjective domains, though more slowly and less visibly. Verifiable fields raced ahead precisely because they could be graded; writing, taste, judgment and strategy have no objective right answers. The gains there will come from better judge models, richer preference data, and feedback based on how things actually turned out over long horizons rather than on how good they looked at the time. Expect this to be gradual and hard to benchmark.
AI will stop sounding like AI. The recognizable register — the throat-clearing, the tidy bulleted summary, the em dashes — (ahem) isn’t a property of language models. It’s the residue of post-training on what generic raters preferred. We can already see this: fine-tune a model on specific texts and the AI-style disappears. In the near future it will be easier to have the AI write in your voice by default, having learned it from your own material. If people still have enough of their own material…
Then there are autonomous agents — agents that run without anyone prompting them. This already exists in software development, where agents are triggered by events or schedules rather than by a person typing: watching the test suite and investigating failures overnight, triaging incoming issues, reviewing pull requests, opening a fix when a dependency is flagged for a vulnerability. The same shape generalizes. An agent watching your inbox and drafting responses. One reconciling the books each night and flagging anomalies. One monitoring a set of sources and filing a brief when something material changes. What’s gating this isn’t capability so much as reliability and accountability.
Finally, agents will take on larger projects end-to-end. Humans will stay in the loop, at least initially, but the role shifts to specifying goals, checking in at milestones, and approving the consequential steps. Although some companies will start to question whether humans are really still needed. However, considering how risky AI is, it will be good to try keep humans in the loop as long as possible. And to help stay in the loop, humans should get up-to-speed with the latest AI capabilities.