Building a brain around a stateless model.

In April 2025 we started building unify-agent, an agent meant to work for one person over a long period. We treated the language model inside it as a stateless unit of computation. Text goes in, text comes out, and nothing is kept between calls. Everything else a working agent needs, from what it remembers to how it keeps talking while it works, we built around the model, a bit like assembling a brain from parts that sit outside the network.

We never thought that structure was the long-term answer. It was the most we could get out of the models of 2025 without training them, and we expected agents to end up trained end to end, with their memory and their actions in the weights. Models improved faster than we planned for. Much of what our harness did now happens inside the model, behind the API. Building more structure around it stopped paying off, and we moved on to the weights.

Why we built a harness at all

Training a frontier model takes a lab's budget, and in 2025 the best models were closed anyway. What you could control from outside a lab was what goes into each call. So we split the agent into parts with one job each, most of them model calls, and used the context window as the boundary between them. Our first technical spec has a part for each job, from the one that picks up the phone to the one that clicks around a browser, mostly passing text to each other through queues.

Before unify, Dan worked on a memory module that trains end to end with the network around it (ESM, ICLR 2021), and we always expected agents to end up there too. The harness was a way to get the most out of the models we had, in case they took a while to get capable. If they got capable quickly, it would have been a stopgap anyway. Dan put it like this on X in May 2026: "Eventually it will all be end-to-end trained, but hijacking the context window as a boundary between nodes is the best short term path for value."

What we built, in the order we built it

Each part was built around the models of its day, and links to the commit where it first went into unify-agent. We have since tested every one of them twice. First we ran the code as it stood a few weeks or months after each idea first went in, with the models that code used. Then we rebuilt each idea from scratch, small and without the old bugs, and tested it on today's models against the simplest strong alternatives we could think of. The rebuilds and their results are in our demos repo. Where a bug decided a result, the entry says so.

Talking and acting in separate loops

In April 2025, when OpenAI had just released o3 and o4-mini, a call to a capable model still took several seconds. Because a person keeps talking while an agent works, we ran the conversation in one loop and the work in another. From May each piece of work came back to the conversation as a handle it could steer while the work ran, and the work could ask the person a question through it.

On the code of June 2025, with o4-mini in every loop, the agent answered "how's it going?" in about 7 seconds, against 48 for a single loop that couldn't be interrupted. A single loop that could be interrupted answered as fast while it kept working in only 4 of 9 runs, and otherwise stopped the work to answer or answered once it had finished. When the work needed a fact from the person, though, the single loop that couldn't be interrupted got the task right more often, in 15 of 18 runs against 11. The two loops also never told the person how the work turned out, in any of the 8 runs that finished, because a bug wrote each result above everything said since, where the conversation, answering the latest message, passed over it. That changed later in June, in a version we didn't test. We haven't analysed the rebuild's runs yet.

Storing skills as functions

We kept what the agent learned about a task as Python functions in a library, calling one another, rather than as notes. A repeated task then just runs its function, and a repair to one shared step reaches every task that uses it.

Posts about a topic: go to linkedin.com, click the search bar, type the topic and press enter, then click the Posts filter.

People with a name: go to linkedin.com, click the search bar, type the name and press enter, then click the People filter.

Messaging someone: go to linkedin.com, click the search bar, type their name and press enter, click the People filter, then click Message on the first result, type the message and press enter.

def search_linkedin(query: str):
    """Search LinkedIn for `query`, starting from the homepage."""
    go_to_linkedin_com()
    click_the_search_bar()
    type_the_text(query)
    press_enter()

def send_message(text: str):
    """Message the first person in the results with `text`."""
    click_message()
    type_the_text(text)
    press_enter()

def find_posts_about(topic: str):
    """Show the LinkedIn posts about `topic`."""
    search_linkedin(topic)
    click_the_filter("Posts")

def find_people_named(name: str):
    """Show the LinkedIn members called `name`."""
    search_linkedin(name)
    click_the_filter("People")

def message_person(name: str, text: str):
    """Send `text` to the LinkedIn member called `name`."""
    find_people_named(name)
    send_message(text)

On the code of May 2025 with GPT-4.1, across 16 related tasks on a simulated professional network whose search changed halfway through, the library finished every task with under a third of the tokens the notes needed. On today's models the rebuild was never more accurate than the simplest alternatives we tried, keeping no memory at all among them, and never cheaper than retrieving past solutions. The stronger models solved every task without it, and with it they cost about 60% more than with no memory.

Requests that carry their conversation

A request that one model writes for another often leaves out what the conversation already settled. Ours were written by o4-mini at the time, and in our tests it left that out of more than a third of them. So the loop passed its conversation down with every call, read-only, and from June hid it from the model so the model never wrote it out.

On the code of June 2025, where the conversation settled what the request left out, a lookup run by o4-mini got 95% right with the conversation, against 75% with the request alone. Where the conversation settled nothing, it halved the lookup's accuracy, and when o4-mini wrote its own requests the gain was within noise. On today's models, with requests that left the detail out, the gain fell from 12 points with Claude Haiku 5.5 doing the lookup to none with Claude Opus 5.5, which simply asks. When the model wrote the request itself, even Haiku named the detail 96% of the time, and passing the conversation tied every alternative at a higher cost.

Memory consolidated in the background

In June 2025 a long context was expensive to fill on every call, and an agent that remembers has to keep deciding what's worth keeping. We summarised its history in the background, away from the conversation, and in July in windows such as the past day and the past week, each built from the one below.

That version had two bugs, never fixed before we took the summaries out in October 2025, that kept most windows from reading the history they summarised, and its numbers say little about the idea. On today's cheaper models the rebuild beat every fixed-size memory we tried on questions about a stretch of time, and matched keeping the whole record in a million-token context for a twentieth of the cost or less. Summarising the raw records at question time did better still, though at 17 times the cost on Claude Haiku 5.5 and about a hundred times on GPT-6 Luna. Across every kind of question, simply extracting facts did as well or better, since a summary drops old quotes and dates.

Functions and guidance, linked both ways

Some of what an agent learns is a rule rather than a procedure. We kept the procedures as functions and the rules as written guidance in a second library, each rule linked to the functions it governs and kept in one place.

On the code of October 2025 with GPT-5, changing a linked rule took one edit, against five for copies pasted into skill folders and about two for a rule kept as a skill of its own. Tasks didn't follow the changed rules any more often, though. The code's own way of updating a rule also failed every time, and the test had to call around it. On today's models linking helped GPT-6 Luna follow rules but left Claude Haiku 5.5 slightly worse than simpler arrangements. From GPT-6.1 Sol up it made almost no difference. A linked rule still took one edit where pasted copies took about three, though a single file of rules needed only one too.

Demonstration without a mode

Teaching the agent by showing it was an ordinary request that carried frames of a shared screen, each with what was said over it, passed by reference so they didn't fill every call.

Two bugs decided our first version's result. In half the runs the frames never reached the model, and when they did, the answers about them came back without the frame they belonged to and were applied in whatever order they arrived. On today's models the rebuild, at 256 to 500 frames, cost 2 to 16% of pasting every frame into the context, and its accuracy held steady on the cheaper models as sessions grew, where pasting fell. It never cleared the bar we set before the runs, though. At 64 and 128 frames it did worse than simpler options, and Claude Opus 5.5 got every request right either way.

A fast brain and a slow brain

A spoken reply has to start within about a second, and in late 2025 a capable model with every tool took several seconds just to decide what to do. So a fast voice model held the conversation and waited for the slow one instead of guessing, while a slow model with the whole context passed it data, never dialogue.

On our February 2026 code, with GPT-5 mini as the fast model and Claude Opus 4.6 as the slow one, the slow model took about ten seconds to decide on a lookup and eighteen to answer it. Two rules for the fast model took invented facts from 56 of 80 replies to lookups down to none, and its replies began in 0.7 seconds. The slow model also kept writing whole replies to small talk, so the agent answered twice.

Execution as a coordinate

By January 2026 our agent acted by writing code, with Claude Opus 4.5 writing it, and that code had to run somewhere. We gave it one tool whose arguments choose, on every call, what state survives and which session the code joins.

On that code it reloaded a slow export a third as often as a fresh sandbox did, but one shared interpreter did just as well and faster. The rebuild, on today's models, found that one interpreter shared across several clients let one client's data leak into another's answers, which choosing per call avoided. One interpreter per client, set up by the harness, avoided both problems and matched or beat it on every model for less. When Claude Sonnet 5.5 named its own sessions, two clients sometimes picked the same one-letter name, and one ran on the other's data.

The slow brain speaks

Over the summer we inverted the split. The slow model wrote every substantive line, spoken word for word, and the fast model only held the floor while it thought.

In scripted calls no turn of small talk got two replies and every answer was heard, at the price of a first word at 1.8 seconds instead of 0.7. On today's models neither split beats the simplest strong alternative. A single fast model with reasoning on and all the records in its prompt answered as accurately as either design, finished its answers sooner and cost less, though it said nothing for its first few seconds. Without reasoning, the same model stated a wrong or invented fact in nearly half its calls.

Stronger models needed less of it

Almost every part above made up for something the models of 2025 couldn't do well. They couldn't think and talk at the same time, so we split the agent into a fast brain and a slow one. Context was expensive to hold, and we summarised history and passed screens by reference to save it. Since then models have got far better at reasoning before they answer and at using a long context, and each of those took over work our harness used to do.

Without the fast brain, a caller asking our agent for an invoice amount in February 2026 would have heard nothing for about ten seconds and waited eighteen for the answer, which on a phone sounds like the line has dropped. Today's models can begin a reply within one to four seconds, though a single model with tools still tends to say nothing while a lookup runs. A canned "one moment" brings its first word to about a second, and with Claude Sonnet 5.5 the answer comes in about five.

The rebuilds tell the same story across the parts we rebuilt. On today's strongest models most of the gains in accuracy are gone. What's left is mostly on the cheaper models, or in cost, such as a memory that answers for a twentieth of the price. That is roughly what we expected. The harness was standing in for things the model couldn't do yet, and as the model learned to do them the harness had less to do.

The next step is the weights

Everything our harness learned, it kept as text that the model reads again on every call. That carries an agent a long way, but the model itself never gets better at the work. We think the next frontier is an agent whose experience goes into its weights and makes it better at its job, the way a person gets better at theirs. The harness is not enough makes that argument, and Continual-ARC is how we measure whether an agent learns from its work at all. What we haven't worked out yet is how to keep training a model on its own work without it forgetting what it already knew.