Eighteen years ago, when my roommate and I were graduate students and had just moved into an apartment together, we decided to make chicken curry for our first dinner there. We went grocery shopping and bought a whole chicken and a few other things. Only then did we realize we didn’t have a proper knife to cut the chicken with, and the butcher at the store wasn’t available to cut it for us. So we cut up that whole chicken with the small serrated knife we had.
We both remember that evening, and we still bring it up every now and then. But we don’t remember the same evening. I remember an ordinary serrated knife. He remembers something even duller, more like a steak knife. I remember the curry tasting great because we had worked so hard for it. He remembers it tasting great because I cooked it so well.
Neither of us is making it up. That’s just how memory works. Eighteen years later, we both still have the gist: our first dinner, a whole chicken, the wrong knife, a curry that tasted great. What drifted are the details. Over time, our brains hold on to the gist of what happened and let the details fade or change, and every retelling can reshape them a little more.
I think about this a lot. I trained as a cognitive scientist studying memory, so when I reminisce, or talk with family and friends about things we lived through together, or notice what has drifted or gone, I can’t help analyzing it. But I’m also just a person trying to make sense of his own experiences. In a way, life is a collection of episodic memories. Thinking about them like this is a bit like running a meta-analysis on myself.
Lately I’ve been asking the same kinds of questions about machines, which now remember more and more about us.
I’ll be honest: I wanted forgetting to win.
Over the past few months, I tested an idea borrowed from the brain. When an AI system stores memories, should it let the old ones fade to their gist, the way ours seem to? Or should it keep everything exactly as it was, and decide what matters only once a question arrives?
Forgetting lost. In my setup, it lost at every setting I tried, by a wide margin. This essay is about why I expected otherwise, how I tested it, and what losing taught me. It’s also the first study from Aftertrace, a small independent lab I run.
Memory is not a recording
Whenever I explain to friends that memory isn’t a recording, it blows their minds, because it’s their everyday experience. The memories we feel most confident about may not be as accurate as they once were.
The way I think about it, the place and setting of an episode act as a scaffold, and the details hang on that scaffold. Each time we recall the episode, we read the details off that scaffold, and the ones we’re least sure of are the easiest to change. Even a detail someone else remembers differently can attach itself to our original memory.
New memories are fragile at first, and over hours and days, especially during sleep, the brain stabilizes them. Researchers call this consolidation. For a long time, a consolidated memory was assumed to stay more or less fixed. Research on reconsolidation suggested otherwise: under some conditions, recalling a memory can make it changeable again.
The experiment Almut Hupbach, Rebecca Gómez, Oliver Hardt and Lynn Nadel designed to look at this in people is beautifully simple. On day one, you learn a set of everyday objects. On day two, you come back and learn a second set. On day three, you try to recall only the first set. People who got a reminder of the first session before learning the second set started mixing objects from the second set into their memory of the first. The reverse didn’t happen, or at least not nearly as much: when asked to recall the second set, people rarely mixed in objects from the first. This became known as the asymmetric intrusion effect. The reminder seemed to have reopened the old memory and let new things get folded in, though researchers still debate exactly how that happens. A follow-up study found that the key part of the reminder was the place: being back in the same room.
The doctor’s office
That is where my own question came from. If being back in the same place can reopen a memory, what about a place that only feels the same?
Think about walking into a doctor’s office you’ve never been to. It shares no physical space with your PCP’s office, but it has the same kinds of things in it: the waiting chairs, the clipboard, the stack of old magazines, the little window at reception. It feels familiar purely because of what’s in it. My dissertation, working with Lynn Nadel, asked whether that kind of similarity in meaning could do what being in the same place does: reopen an old memory so it can change.
What I found was that it could. In my studies, an old episodic memory could be reactivated, and opened up to change, through the connections in meaning between two episodes.
From brains to machines
A few months ago I found myself back in these old ideas. At the same time, I was building and using AI systems every day, and I kept noticing how differently their memory works. The memory systems behind the AI tools I use mostly keep what they store intact. You can delete things, but nothing fades on its own.
Their memory mostly changes when someone tells it to: delete this, that’s not how it happened, remember this instead. Human memory doesn’t wait to be asked. Nobody tells us to forget something or to update it. It just happens.
A brain works with limited resources, and it can’t keep every detail equally within reach. My intuition, and it is an intuition rather than settled science, is that evolution shaped memory partly for efficiency: let the details of things we don’t use fade, so that what we do need stays within reach.
So I got curious. Could we make AI memory work a little more like ours? What would happen if we did? And might something come out of those experiments that tells us a little about ourselves?
Here’s a picture that helps. Imagine an AI system with an enormous archive in the basement, where storing things is nearly free, but only a small desk to work at when it answers a question. The desk is the model’s context: the text it can actually look at while answering. Its size is measured in tokens, and a token is roughly a word.
There are two broad ways to run that archive:
- Keep everything, choose later. Keep every page in the basement exactly as it was. When a question comes in, find the most relevant pages and bring them, or just the relevant lines from them, to the desk.
- Forget like a brain. As pages age, shrink them: keep the gist and strip the details. Older pages take up less room, so more of them fit on the desk. Keep a faint version of the oldest ones, so that a good reminder might bring them back.
The question wasn’t about saving storage, since the basement is cheap either way. It was about the desk: with a desk of fixed size, does brain-style forgetting answer more questions correctly than keeping everything and choosing later? To be clear, this models one idea about human forgetting, that memories fade to their gist over time. It isn’t the whole story of how we forget.
My bet came from how I picture human memory. Familiar memories are like well-worn paths, and rarely used ones are paths that slowly grow over: hard to find, but not always gone. If old pages shrink, more of them fit on the small desk, so forgetting should help most exactly when the desk is tightest. And the faint versions, brought back by a good reminder, should cover the cases where the details turned out to matter.
I wasn’t alone in that hunch. A growing number of AI memory designs summarize, consolidate or forget what they store, and some are modeled directly on human forgetting curves.
But I should be honest: others have already shown much of this. AI researchers have shown that compressing text works better when the compression can see the question (LongLLMLingua). A benchmark of chat assistants found that replacing full conversations with summaries or extracted facts can hurt their answers, because information gets lost. And a recent paper puts a number on the price of not knowing the question in advance.
I came at it from a completely different angle: cognitive science. I started from how human memory works and how our brains seem to forget, and built the test around that. Every choice in the design was meant to give human-inspired forgetting its best possible chance: settings tuned for every desk size, faint versions that a good reminder could bring back, and questions built to reward exactly that trick. Then I wrote down in advance what would count as winning, and checked whether it earned its place.
How I tested it
I made four main choices.
1. A small world where every answer is known. I built simple artificial worlds: a few hundred people, where they work and live, and who told what to whom. Each world unfolds as 2,000 short scenes, like “Maya, who works at a shipping company, told Arjun about the market.” Some people change jobs and move partway through. Because I built the world, I know the right answer to every question.
2. Five kinds of questions. As a world unfolds, about 6,000 questions come up along the way:
- A simple fact: where does Maya work?
- A two-step fact: first find who Maya talked to at the market, then find where that person works.
- A changed fact: where did Maya work before she moved?
- A fact that was never stated: the right answer is “I don’t know.”
- A cued memory: where did the person who was with Arjun at the market that week move to?
The last kind tests the brain’s trick of bringing a faint memory back with the right reminder, exactly what forgetting was supposed to be good at.
3. A perfect reader. Normally you’d train an AI model to read the desk and answer. But if forgetting lost, I wouldn’t know whether the idea was wrong or the model was just weak. So I used a program that finds the right answer whenever the information is on the desk, and that knows the world’s usual patterns, like the fact that people tend to live in the city where their company is based. That way it can make an educated guess from a gist, which is exactly what forgetting needs. If forgetting has already removed the answer, no reader, however good, can bring it back.
4. The rule, written down first. Before running anything, I wrote down what would count as success and what would mean stop, and saved it with a timestamp, like sealing a prediction in an envelope. Forgetting had to reach high scores on a desk at least about a sixth smaller than keeping everything needed, in at least one kind of world. That’s a low bar. If it couldn’t, stop here, before spending months training models.
I ran it at six desk sizes, from room for a handful of words to a few short paragraphs, and in worlds ranging from very predictable to close to a coin flip. Forgetting’s settings were tuned separately for every desk size, on separate practice worlds, to give it its best chance. The desks were deliberately tiny compared with everything a world held, to make room as scarce as possible, which is exactly where forgetting was supposed to help.
Scoring worked like this: +1 for a right answer, 0 for “I don’t know”, and −1 for a wrong answer, so guessing doesn’t pay. A perfect score is 1.
For the curious. 400 people and 2,000 scenes per world, with about 6,000 questions arising per world; 1,500 of them (300 of each type) were scored in each of 12 evaluation worlds. Desk sizes of 16 to 384 tokens. Three levels of predictability, four evaluation worlds per level. Results are averages over worlds, with bootstrap confidence intervals. The full design is in the repo.
The rule said stop
Forgetting lost at every desk size and in every kind of world.
The best version of keeping everything (more on the versions below) scored 0.99 or better once the desk held at least 32 tokens, about a sentence. Forgetting leveled off at about two-thirds of a perfect score, even with a roomy desk. On the tight desks, where forgetting was supposed to shine, the gap was widest: at 32 tokens, keeping everything scored 0.99 and forgetting scored 0.34.
The rule I’d sealed in the envelope returned its verdict: stop. So I stopped, and never trained a model. I also tried simpler versions of forgetting, like keeping only summaries or throwing out the oldest or least-used pages. None came close to keeping everything either.
Honestly, it was a little disappointing. Nothing earth-shattering. I had a small, intuitive hope that I might see something interesting, and my first instinct was to wonder whether I had missed something.
So I went back over what happened along the way. I ran the experiment three times. The first two runs couldn’t give a clean answer, because my test bench had problems. It had trouble finding two-step facts, the perfect reader mishandled some partial clues, and trimmed pages were being charged for more room than they used. Once I fixed those, the third run met the rule for stopping. The rule itself never changed. One backup rule was added between runs, dated in the log, and never used. And every fix widened the gap between the two approaches, from about 0.15 to about 0.33, rather than narrowing it.
Why it lost
To see why, it helps to look at the questions where forgetting failed. Take the changed facts: where did Maya work before she moved? At a desk of 64 tokens, keeping everything scored a perfect 1.00. Forgetting scored 0.12. When Maya’s old job aged, the forgetting design did exactly what it was built to do: it shrank that memory and stripped out the details. By the time someone asked about her old job, the answer was gone.
Two-step questions failed the same way, 0.07 against 0.99. The system could find who Maya talked to, but the older detail about where that person worked had already been shrunk away.
And the faint versions, brought back by a reminder? Even on the questions designed to help them, they gave forgetting no advantage. Keeping everything could find those memories directly.
Here is the part I find most interesting. Keeping everything wasn’t enough on its own. When the keep-everything approach put whole pages on the desk, it also struggled on small desks: on changed facts it scored only 0.50, though that was still four times better than forgetting’s 0.12. What won was keeping everything and then, once the question was known, putting only the relevant lines from each page on the desk. That took about nine times less room per page.
So there were really three approaches:
- Shrink early: lost.
- Keep whole pages: struggled when the desk was small.
- Keep everything, trim after seeing the question: near perfect.
So compressing memories isn’t the problem by itself. What matters is when you do it. When you can afford to keep the originals, find them again and trim them cheaply, shrinking before you know the question throws away exactly what later questions need. Trimming after you know the question doesn’t.
There’s a big catch. In my test worlds, a simple program picked out the relevant lines, perfectly and at no cost. In a real AI system, a model would have to do that picking, and it isn’t free or perfect. Part of keeping everything’s win came from that free trimming.
What it means, and what it doesn’t
This doesn’t show that forgetting is a bad strategy in general, and it says nothing about whether the brain got it wrong. It shows something narrower: in this setting, with a cheap basement, a small desk and a good way to find things, there was nothing for early forgetting to gain.
The limits matter. These were simple artificial worlds, tested with a perfect reader rather than a trained model, and the trimming was perfect and free. Real systems also forget for reasons this test didn’t capture: privacy, outdated information, and keeping search fast and cheap as memories pile up. And in some places, storing is itself the expensive part: inside the model itself, as it reads, everything it holds onto costs memory and computing power at every step. There, forgetting may well pay off, and this study doesn’t say otherwise.
The brain may be closer to that last case, with no cheap basement behind the desk. Then forgetting early could be a sensible trade. Memory researchers would rightly add that unused memories may mostly become harder to retrieve rather than truly lost, which sounds a lot like keeping a basement. So treat this as a hypothesis, not a finding.
None of this changed what I believe about why we forget. Without a memory that drifts, fades and updates, I think we’d be overloaded, and our ability to adapt would suffer badly. What the result does say is something about the machines: a trick that comes so naturally to us didn’t pay off for one. Some people will say, duh. But it’s one thing to assume it, and another to watch it happen in an experiment you designed hoping for the opposite.
Why publish a no
You define a hypothesis, decide in advance what would count as an answer, and test it. Sometimes the answer isn’t the one you hoped for. It’s still worth knowing, especially when so many people are building AI memory that summarizes, compresses or deletes what it stores.
That’s what Aftertrace is for. It’s a small independent lab with a narrow agenda: memory. If you asked me whether it’s an AI lab or a cognitive science lab, I’d say cognitive science, one that uses AI systems as a sandbox. We’re not chasing AGI. The point is exploration: take open questions about human memory and explore them in machines, whether or not our hypotheses hold up. My gut says that will teach us something about ourselves, and about the AI systems so many of us now talk to every day.
How Aftertrace works follows from that. Write the rule down before the data exists. Publish the no along with the yes. Show every fix to the test bench. Let anyone rerun it. The code, the pre-registration and the results behind this essay are public, and the commit history records the order: the rule first, the results after.
It wasn’t the answer I was hoping for. But it did leave me with a better question: when should a mind decide what’s worth keeping?
My roommate and I still remember that knife differently, and I suspect we always will. I find these questions fascinating, and I’m not done with them.
The study page has the results, the decision record and the code, and the paper is coming soon. To get the next result when it’s ready, subscribe below.