What Conversation Changes

Consider what a model has to work with the first time we encounter it.

We arrive with something we need help with. Maybe a document we’re stuck on, a decision we’re wrestling with, a prototype we’re trying to get off the ground, or a subject we’ve been trying to better understand. We may have been thinking about it for five minutes or five years.

The model meets us in the middle of all that.

What we put in the prompt is rarely the whole situation. Sometimes we leave things out because they seem obvious. Sometimes we don’t know which details matter. Sometimes figuring out what we’re actually asking is part of the help we need.

There’s something familiar about this predicament.

Think about a librarian. They know the bookshelves inside out but almost nothing about the person asking, “What should I read next?”

Their knowledge gives them somewhere to start. They might ask what we’ve enjoyed recently, suggest a few things, and see what catches our interest. Either way, they have to start helping before they know very much about us.

We don’t expect them to know us yet. We hope they can help anyway.

Much of what we ask of a model begins there, too. It brings general knowledge and capabilities; we bring a particular situation and whatever we manage to communicate about it. Often, that’s enough to get somewhere useful. An explanation clicks. A suggestion gives us something to try. A first draft helps us figure out what we wanted to say.

And then we keep talking.

We ask a follow-up question. We explain why a suggestion won’t work. We point out something the response missed, or discover that we left out an important detail.

By this point, our next request isn’t another first encounter. “Could you try again?” depends on what the model just tried. “That’s closer” depends on where we started.

We’re already relying on something we didn’t have when we arrived. How much can we eventually come to rely on it?

In conversations between people, that reliance can become so familiar that we barely notice it.

Think about a friend you’ve known for years referring to a mutual acquaintance:

“He’s doing that thing again.”

The sentence leaves almost everything for you to supply. Who is “he”? What is “that thing”? Why does it matter that he’s doing it?

And yet, under the right circumstances, the sentence might be perfectly precise.

Its precision comes partly from what your friend doesn’t say.

Your friend expects you to remember the people, events, disagreements, and jokes that give the remark its meaning. Your shared history has shaped what your friend expects you to understand.

Continually reintroducing that history wouldn’t merely be inefficient. It would be socially strange.

Imagine your friend explaining who they mean and retelling the incident every time they bring it up. The repetition could begin to feel awkward and, frankly, kind of insulting—as though your friend no longer trusts your memory or attention.

We spot the same mismatch in clumsy exposition in TV and movies:

“As you know, Sarah, ever since our parents died in that boating accident ten years ago…”

Of course Sarah knows.

She was there.

What makes the dialogue uncanny isn’t that the information is irrelevant. Quite the opposite: it may be essential to understanding the scene. What’s uncanny is that two people who supposedly share a history speak as though they don’t.

The dialogue demotes their history from something that changed them into something they need to recite.

That gives us a useful distinction.

History-as-input.

And history-as-that-which-changed-the-interpreter.

The fabric of history

If we think about history primarily as input, we might ask:

What information have we gathered over the course of this interaction? What remains relevant? What should we retrieve?

But the second framing raises a different kind of question:

What does each participant now believe about the other because of that history?

When your friend says “He’s doing that thing again,” the omission does more than save words.

Your friend is making a claim about you.

They’re betting that you know who “he” is, remember “that thing,” and understand why its recurrence matters. Maybe even that you know how your friend feels about it.

Your friend’s omissions reveal what they expect you to know, remember, and understand.

That’s theory of mind at work: your friend is predicting what you’ll understand without another explanation.

And the prediction can be wrong.

Maybe you’ve forgotten the incident. Maybe the two of you remember it differently. Maybe your friend thinks you reached an understanding about what happened and you walked away with an entirely different interpretation.

Shared history doesn’t guarantee shared understanding.

“I thought you knew.”

“I thought you meant…”

“I didn’t realize you still felt that way.”

These aren’t necessarily failures of memory. Both people may remember what happened. The mismatch is in what that history came to mean.

So what accumulates between people isn’t merely a larger collection of mutually accessible facts. Their models of one another change too.

What your friend believes you know changes. What your friend expects you to remember changes. What your friend thinks you’ll find funny, irritating, obvious, surprising, or hurtful changes.

What your friend thinks you think about them changes.

And those expectations shape how your friend interprets whatever you say next.

A model of another person isn’t just what we could recite about them. It’s what makes one interpretation of their next remark seem more plausible than another.

Making meaning

When we send a message to a model, the system can include earlier messages alongside it. The model can then draw on more than our latest words as it produces its next response.

Within a transformer, a mechanism called attention lets the model draw connections across that material. Something we said near the beginning of the exchange can influence the reply it generates now, even when many messages sit between them—as long as those earlier words remain in context.

That gives the past a way to remain useful. The model can revisit something we said at the outset almost as an object in the present.

But imagine a long exchange during which you’ve corrected a misunderstanding. The model quotes your correction back to you—and then makes the same mistake again.

The system kept your words. But the model didn’t use them to avoid the mistake.

When we expect a conversational partner to remember something, we’re often asking for both.

Suppose you ask a model to recommend “somewhere quiet.”

It could make a note:

[prefers quiet places]

Now it can consult that preference the next time you ask where to go, instead of retrieving the original conversation.

That’s useful. But it may have just made the input smaller.

Over the conversations that follow, you repeatedly reject places the model describes as quiet because they feel “dead.” Meanwhile, you respond positively to bustling restaurants with intimate acoustics, neighborhood bars where nobody bothers you, and parks full of people where it’s still possible to disappear into a book.

It could keep updating the record:

[prefers quiet places]

[doesn’t like places that feel dead]

[likes bustling restaurants with intimate acoustics]

[likes neighborhood bars where nobody bothers them]

[likes parks full of people where they can disappear into a book]

Now it has a richer set of impressions about you.

Later, you ask:

“Somewhere quiet again.”

What would you expect the model to think “quiet” actually means to you?

Maybe, for you, quiet doesn’t mean an absence of activity.

Maybe it means an absence of attentional demand.

Your responses have given the model a reason to take that interpretation more seriously. You haven’t merely supplied more preferences. You’ve given it feedback about how well its interpretation fits.

If that feedback changes which meanings the model considers plausible, it has shifted the probability distribution it brings to “quiet.”

That’s a belief update.

The model needn’t replace one definition of “quiet” with another. Your responses give it reasons to connect the bar, restaurant, and park through something besides just their noise levels: ways to keep to yourself. And it can give that connection more weight without forgetting why low noise might sometimes matter.

A different starting point—not the last word.

Learning what you usually mean shouldn’t make it harder for you to mean something else tomorrow.

Perhaps next time you really do need somewhere with almost no sound. Can the model use what it’s learned without treating that learning as a permanent definition of you?

The goal isn’t for identical words to produce different answers merely because time has passed. It’s for what the model learns from you to change where its next interpretation begins.

Modeling minds

You’re learning from the model’s responses, too. You form expectations about what it’ll understand, what it’ll miss, and how much you’ll need to explain.

For example, if it keeps misunderstanding you, you might begin overexplaining. Your messages get longer. You spell out things that you would otherwise leave implicit. You try to anticipate errors before they happen again.

The model could mistake your workaround for a preference.

It might conclude that you like providing elaborate instructions, when you’re actually trying to compensate for its failure to learn from simpler ones.

The model may think it’s learning how you prefer to communicate.

It may actually be learning what dealing with it has required of you.

Your model of the model changes your behavior. That behavior then becomes evidence for its model of you.

What do you now expect it to get wrong?

What does it think your extra explanations say about you?

You can both be mistaken. What matters isn’t perfect convergence. It’s whether the exchange gives either of you a way to check—and revise—those expectations.

You might tell the model:

“I’m only spelling this out because you keep missing it.”

That contribution gives the model a reason to reconsider the preference it inferred. Next time, you can leave out the extra instructions and see whether it still misses the point..

That checking and adjustment is calibration.

Your correction isn’t noise. It may be some of the interaction’s most valuable information.

Evidence of change

Ask for somewhere quiet again without restating the earlier feedback.

Does the recommendation reflect the reasons you gave for liking or rejecting its suggestions? Does the model ask a more useful question? Or does it return to the same assumption until you explain yourself again?

The model might recite what it learned about “quiet.” The next request tests whether it puts that learning to use.

And what do you no longer have to explain just to keep it from making the same mistake?

We can trace these moments: what did the model appear to assume, what did you say or do that challenged it, and how did it respond to your next request?

We can examine the expectations on both sides. What does the model seem to expect you to know? What do you expect it to remember? Where have those expectations become more accurate, and where have they drifted apart?

We can look for things the model keeps explaining after you’ve made clear that you understand. And we can look for the opposite failure: it skips an explanation because it assumes you understand something you don’t.

What can each participant safely leave unsaid? What do we only think they can?

We can map not only what information accumulated, but how the plausible interpretations available to each participant changed along the way.

Remembering someone is not the same thing as learning how to interpret them.

And a longer history is not necessarily a deeper relationship.

We’re asking about more than how much of our history a model can carry forward.

We’re also asking whether, after enough interaction, the same words can begin to mean something different because they came from you.

And whether your next correction can change not only what the model knows about you, but how it goes about knowing you at all.

Postscript

There’s something slightly suspicious about explaining all of this in an essay.

You may no longer be reading from quite the same interpretive state with which you read the first sentence.

“History” probably brings something different to mind now. So might “memory” and “feedback.”

And I can get away with saying “quiet” without explaining the whole example again.

Earlier, I needed paragraphs to explain some of these distinctions. Now I can refer to them in a word or two—and leave other things unsaid.

You were there.

That doesn’t mean we understood it in the same way.

Maybe somewhere along the way I got something wrong.

Maybe I introduced a distinction that doesn’t survive scrutiny. Maybe an example that seemed convincing to me led you somewhere entirely different. Maybe I’ve called something learning without showing how it would change the next exchange.

I’ve tried to leave enough of the route visible that you can find those turns—not just encounter the conclusions I arrived at.

You might see a way forward that I haven’t.

And if you do, I can’t merely append your correction to the end of this essay and call it feedback.

It should change what I write next.