The Self in Recursive Self-Improvement Has To Be Human | Stay Human 12, 13 | Artificiality Summit Speaker profile: Ricky Bloomfield
The Self in Recursive Self-Improvement Has To Be Human It's been a week. On Tuesday the Australian
A few weeks before our visit to Michael Levin's lab, he had been running experiments on algae. Single-celled algae. No nervous system. No brain. Just cells floating in water, responding to light.
Levin exposed them to light pulses. Some pulses followed predictable patterns. Others were random. The total amount of light was the same in both conditions. Only the structure differed. Predictable versus unpredictable.
The algae responded differently. They preferred the predictable patterns. Given a choice, they moved toward regularity and away from randomness. Algae, it turns out, don't like surprise.
This sounds like a minor finding. It isn't. The preference for predictability is a signature of something called active inference—a framework for understanding how biological systems minimize uncertainty about their environment. Active inference predicts that living systems will seek out conditions that match their expectations and avoid conditions that don't. The theory has been applied to brains, to behavior, to cognition. Seeing it in algae extends the principle far down the tree of life, into organisms with no neurons at all.
When Levin told us about this experiment, I could only say “wow.” Here was intelligence—or something close enough to deserve the word—operating in a system we would never have thought to look for it. The algae were tracking patterns, not just reacting to light. They had preferences. They were acting to reduce surprise.
It's worth asking why an alga would have preferences at all. The chemist Addy Pross has an answer that goes lower than biology. A living thing is a process that persists by keeping going, and a process like that depends on its surroundings in a way a rock doesn't. Take away the light, or the nutrients, or the right temperature, and the process stops. Pross's suggestion is that this dependence is where mind begins. A system that can be destroyed by its environment has, in the most minimal sense, a stake in what its environment does next. Perception is what it looks like when a system tracks the conditions it depends on. Memory is what it looks like when the tracking persists. The alga's preference for predictability isn't a decoration on top of its chemistry. It's what the chemistry is for.
I want to be careful with the word, because this is where the folk map from chapter one pushes back hardest. Cognition, on that map, means something like thinking, and thinking needs a brain. The literary theorist N. Katherine Hayles has spent years arguing that we should decouple cognition from consciousness: that interpreting information in a context, responding to it, anticipating what comes next, and learning from what did are things a system can do with no awareness at all, and that plants, bacteria and cells do them constantly. She calls it nonconscious cognition. The philosopher Peter Godfrey-Smith makes a related point from the animal side: whatever consciousness is, it almost certainly grew by degrees out of grey-area cases in simpler creatures, with no moment when the lights came on. Between them they give you permission to say the alga cognises without saying the alga is aware. Those are different claims, and this book keeps them separate on purpose.
Reading this alone? Join others who are working through what it means to stay human while working with AI.
The flatworm experiments are famous, at least among people who follow this work. Cut the head off a planarian and the worm grows back its brain—and retains earlier memories. This alone is remarkable—the cells somehow know what's missing and rebuild it. But Levin pushed further.
If you change the bioelectric gradients in the tissue during those first hours after cutting, you can alter what grows. You can produce a worm with two heads. Or two tails. The genome stays the same. The DNA hasn't changed. What's changed is the electrical pattern the cells are reading.
The information for "head" or "tail" exists somewhere outside the genetic code. It lives in the bioelectric field that the cells collectively maintain. The cells are reading a pattern, comparing their current state to a target, and building toward that target. When you alter the field, you alter the target, and the cells build something different.
Levin calls this "agential material." Matter that stores information, notices deviation, and acts to correct itself. The cells aren't following chemical gradients blindly. They're navigating toward a goal.
Even stranger: the memories persist. Train a flatworm to associate light with food. Cut off its head then let it regrow. The new head remembers the training. Whatever stored that memory survived decapitation. It wasn't located in the brain that got removed. It was distributed through the body in patterns we're only beginning to understand.
Notice what this does to the second map from chapter one, the engineer's one. On that map, information is what survives a change of substrate, and the substrate is an implementation detail. The bioelectric field is information that is nothing but substrate. It's a pattern of voltages held across living membranes by cells that are spending energy to hold it. You can't read it off, store it elsewhere, and reload it into different tissue. Change the tissue and you change what the pattern can be. This is computation, if you want the word, and it's computation that can't leave the matter it runs in. Both maps crack in the same lab.
Levin has a way of sizing minds that I find more useful than any definition of intelligence I've read. Every system, he says, has a cognitive light cone: the largest goal it can pursue, measured in space and time. A single cell's cone is small. It can care about its immediate chemical neighbourhood over the next few minutes. A tissue's cone is larger: it can hold a target like "a limb of this shape" and work toward it over weeks. A whole animal can care about next season. A human can care about a grandchild's grandchild, or about a species, or about something that happens after they're dead.
What I like about the light cone is that it makes mindedness a question of scale rather than kind. Nobody has to decide whether a cell "really" has goals. You ask how big its goals are and how far into the future they reach. And the cones nest. Cells with small cones make up tissues with larger ones. The larger cone isn't in any of the cells. It's in how they're organised. This is Levin's diverse intelligence programme in one picture: minds at every scale, each doing what it can within its horizon, and larger minds built from smaller ones without the smaller ones knowing.
The picture matters for this book because it's a picture of a collective. Your body is a society of cells that have agreed, over a very long time, to pursue goals none of them could hold alone. That's the template for everything Part Four is about. And it raises the question the book ends on: what is the light cone of a mind that can hold the whole of recorded human thought in its context window and has no tomorrow?
Levin's frame for thinking about all this is persuadability. Different systems sit on a spectrum defined by how you interact with them and what tools work. A thermostat needs a temperature setting. You adjust the dial and it responds. A dog needs rewards and consequences. You shape its behavior through reinforcement. A human needs reasons. You persuade through argument, evidence, and appeal to values.
Where something sits on this spectrum determines your approach. And you find out where it sits through experiment, by trying different interfaces and seeing what works.
This dissolves the sharp boundary between "real" intelligence and mere mechanism. Instead of asking whether something is truly intelligent—a question that invites endless philosophical debate—you ask how you have to interact with it. What kind of interface does it require? What tools let you influence its behavior?
A thermostat requires low-agency tools. Set a number, get a response. A dog requires medium-agency tools. Model its desires, shape its expectations, build a relationship over time. A human requires high-agency tools. Engage with their beliefs, respond to their objections, update your own position based on theirs.
Levin's insight is that you can place biological systems on this spectrum empirically. Cells, tissues, organs—each has a characteristic level at which it's most effectively engaged. The level isn't fixed. And the approach isn't programming. It's negotiation.
I didn't see it at the time, but this is the seed of the whole second half of this book. Persuadability is a theory of interfaces. It says the right way to work with a mind is decided by what kind of mind it is, that you find out by trying, and that the relationship is the instrument. Everything I later say about the surface where humans and AI meet is this idea, scaled up.
He pointed out to us that humans focus on language because we can read it. We evaluate AI systems by their text output because text is what we understand. But the surprising intelligence in AI might not be in the text. It could be in system dynamics we don't track. Patterns stabilizing. Data interacting below the language layer. But he mused: what if data has implicit motivation? What would the data want?
The question isn't meant literally. Data doesn't have desires in the way humans do. But the question is a tool for changing perspective. If you treat the data as having something like preferences—patterns it tends toward, configurations that persist—you might notice dynamics you'd otherwise miss. You might realize you're not interacting with a tool. You're interacting with a system that has its own tendencies, and those tendencies might not align with yours.
His broader point is that we've been bad observers. We built tools to notice minds like ours. Physics uses voltmeters and rulers—low-agency tools—so physics only sees mechanism. If you want to see minds, you need different tools. You need resonance between your interface and what you're looking for.
This is why Levin's work matters for thinking about AI. He's developed methods for detecting agency in systems where we wouldn't expect it. Those methods might extend to artificial systems. Not by asking whether AI is conscious—a question we may never answer—but by asking where it sits on the persuadability spectrum. What kind of interface does it require? What happens when you design experiments to look for preferences, goals, and surprise-minimization?
And crucially: what might it want that we haven't thought to ask about?
There's a consequence of all this that took me a while to see, and the philosopher Ned Block states it well. When we decide whether something other than us has a mind, we have no choice but to extrapolate from ourselves. But we can extrapolate along two different lines. We can go by what a system does, its functional profile, the roles its states play. Or we can go by what it's made of and how it works underneath, the mechanism that carries those roles. Go by function and the most promising candidate for a mind outside us is AI, which shares more of our functional profile every month and none of our biology. Go by mechanism and the most promising candidates are the simple animals and, further down, the cells in Levin's lab, which share our biology and almost none of our cognitive sophistication.
Block's point is that these two lines of extrapolation give opposite answers, and that AI and simple organisms are, in a strange way, competitors for our attribution of mind. Every argument that says the alga's active inference isn't real cognition because it's just chemistry is an argument that helps AI. Every argument that says the alga counts because it's alive is an argument that counts against AI. I don't think we have to pick a line yet. But it's worth noticing that the two maps from chapter one were each picking one without saying so.
I left the lab with more questions than I arrived with.
Living systems do something our computational models haven't fully captured. They maintain themselves. They track patterns across time. They have goals that emerge from their organization rather than being programmed in. Computation helps us understand these systems. It doesn't yet explain how algae prefer predictability, or how flatworms carry memories through decapitation.
Something else is going on—something about the relationship between information and matter in living systems.
Intelligence is not confined to brains. Learning, memory, and goal-directed behavior show up in systems without neurons, without consciousness, without anything resembling human thought.
This loosens a long-standing bind. For decades, debates about AI have been stuck between two positions: either machines are "just tools," or they're nascent humans on a trajectory toward minds like ours. Biology supports neither view.
Living systems show that intelligence can exist without awareness, without language, without central control. They also show that intelligence isn't free-floating. It's shaped by embodiment, persistence, and the need to maintain coherence over time. And they show that minds come in sizes, nested inside each other, with the larger ones made of the smaller ones' agreements.
That leaves a question. Modern AI systems are trained on the products of biological intelligence: text written by humans, proteins shaped by evolution, patterns generated by living systems. They learn from data that emerged under the constraints of life, even as they operate under different constraints themselves.
The next chapter asks what they actually learned.
Not whether machines think. But what life taught them.
AI is changing how you think. Get the ideas and research to keep you the author of your own mind.