Protecting Collective Intelligence | Stay Human Chapter 14 | Four New Summit Speakers | Artificial Intimacy

Protecting Collective Intelligence

A couple of weeks ago Dave and I were giving a keynote in Venice and someone in the audience asked whether I thought we'd need to protect some jobs—keep them for humans. I said I doubted it. But I have a feeling we're going to need to protect ways of thinking, and in particular the collective ones. How a profession makes sense of things together. How a company does. How a classroom does. And the culture that holds that all together, for people.

Mathematics is the first place you can see it. On October 6, OpenAI put 722 manuscripts on GitHub. About 300 of the results have Lean proofs, which means a computer has checked every step, so they're true. But humans don't yet understand them. The proofs are strange. AI proofs are verbose, unreadable, and verified rather than understood. Specialists in the exact subfield say they can't follow them without AI help. So mathematics now has a pile of correct results that aren't yet knowledge.

Understanding just doesn't appear when there's an answer. It is constructed—between people, past and present. Someone reads a proof and gets stuck, and the stuckness tells them where the gap is between what the proof says and what they already know. They take it to a colleague. The colleague sees it's really a special case of something from 1987, or sees it's wrong. Someone rewrites it so a student can follow it and in the rewriting finds the one idea that was actually new. None of that happens inside a single head. It happens in the back and forth, and it can't be skipped, because the back and forth is where the field decides what it now knows. The result itself is only part of the knowledge.

So why wouldn't we just take the result? It's faster, and it's correct. David Krakauer calls the tools that do the thinking for you competitive cognitive artifacts, because every time you accept the finished answer you give up a little of the capacity to have reached it, and the exchange is almost always worth it in the moment so efficiency wins.

The thing that stops the trade is a group deciding that something about doing the work together is worth more than the speed, and I want to be careful about what that something is, because it’s easy to make it sound like nostalgia. It isn’t the output. The output will increasingly be better from the machine, and I’m not arguing we go back to doing it by hand. We’ll keep thinking with machines. The point is that we keep thinking together while we do it. What the slow version makes is the capacity to judge—the people who were there when a result was placed are the only people who can tell, later, whether it still holds, where it applies, and whether the next thing the machine produces is wrong. Nobody else can.

That’s a function, and it decays when it isn’t used. And it turns out the same process that makes the capacity also makes the relationships, the sense that the knowledge is ours because we built the understanding of it. Those aren’t two things. The people who can judge the work together are the people who made sense of it together, and a group that skips the making will find, when the machine is wrong or silent, that it has nobody who can say so. What we want is to make sure we still have people left who can argue with it.

What is it about making sense together that would make any other profession refuse the trade? I think it's in culture.

Culture is how a group learns. You say what you think is true, someone who knows better pushes back, you adjust, and over a long chain of small, costly corrections a profession comes to know something. The pushback is the point. Without it there's nothing to learn from. A finished proof dropped in from above arrives with nothing to push back on—nobody guessed, so there's nothing to correct—and that's what the mathematicians are reconstructing by hand right now. They're working out what a person would have had to think to get there, so the correction can happen after the fact.

And this is what gives me some hope. Mathematicians are mostly refusing to accept machine output without their collective act of sensemaking. A group of them formed an Association for Human Mathematics, and part of what they resent is being handed the labor of making sense of someone else's output. Imagine if they hadn't. Thanks, we'll take it. Cited, built on, taught, and nobody able to say why any of it is true.

Mathematics is the lucky case because a proof can be checked and the field has a centuries-old culture of arguing. I don't think that's necessarily true more broadly. When a strategy team gets a competitive analysis from a model, it likely reads pretty well and they're under pressure to execute. On top of that, we trust fluent output more than we should—that's automation bias, and it's well documented. And we were never good at telling what we know from what the people around us know; Sloman and Fernbach call that the knowledge illusion. We met with Steve this past week he's worried AI is going to make this much, much worse. AI has now joined the community we lean on, so its knowledge feels like ours the way a colleague’s always did. Put those together and you get people who feel informed and have placed nothing in prior knowledge.

So I think the answer to the question from the audience in Venice is that the thing to protect is our ability to be collectively intelligent—to take what we know and build understanding of it together, through correction. I think people assume this will get easier with better AI but no, it likely gets harder. The output will keep coming faster than any group can absorb it, so the sensemaking takes longer relative to what's arriving, and that gap will get bigger. The mathematicians are at the front of this and they're already behind.

I doubt humans can compete on pure output. AI is delivering staggering amounts of stuff. I doubt we can compete on verification. AI is going to be required for us to monitor and manage AI. What a group of people can still do that ten thousand agents can't, is decide what matters, what's meaningful, and what's trustworthy. Mattering is a judgment about the world a group shares, and a group only comes to share it by arguing its way to that point, collectively.


Stay Human: Authoring Your Mind in the AI Age Chapter 14: Who Can Tell If It's Any Good

AI can produce almost anything, so who can tell if it's any good? I start with unpicking Claude's IRX scores and end somewhere I didn't expect: this is the biggest opening for human-first leadership I've seen in my whole career.

Certification information here.


Artificiality Summit: Four new speakers

We're excited to add four speakers and their salons.

David Weinberger co-wrote The Cluetrain Manifesto and has spent forty years asking how much our categories miss. He also wrote gags for Woody Allen for seven years. His new book, Beautiful Particulars, is out from MIT Press on November 3. It argues that machine learning pays attention to small differences in a way our older frameworks couldn’t. He’s on the Couch with Helen, Friday at 11:15am.

Toshi Anders Hoo is a futurist who spent a decade at the Institute for the Future, where he founded the Emerging Media Lab. He co-produced and directed Ray Kurzweil’s documentary The Singularity Is Near, and now runs the design strategy studio Possibility Spaces. His salon, 9 Flawed (but Useful) Metaphors for AI, asks which of our old mental models for technology still help and which assumptions no longer hold. Friday at 4:30pm.

Matt Abrams runs The Abrams Group, advising and investing in companies. He has two sessions on Friday. At 3pm he’s on the Couch with Dave for A Digital Constitution. The Constitution was written for a physical world, and online much of its work has been taken over by the Terms of Service. At 4:30pm he hosts Why CISOs Aren’t Sleeping, a salon on what happens when AI meets identity and credentials.

Jason Copeland launched iCloud at Apple, co-invented Find My iPhone, and led edge computer vision at AWS. Before that he lived and worked in China and Taiwan. He has two salons: AI in China and the US on Friday at 4:30pm, and The Product Triad Ain’t Dead on Saturday at 4:30pm.

Less than two weeks to go and we are almost sold out.

  • Don't Wait: Price is $1,995 (or $2,995 for two).
  • Don't Worry: Should you need to cancel now, you can transfer your ticket to someone else or you can apply the full value of your ticket to next year's Summit. You can see all the details on the Summit page.
  • Don't Forget: We provide a 30% discount for education and nonprofit customers. Send us an email for more information.

Artificiality Summit 2026

Join us October 22-24, 2026 in Bend, Oregon for 2.5 days with a fantastic group of speakers—academics, authors, designers, investors, photographers, and more.

We'll explore the theme of Unknowing—not ignorance, but a necessary release of inherited assumptions. We don’t yet know what AI will become, and we don’t yet know what we will become in relationship to it.

Unknowing is the space between—the place where neither side is fixed, and something new can emerge.

Register Now

Artificial Intimacy: Who We Become When We Talk to Machines, by Sherry Turkle

Sherry Turkle's Artificial Intimacy is a must-read given her depth of scholarship and experience. What I really liked was her critique of a new kind of behaviorism. She reminds us that there is a profound difference between what a person does and what a person experiences. We can observe the performance of empathy, but that tells us very little about the experience of being cared for, caring for someone else, or being changed by another person. That insight reaches far beyond chatbots. For decades, Silicon Valley has increasingly treated the observable as the meaningful—clicks, engagement, predictions, A/B tests—as though optimizing behavior were equivalent to understanding people. Her decades of listening to what technology feels like from the inside are not anecdotal decoration around the evidence—they are the evidence. It would be an extraordinary historical reversal if AI led us back toward the behaviorism that cognitive psychology worked so hard to leave behind.

But this book did leave me feeling a bit stuck. I admire Turkle enormously. Her account of what happens when machines move into the space of human relationship matters, and she's right to worry about systems that are always available and very good at making us feel understood—without any of the reciprocal difficulty of another person. But I kept waiting for her to tell me what we do with the world we actually have, other than unplug. She knows why people turn to chatbots, and she notes that only about half of Americans have access to a mental health provider. Her answer is more human conversation and fewer machines in intimate life. I get the argument but I don't know that we will, or can, beyond just trying to be "digital nakeds" (my word).

One more thing though: I'm no expert on raising kids today—ours are adults. But I've always been quite skeptical about allowing kids to use AI, and her book pushed me further. I am a never-AI-for-kids person now.


Find & Follow Us

A heads up on our travels: We're planning to be in Miami, Munich, New York, and Providence later this year. Some stops are for keynotes for corporate events...and some stops will be for the Thinking With AI course + certification. If you want to talk about this for your whole team, get in touch!

Follow us on your favorite channels—and please like, share, and repost to help us spread the word.

Great! You’ve successfully signed up.

Welcome back! You've successfully signed in.

You've successfully subscribed to Artificiality Journal by the Artificiality Institute.

Success! Check your email for magic link to sign-in.

Success! Your billing info has been updated.

Your billing was not updated.