The intimacy economy is here, and Meta's Muse shows why agent design matters

The trade, at scale

Muse, Meta's new personal agent, is the top app in the App Store. Millions of people have handed it the keys to their messages, finances, and health records. The app has an average rating of 4.9 stars, so people clearly like it—even as, according to WIRED, Muse updates its records on every person in their lives every hour.

This doesn't surprise us. In October 2024, at the Artificiality Summit in Bend, Oregon, Helen and I argued that we were transitioning from the attention economy to the intimacy economy. We said that people would trade an intimate understanding of themselves for AI that makes their lives easier. We see that trade with Muse—people are handing over intimate details to a company whose record with user data includes Cambridge Analytica and a $5 billion penalty from the US Federal Trade Commission. Thirteen days before Muse launched, Meta settled with US states for up to $17.1 billion over harms to young users.

I'm not writing this to say that using a personal agent is a bad idea. Sharing intimate information makes a lot of sense because that context is what can make a personal AI useful. But the question I want to address is what we have been asking for years: if we are going to give AI agents intimate access to our lives, what designs should we demand?

What we said, and when

We have been making this argument for three years. In October 2023, I wrote that the intimacy economy would follow the information and attention economies. The internet tapped our need to learn and know. The social web tapped our need for attention. Generative AI would tap our need for intimacy. But I warned:

But we should learn from what’s come before. Commercially motivated, unconstrained, and poorly designed solutions for information scarcity eroded the veracity of information. A similar pattern happened with attention. 

Even then, the question was design and business model. I asked whether intimate AI would live "as Siri is on an iPhone, embedded software," or whether we would pay an "intimacy tax" through ads.

In May 2024, after GPT-4o and Google I/O, I wrote that the intimacy economy had arrived. In June, we argued that trust would be its foundation. In July, we introduced the intimacy surface. That October, at our 2024 Summit in Bend, Helen and I brought it together in our keynote:

For the past 20 years, we’ve lived in the attention economy. We’ve paid attention to get content from our digital services, and in return, we’ve offered up our own content to be paid attention to. But we believe this is shifting. We are moving from an attention economy to an intimacy economy—one in which we will trade an intimate understanding of ourselves for AI services that promise to make our lives easier.

Why will we make this trade? Because an AI system’s effectiveness depends on knowing us intimately. You may have already felt this. When you chat with a conversational AI, you find yourself sharing your needs, your wants, even your dreams and fears. This intimate understanding allows AI to grasp the context of who you are, and from that context, infer what you might want. In some cases, it might even act on your behalf without you explicitly telling it to.

This kind of intimacy holds tremendous promise. A system that understands us deeply could serve us in ways that feel intuitive, anticipating our needs before we even voice them. 

But with great promise comes great peril. We’ve witnessed how the attention economy drilled down into our focus, fracking it into tiny, tradable pieces. What happens if this same extractive logic is applied to our intimacy? Will our most personal, private selves be mined and commodified just as our attention was? Will data mining evolve into life mining?

Our hope is that humans will have the ability to opt into this intimacy through what we call the intimacy surface—a dynamic, multidimensional space where humans and AI meet. This surface adapts to the level of trust we have, expanding or contracting based on how much of ourselves we’re willing to reveal. The more we trust, the more AI can understand and serve us. But the trade-off is clear: the more context we provide, the more risk we take as well—making trust in the companies that hold our intimacy the foundational metric of the Intimacy Economy.

A year later, Meta announced it would personalize content and ads using people's conversations with its AI, and news reports said there would be no opt-out. I called it intimacy extraction. Muse is the next step. It reaches past our chats into our accounts, our messages, our health, and the people in our lives.

Why we make the trade

Meta launched Muse on Sept. 8. Nearly 140,000 people have rated it since, and Apple's automated summary of the reviews describes a fast, capable assistant that people use to automate tasks and keep on top of reminders and email.

That usefulness is purchased with context. Just as we have paid for digital technology with our attention, we may now pay for AI’s usefulness by giving it a much broader context about what matters to us. We wrote in 2024 that the more you tell an AI system about the context of you, the better it can interpret what you want. If your agent is going to help you with an auto repair, it needs to know what kind of car you have, where you take it to get it repaired, what the issue is, and whether your car is still under warranty. Without that information, it can't be much help.

How much information people are handing to Meta isn't a mystery if they look. In the app description, the "data linked to you" includes health & fitness, purchases, financial info, location, contact info, contacts, user content, search history, browsing history, identifiers, usage data, sensitive data, and diagnostics. Millions of people downloaded it anyway. 

So this is the trade. An agent that knows nothing about you can do a lot less for you. The real question is what happens to intimate data once you hand it over: where it lives, who can see it, what it is used for, and whose intimacy it is.

Whose intimacy is it?

Since launch, researchers have found ways to get Muse to reveal its own instructions, and WIRED's Lily Hay Newman and Matt Burgess reported what was inside. One instruction sets Muse to work every hour on a running file for each of the people around you, from family and friends to coworkers and people you only follow online.

Muse's relationship pages are the clearest picture yet of the intimacy economy in practice. Each page builds a profile: a person's home and job, the dates they care about, and how close the two of you are. Another section nudges you to keep the relationship going, with suggestions for when to reach out and how to support them. This means that Muse isn't just tracking your information and preferences—it's tracking those of everyone you know as well. With Meta's glasses, you can at least hope to spot the person wearing them. You can't see anyone else's Muse. And Muse might be surveilling you for anyone you know—all without your knowledge or consent. 

WIRED reports that each user's data sits in its own virtual machine, walled off from other agents. Meta says users can erase what Muse remembers and review a log of everything it has done. That may all be true. But the leaked files show how much Muse writes down on its own.

Peter James, the developer who exported them, found a nightly "dream" job that reviews recent conversations and writes notes about the user's preferences and style. And users must opt out if they don't want their Muse interactions used to train Meta's models.

I don't know what Meta will do with a map of the relationships of millions of people. Meta says Muse data isn't shared with its ad systems. But given its history, I'm not sure I believe it. In October 2025, Meta changed its policy to use AI chats for ad targeting. It's not hard to imagine the company doing the same with Muse. 

Two architectures for the same trade

The useful comparison is not Muse versus no Muse. It is Muse versus a different design for the same trade. Apple's new Siri, which shipped with iOS 27 in September, also reads your messages, email, and photos to act on your behalf. 

Both Muse and Siri have rough edges, and their features will change over time. Architecture lasts longer, and it's what this essay is about.

Where the two designs part ways is what happens to your intimacy after you hand it over. Muse lives in Meta's cloud. Each user gets a dedicated virtual machine that holds the agent, its memory, its relationship pages, and the logins for every service you connect. Everything Muse learns about you is stored on Meta's servers. Siri keeps what it knows about you on your phone. When a request needs more computing power than the phone has, it goes to Private Cloud Compute, which uses your data to answer that one request and then discards it. Your data passes through Apple's servers. It doesn't live there.

The other difference is who can see inside. Meta says each Muse machine is sealed off from other users' agents. That protects you from other users, not from Meta. Meta's own announcement makes the distinction: it promises a future version, encrypted with a key only you hold, so that Meta itself can't get in. That version is due later in 2026, with no firm date.

Apple built Private Cloud Compute to that standard from the start. It's designed so that no one at Apple can see your data, attackers can't single out one user, and outside researchers can check the software themselves. With Muse, we know how the system works only because a developer asked it to archive its own filesystem and send it to his Google Drive. Meta says those files were meant to be accessible, in the interest of transparency. But a system you can poke at is not the same as a system built to be checked.

What the companies do with your data differs too. Muse trains on your interactions unless you opt out. Siri uses your data to answer a request, not to train models or build profiles. And Meta makes its money from advertising, while Apple makes its money from hardware and services.

The difference matters more than any feature. One design borrows your intimacy to help you, then gives it back. The other keeps it.

What we should demand

Design is a choice. In 2024, we proposed the intimacy surface: a space where humans and AI meet that expands or contracts with our trust. Here is what I think we should choose, and demand, from any agent we let into our lives.

1. Borrow, don't keep. An agent should pull in context to complete a task and then let it go. Standing profiles, held by the company, should be the exception we opt into, not the default.
2. Designed, not promised. Privacy that depends on a policy can change with the next policy. Privacy built into architecture, and open to independent inspection, is much harder to undo.
3. Other people's intimacy is not ours to trade. The people in our lives did not sign up. Anything an agent knows about them should stay with us, be visible to us, and never feed a company's social graph.
4. Forget means forget. If I tell an agent to forget something, it should be gone, not hidden.
5. Opt in, not opt out. The intimacy surface should grow as trust is earned. Training on my life should never be the default I have to find and switch off.
6. Follow the money. Ask how the company makes its money. If it profits from knowing more about you, assume it will try to know more about you.

This is what we mean by designing for human capacities, not from them. An agent built for us uses our intimacy to help us. An agent built from us uses it to build something for itself.

The choice in front of us

Muse's success settles one question. People want agents, and they will give agents their intimacy to get them.

The question that remains is the one we closed our keynote with: trust in the companies that hold our intimacy is the foundational metric of the intimacy economy. Trust is earned through design, not declared through policy. And it is tested most when the people whose intimacy is at stake are not the ones making the choice.

Life mining is not inevitable. We get to choose which agents we let in, and we get to say what we expect of them. The agents we choose now will know us better than any product we have ever used. They should help us remain the authors of our own lives, and leave the people we love out of someone else's ledger.