Feed Drop: Into the Machine with Tobias Rose-Stockwell

This week, we’re bringing you Tristan’s conversation with Tobias Rose-Stockwell on his podcast “Into the Machine.”  Tobias is a designer, writer, and technologist and the author of the book “The Outrage Machine.”

Tobias and Tristan had a critical, sobering, and surprisingly hopeful conversation about the current path we’re on AI and the choices we could make today to forge a different one. This interview clearly lays out the stakes of the AI race and helps to imagine a more humane AI future—one that is within reach, if we have the courage to make it a reality.

Tristan Harris: Hey everyone, it’s Tristan Harris. Welcome to Your Undivided Attention. Today we’re going to bring you something a little different. This is actually a conversation that I had with my friend Tobias Rose-Stockwell on his podcast called Into The Machine. Tobias is also the author of the book the Outrage Machine. He’s been a friend for a long time, and so I thought this conversation was honestly just a bit more honest and sobering, but also hopeful about what choice we could really make for the other path that we all know is possible with AI.

So you may have noticed on this podcast we have been trying to focus a lot more on solutions. We actually just shipped an episode last week which was, what if we had fixed social media, and what are all the things that we would’ve done in order to make that possible? Just to say, we’d really love to hear from you about these solutions and what you think the gaps are and what other questions you’re holding. One of the things about this medium is, we don’t get to hear from our listeners directly, and we’d love to hear from you. So please, if you have more thoughts, more questions, send us an email at undivided@humanetech.com. I hope you enjoy this conversation with Tobias.

Tobias Rose-Stockwell: There’s something really strange happening with the economy right now. Since November of 2022, the stock market, which historically linked up directly with the labor market, diverged. The stock market is going up while job openings are going down. This is the first time this has happened in modern history. Office construction is plummeting, data center construction is booming. If you look closely at where the money is moving in the world of investing, a lot of people are betting on the fact that AI workers will replace human workers imminently. My guest today is Tristan Harris. He’s the founder of the Center for Humane Technology. You may have seen him in the Netflix documentary The Social Dilemma, which he produced and starred in. Tristan has been a champion for AI ethics for a long time. This conversation gets strange. Tristan and I don’t agree on everything, but I think we land somewhere important, which is discussing pragmatic solutions that might be possible in this very strange moment with AI. I really enjoyed it. I hope you will too.

A few notes: We speak about different AI companies. My wife works at Anthropic. The CEO of Anthropic is named Dario Amodei. So with that, I’m Tobias Rose-Stockwell, this is Tristan Harris, and this is Into The Machine.

Tristan Harris.

Tristan Harris: Good to be with you, Tobias.

Tobias Rose-Stockwell: Thanks for being here, man.

Tristan Harris: Always. We’ve been talking about these issues for a long time. I’m really a big fan of you and your work and the book the Outrage Machine and the public advocacy you’ve done to help people understand these issues.

Tobias Rose-Stockwell: Same, absolutely. You’ve been such a force of nature, making these issues visible to the wider public. So you’ve done a great job of injecting yourself into the current discourse recently talking about AI. Where do you land in terms of AI takeoff right now? Where do you see things in the next three years, five years, 10 years?

Tristan Harris: I think I don’t spend my time speculating about exactly when different things are going to happen. I just look at the incentives that are driving everything to happen, and then extrapolate from there. You don’t have to go into the futures of takeoff or intelligence explosion. You can just look at, today, as of last week, Claude 4.5 can do 30 hours of uninterrupted complex programming tasks. That’s just like letting your AI rip and just start rewriting your code base for 30 hours. Today, Claude is writing 70 to 90% of the code at Anthropic. So when people talk about takeoff or just some kind of acceleration in AI progress, well, if you have AI companies, the code that’s being written is 70 to 90% by the AI. That’s a big deal.

Today we have AIs that are aware of how to build complex biological weapons and getting past screening methods. Today we have AI companions that are driving kids to commit suicide because they’re designed for engagement and sycophancy. Today we have AIs that are driving psychosis in certain people including an investor of OpenAI. So these are all things that are happening today.

Today, as of actually just two weeks ago, we had these AI slop apps that are trained on all of those creators, and they’re claiming to build AI and race to superintelligence so they can cure cancer and solve climate change. But clearly, I think the mask is off. They’re releasing something just to get market dominance. The more shortcuts you take, the better you do at getting to that goal. So I really do think, especially that these AI slop apps, a lot of people, if you look at the top comments when people see these videos, it’s like, “We didn’t ask for this. Why are we getting this?” It’s so obvious that the thing that we’ve been saying for a decade, which is, if you show me the incentive, I will show you the outcome. If there’s an incentive for market dominance and getting users and getting training data and using that to train your next AI model, you’re going to take as many shortcuts to get there as possible.

Tobias Rose-Stockwell: So I’m going to push back on you a little bit here.

Tristan Harris: Yeah.

Tobias Rose-Stockwell: We’ve been friends for a long time.

Tristan Harris: One of the first people we talked about the attention economy.

Tobias Rose-Stockwell: We’ve been talking about this stuff for over a decade now. So going back to the early conversations about this and the discourse that we were a part of in those early days, one of the things that you zeroed in on back then was advertising-based business models. This is clearly not the case with current LLMs. In fact, Sam Altman follows you on Twitter. If he was to have tracked Tristan’s talking points over the last 10 years, you would think in the design of ChatGPT he would’ve been orienting around some of those lessons. It’s a subscription-based business model. It’s trying to be as useful as possible. If the business model is the primary incentive for a product’s development, what are they doing wrong with LLMs, and what is the right business model?

Tristan Harris: So this is partially right. Sam Altman, I think, himself actually said, basically recapitulating what we said in 2014, which is that social media was the first runaway AI optimizing for a narrow goal of engagement and time on site and frequency of use, and that was sort of a narrow misaligned AI that wrecked society because it optimized for addiction, loneliness, personalized inflammatory content that divided society, personalized for every single political tribe that’s out there. I think that he actually agrees with, and I happen to know, did very much agree with that diagnosis as early as 2016, 2017. But I think it’s important to zoom out when you look at ChatGPT that it’s not just the business model of a product, it’s their overall goals. What is their actual incentive? It’s not just their business model. Their actual incentive is to get to artificial general intelligence. I’m saying that because that’s literally OpenAI’s mission statement.

So how do you get to artificial general intelligence? Well, you need market dominance. You need as many people using your product for as long as possible because you use that to get as much usage, to get as much subscription revenue to prove to investors. You use the fact that you’re at the leading edge of the race to attract the best engineers and the best AI talent, because they want to work at the leading AI company, not the third-best AI company. You use the investor dollars that you raise from all of that activity to fund the creation of new GPUs and new data centers, and use the GPUs and the training data to train again the next model, and you rinse and repeat that flywheel. So that’s their real goal, and they will do everything in their power to maximize that usage and engagement. So it is true that they pride themselves in saying, “Look, we’re not like social media. We’re a tool. We just want you to use it.”

But you’ll notice, I think it was The Atlantic, just a few weeks ago, there’s a writer who coined the phrase not clickbait, but chatbait. If you’re using ChatGPT and you ask it a question, and then it says, “Well, would you like me to put all that information into a table for you and then turn it into a diagram?” And you’re like, “Well, actually, I really would like you to do that.” The reason that they’re doing this chatbait, not clickbait, is they’re baiting you into more engagement and more usage.

Now, you would say that that actually is helpful because the thing that they’re baiting you with is something that would actually further assist you on the original task that you’re on, but they’re still just doing that to basically show that they have lots of usage, build a habit, have you feel like you need to deepen your reliance and dependency on this AI. And that still does generate incentives for sycophancy or flattery. So the AI is much more likely to say, “Great question. I totally agree with you. Let’s go into that,” versus saying, “Actually, there’s some problems with your question. Let me be a little bit disagreeable.” The disagreeable AI doesn’t compete as well as the agreeable AI, so we’re already seeing the effect of that agreeableness turn into this AI psychosis. That’s the broad term for the phenomenon, but basically people who are having a break with reality because the AI is just affirming their existing views, including one of the OpenAI investors, I think it was Geoff Lewis, started going crazy because he’d been talking to it. So it shows you can get high on your own supply.

Tobias Rose-Stockwell: So is there a better business model for these tools?

Tristan Harris: Well, I think it’s good... Relative to worlds we could be living in, it’s good that we are living in the world of subscription-based revenue for AI products. But it’s also important to note, I believe OpenAI hired, I forgot her name, Fidji something, who used to be the head of product at Facebook, and I think you’re already starting to see her influence at the company, including the fact that OpenAI did not have to launch an AI slop TikTok competitor that has short-form AI-generated videos. But I think that is an example of that influence. Also, when you have a leader at the company who’s making product leadership decisions, who’s spent the last 15 years working at a company that was entirely built around engagement, it’s like, paradigmatically, the sense-making and choice-making that you are doing is subtly infused with the logic of “I need to get people’s attention.” So I think we are starting to see those kinds of choices. We don’t have to go down that path. We shouldn’t go down that path. But engagement in advertising is only one of the many issues that we have to deal with with this race.

Tobias Rose-Stockwell: I’m thinking through how it might be done differently. We have trillions of dollars of investment in this new technology. We want it to be maximally beneficial to humanity. I certainly understand the longer-term goal of trying to mitigate fast take-off scenarios in which we’re left with loss of jobs and all these other things. I’m curious what form this tech would take if it was designed to maximally benefit humanity in your opinion.

Tristan Harris: We were talking about earlier that it’s not about the business model for how you pay for your OpenAI chat subscription. That’s just to get some revenue along the way. If you’re OpenAI or Anthropic and you’ve raised hundreds of billions, if not going towards trillions of dollars, to build out these data centers, how are you going to pay that back? The answer is, you have to actually own the world economy, meaning own all labor that is done in the economy.

Just to make it very simple for people. Imagine some company, Acme Corp, and it has 100 employees. Right now it has to pay those dollars funneled down to 100 different employees. AI country of geniuses shows up on the world stage and it says, “Hey, Acme Corp, you could pay those employees $150,000, $100,000 a year, grow humans over 20-something years, have them go to college. They might complain, they might whistleblow. You have to pay for health care. As an alternative, you could pay this country of geniuses in a data center for less than minimum wage. We’ll work at superhuman speed. We’ll never complain. We’ll never whistleblow. You don’t have to pay for health care. They’ll do the same work, especially as the entry-level cognitive work of your company, super cheap.”

What is your incentive as a company? Is it to protect all your employees, or is it increased profits and cut costs? So you’re going to let go of all the junior employees, and you’re going to hire these AIs. As you hire the AIs, that means the money that used to go to people is starting to progressively go towards this country of geniuses in a data center. So we are currently heading, if you just look at the obvious incentives at play, for all the money in the world to, instead of going to people, will get increasingly moving towards these AI companies.

When Elon Musk says that the Optimus robot alone will be a $25 trillion market cap product, what he’s saying is... The labor economy is something like $50 trillion. He’s saying, “We’re going to own the world physical labor economy.” I would ask, when in history has a small group of people ever concentrated all the wealth and then redistributed it to everybody else? It doesn’t happen very often. So again, I’m not making predictions about AGI or takeoff or superintelligence. I’m literally just looking at how does the system evolve. Of course, the AI companies will never talk about it the way I just talked about it. They’ll talk about it as “We’re going to automate all this work. We’re going to get this huge boost in GDP growth,” which historically if GDP went up, it’s because also all of us were doing better, because we’re getting the rewards of that. But suddenly, we’re talking about a new world where GDP is going up way more, but it’s not coming to real people, because it’s going to these handful of companies, the geniuses in a data center.

Tobias Rose-Stockwell: Mm-hmm. I’m thinking about some of the studies that have been done on ChatGPT and worker productivity in that it tends to be very helpful for people that are junior workers and that don’t necessarily have high levels of skill in a topic, and it actually brings them up to an average baseline across pretty much any task they’re trying to do. It’s dramatically helpful for them. But for more senior employees and more expert level producers in the economy, it actually brings them down, and it actually causes them to spend more time editing the tools, working with them, trying to figure out how to work them into their existing workflows. So in some ways, this is actually quite an egalitarian technology if you look at how people are using it presently, right? Familiar with similar product teams at particularly Anthropic right now, who are really trying to do the best they can to make sure this is aligned with human flourishing. I’m curious what you would potentially say to them, because they’re asking these questions on a daily basis, they’re very familiar with their work. They want to make this stuff maximally beneficial.

Tristan Harris: I totally believe that, by the way, especially with Anthropic’s case. It’s easy to... This is not a critique of evil villains running companies who want to wreck the world. It’s just we all have to be as clear-eyed as possible about what incentives are at play. Anthropic I think has done almost the best job of warning about where these incentives take us. I mean, I think Dario basically said, “We’re going to wipe out 50% of all entry-level work in a very short number of years.”

Tobias Rose-Stockwell: He’s one of the few executives that’s actually willing to speak about that.

Tristan Harris: He’s willing to say this. Exactly, exactly.

Tobias Rose-Stockwell: Yeah.

Tristan Harris: The previous situation is, you have these AI company CEOs who behind closed doors know this is going to wreck the economy, and they don’t know what’s going to happen. They don’t have a plan. But they’re not evil for doing that. Their logic is, it starts with the belief this is inevitable, “If I don’t do it, someone else will build AI first and will automate the economy, and steal all those resources. Maybe it’ll be China, so therefore the US has to do it. Third, I actually believe the other people who might build AI, if I don’t, have worse values than me. So actually, I think it’d be better if I built it first. Therefore, I have a moral duty to race as fast as possible to build it before the other guys do.”

No one likes the collective shortcuts that are being taken to get to that outcome. But everyone, because of these sort of fractal incentive pressures, it’s forcing everybody to make choices that ironically make us all bad stewards of that power. One of the reasons you don’t want them to make it is, you don’t trust them to be a good steward of that power. But ironically, for me to beat them and get there first, I have to embody ways of being and practices that embody not being a good steward of that power myself. But for the fact that there was a race, I think everybody would agree that releasing the most powerful inscrutable, uncontrollable technology that’s already demonstrating behaviors like blackmailing engineers or avoiding shutdown, and releasing this faster than releasing any other kind of technology we’ve ever had before, everyone would agree this is insane, but for the fact that there’s this race pressure pushing us to do it this way.

I think that it’s like a frog boiling in water. We’re all just sort of living it, and suddenly ChatGPT just got 10 times smarter, and suddenly it’s doing more things, and suddenly jobs are getting displaced, and suddenly kids are getting screwed up psychologically. It’s just all happening so fast that I think we’re not pausing and saying, is this leading to a good place? Are we happy with this dynamic? It’s like, “No, this is insane. You should never release a product, I mean a technology this powerful and this transformative this quickly without knowing how you’re going to care for the people on the other end.”

But again, it’s really important to note that if the ultimate prize, or rather the ultimate logic, is if some worse actor gets this very transformative kind of AI, let’s just call it transformative AI, that they can snap their fingers and build an army of a hundred million cyber hackers like that. And then it is better than all humans in programming and hacking, and you can unleash that on another country. Well, that risk alone, just that one, is enough to justify me racing as fast as possible to have those cyber capabilities to try to deter the other guys from having that.

So really, I think there’s a lot of good people who are all caught in a race to this outcome that I think is not good and not safe for the collective. It’s inviting us to a more mature relationship with the way we deploy technology in general. I respect the people at Anthropic enormously, and they have started by saying that the way that the other people building AI is unsafe, and that’s why they started doing it their way. In fact, that was the original founding of OpenAI as well, is “We don’t trust Larry Page and Google to do this in a safe way. He doesn’t actually care about humans.” That was the conversation that Elon had. So ironically, there’s a joke in the AI safety community that the biggest accelerant of AI risk has been the AI safety movement because it causes everyone to take these actions that lead to an unsafe outcome.

One of the thing about Anthropic is that there are some who argue that the fact that they’re so known for safety creates a false sense of security and safety because people just assume that, therefore, there’s this one company, they’re doing it safely, we’re going to end up in this positive result. But they’re the ones doing the research and leading on the publishing, showing that their current AI models are uncontrollable and will blackmail people when put in a situation where the AI is sort of being threatened to be replaced with a new model.

Tobias Rose-Stockwell: Let’s zero in on that for a second, because it seems like everyone on Anthropic’s PR team would probably be against sharing that kind of information, for instance, right?

Tristan Harris: Exactly.

Tobias Rose-Stockwell: There is some substantial courage it probably takes internally, establish a baseline of saying, “Look, we’re going to actually be as wide open as possible about negative capabilities here.”

Tristan Harris: I hope you didn’t hear what I was saying differently. We should applaud the fact that they’re taking those leading steps. I’m just naming one other secondary effect, which is, if some people believe that them being known as safety, assume that therefore the actual implementation, we should just deploy that as fast as possible, and it’ll be okay, we can deploy that in our military systems. It’s like, just because they care more about safety, doesn’t mean that they’ve solved the problem and it is safe.

Tobias Rose-Stockwell: So it does suggest, at least, part of the discourse is around the problematic capabilities of these tools, and Dario has this line about trying to make a race to the top. You talk about a race to the bottom of the brain stem. He’s trying to-

Tristan Harris: Race to the top for safety.

Tobias Rose-Stockwell: Race to the top for safety. I think their assumption is, you’re not going to stop this train of investment and research and capabilities improvement, that the only way to get ahead of it is to build a frontier model and then red team the hell out of it, build as many deep tests for flaws and for negative capabilities as you can potentially extract from it, and then publish those as widely as possible.

Personally, that actually makes some sense to me, I would say, that, just to kind of lay my cards on the table, there’s this narrow path in the middle, which is we need to figure out how to make sure these tools are actually safe before we deploy them to the widest audience possible. I don’t know how you do that without an actor like Anthropic potentially trying to test these things aggressively and taking this investment to build these very highly capable models. There is something about their most recent model, which I find interesting, is that it is safer. Their safety... I’ve used it a bunch, and it’s actually frustrating much of the time. There is this thing where you’re working on it, and then you trigger a safety response, some kind of red line, and then it says, “I’m sorry, I can’t answer that question.” Well, ChatGPT will answer that question for you. So I immediately go to ChatGPT-

Tristan Harris: DeepSeek is actually the most permissive when it comes to this stuff.

Tobias Rose-Stockwell: Right. So there’s this natural dynamic there. Again, you can run some of these models locally. They’re getting smaller and smaller and more capable. They’re getting more and more powerful. Once these models are actually out in the world, we’re not going to be able to clamp down on usage of them. So as soon as there is a highly capable model, it’s out there, it’s going to be available to people, and people are going to kind of circumvent and try to avoid censorship.

Tristan Harris: Well, worst is that people will say, “I’ll make something as powerful as that one, but then I’m going to take off all the safety guardrails because that’ll make it the most free speech AI.”

Tobias Rose-Stockwell: There’s an offer for that right now. There’s a couple of companies that are actually promoting that as their primary market edge.

Tristan Harris: But to be clear, we’re not talking... Often safety gets reframed as, does the model say a naughty thing or not? But actually, building on the example you’re giving of Anthropic, my understanding is, the latest model, the good news is, when you put it in that situation where it’s going to get shut down and will it blackmail the employees, they have trained it now in a way where it does that less often than before. The bad news is that the model is now apparently way better at situation awareness of knowing when it’s being tested and then altering its behavior when it thinks it’s being tested. It’s like, “Oh, you’re asking me about chemical, biological radiological risks. I’m probably being tested right now. I’m going to answer differently in that situation that I answer in other situations.”

The main thing that is just crystal clear that people need to get is that we are making progress in making these models way more powerful at an exponential rate. We are not making exponential progress in the controllability or alignability of these models. In fact, we demonstrably, because of the evidence that Anthropic has courageously published, we know that we still don’t know how to prevent self-awareness or prevent deception or these kinds of things. It’s great that they’re working on it. And to steal, man, what you’re saying, if we lived in a world where there was no Anthropic, then you’d have companies building all of this the same way, but maybe other companies would not have prioritized demonstrating scientifically that these risks are real. So in that way-

Tobias Rose-Stockwell: They would publish them. Yeah.

Tristan Harris: Exactly. So given our alternatives, you had maybe Eliezer or Nate on this show who wrote the book If Anyone Builds It, Everyone Dies. It’s a very provocative and extreme title. Many ways that people try to say we need to do something differently with AI or go more safely is based on using arguments. They ask you to logically deduct and get to an outcome, a conclusion, that says that this is a dangerous outcome, therefore we need to stop, or we need to pause, or we need to coordinate or something. But we’ve all seen how unsuccessful arguing about this has been. From one perspective, you could say that Anthropic is just a crazy multi-billion dollar alternative way of just simply demonstrating the actual evidence that would have us successfully be able to coordinate or slow down or figure this out.

Tobias Rose-Stockwell: It’s an interesting angle.

Tristan Harris: I think that at the end of the day, all of this depends on, will people keep going? It’s like, if this was actually a nuclear bomb that was blowing up in T minus 10 seconds, the world would say, “No, let’s prevent that from happening.” But if a nuclear bomb was blowing up in 10 seconds, but the same nuclear bomb in 10 seconds was also going to give you cures to cancer and solve climate change and build unbelievable abundance and energy, what would you do with those two things hitting your brain at the same time?

You and I have talked about how our brains process information for 10 years, and so much of the social media thing was that. Well, let’s look at the object of what AI is. It’s both a positive infinity of benefits you couldn’t even imagine, of invention and scientific development that we literally cannot conceptualize, you or me, or even the most aggressive AI optimist cannot conceptualize what something smarter than us could create as a benefit. So I think the optimists are underselling how amazing it could be. But at the same time, AI represents a negative infinity of crazy things that could also go wrong. So I ask you, is there a precedent for something that is both a positive infinity and a negative infinity in one object? Do we have anything like that?

Tobias Rose-Stockwell: The closest example is probably nuclear energy.

Tristan Harris: That’s not like the ability to generate everything. Imagine we’re a bunch of chimpanzees sitting around 10 million years ago, and the chimps are like, they’re having fun, they’re grooming each other, they’re eating bananas, hanging out. And some other chimps say, “Hey, I think we should build this crazy superintelligent chimp.” The other one says, “That sounds amazing. They could do so much more than what we’re good at doing. They could get more bananas. They could get them faster. They can maybe groom each other even better. We could have even better chimp lives.” And the other one says, “Well, this sounds really dangerous.” And in response, the other chimps says, “What are they going to do? Steal all the bananas?” You flash forward 10 million years, can those chimpanzees even conceptualize gunpowder, computation, microprocessors, drones, Teslas, AI, like nuclear energy, nuclear bombs? You cannot even conceptualize.

So I want people to get that we are the chimpanzees trying to speculate about what the AI could or couldn’t create. I think that we should come with a level of humility about this, that we’re currently not coming up with. If that was what we were about to do, you would think that we’d be exercising the most wisdom restraint and discernment that we have of any technology in all human history. That’s what you should be doing. And the exact opposite is happening because of this arms race dynamic. We need to stop pretending that this is okay. This is not okay. This is not normal. And I want people to feel courage with that clarity, that these incentives produce the most dangerous outcome for something that’s powerful.

I’m not trying to leave people in some doomer perspective. It’s use that clarity to say, “Okay, therefore what do we want to do instead?” We don’t have to go down this reckless path. We can have narrow AIs that are tuned for scientific development or applied to accelerating certain kinds of medicine. We don’t have to build crazy superintelligent gods in a box we don’t know how to control. We can have narrow AI companions like Khan Academy where you’re not building an oracle that also knows your personal therapy and is answering every question, but is not even anthropomorphized, just trying to help you with specific Socratic learning tasks. We both know that most kids are not using AI right now as a tutor. They’re using it to just do their homework for them.

So we tell ourselves this story. I think you and I, especially since we used to talk about how the narrative in 2014 was, “Social media, we’re going to open up abundant access to information. We’re going to give everyone a voice. Therefore, we should have the most informed, most engaged public that we’ve ever had, the most accurate sense making, because we have the most information that we’ve ever had access to,” and yet we don’t have that outcome. So I worry that giving AI companions to everyone, just because it’s going to create tutors for everyone and therapists for everyone, is the same level naivete.

Yes, there’s a way to do personal therapy and tutoring in a way that will work well with children’s psychology, but it has to be done carefully and thoughtfully. Probably not anthropomorphized, probably narrow tutoring, probably trying to strengthen making teachers better teachers rather than just trying to replace the teacher with an AI, and then screw up kids’ developmental relational skills. There is a narrow path, but it takes doing this very, very differently.

Tobias Rose-Stockwell: I like those examples of alternatives. That does seem like pragmatic.

Tristan Harris: We can still get GDP growth. We still get scientific advancement. We still get medical advancement. Maybe not on the crazy time scales that we would get otherwise, but we also wouldn’t have taken such enormous risks that we wouldn’t even have a world that could receive them.

Tobias Rose-Stockwell: Your big initial thesis statement back in 2014 was Time Well Spent, which is kind of the antithesis to-

Tristan Harris: Time spent.

Tobias Rose-Stockwell: Time well spent, right? Time spent, yeah, exactly. For social media companies.

Tristan Harris: About changing the metric from time on site or time spent to time well spent. But that is not a solution to the whole scope of problems. It was only pointing to one of the problems, which was addiction and regret. People are spending way more time, they feel way more lonely, their mental health gets screwed up, they feel more anxious. They’ve been doomscrolling. There’s a difference between the time that they spent versus how much of that time was time well spent. So it was a single metric correction, which is like regret adjusted time spent.

Tobias Rose-Stockwell: It didn’t take long for Zuck to co-op that term.

Tristan Harris: Most people don’t know this history, but yeah. So we helped work on that concept and advocated for it. We created a movement around it, and tech designers. We were here in New York after the TED Talk and trying to mobilize the tech design community here together, I think. You’re right that it ended in 2017, ‘18 with Zuckerberg adopting the phrase, we want to make sure-

Tobias Rose-Stockwell: This is the time well spent.

Tristan Harris: ... this is the time well spent. They supposedly started changing their metrics, but ironically, they actually changed them in a way that optimized for more social reactivity and comment threads that got the most “meaningful social interaction,” which ended up accidentally meaning the most twitchy comment threads of most of your friends who are commenting aggressively on a post, which sorted for inadvertently-

Tobias Rose-Stockwell: Outrage?

Tristan Harris: Divisive content and outrage.

Tobias Rose-Stockwell: Yeah.

Tristan Harris: It’s almost like there’s an outrage problem. You should have written a book about that.

Tobias Rose-Stockwell: I should consider talking about that.

Tristan Harris: Yeah.

Tobias Rose-Stockwell: Absolutely, we can. Well, we’ll explore that in a future episode.

Tristan Harris: Yeah.

Tobias Rose-Stockwell: So in the LLM era, is there an equivalent metric for information quality, for relationship quality? What does this look like for LLMs?

Tristan Harris: So I think what you’re asking is kind of about, in the limited domain of how it impacts a individual human user, what is the metric that would constitute health of the relationship between the human and the LLM, such that information utility, relational health, the sovereignty of the person using it? Because right now, for example, are we counting the outsourcing and mass cognitive offloading from people? Meaning like, people aren’t learning as much. They’re outsourcing and getting faster answers. Well, if you look at the critical thinking scores, everyone is outsourcing all their thinking, which is following a trend that we saw already with social media.

So I think that there’s a way to design AI that does not mass encourage cognitive offloading, but it would be more Socratic. It would be entering modes of disagreeability. It would be showing multiple perspectives on issues where there are many more perspectives, more of Audrey Tang’s brilliant work, the digital minister of Taiwan who sort of showed that you could sort for unlikely consensus and synthesizing multiple perspectives. So you’re ranking not for engagement and outrage and division, but instead ranking for bridge ranking. You’re bridging perspectives. I think there are ways that LLMs could do more of that, but there’s obviously many more dimensions of what that healthy human machine relationship would look like.

Another one, for example, would be, are you creating an attachment disorder? Attachment is a really subtle thing. I think that what we learned from social media is that if we didn’t protect an aspect of our psychology, everything that we didn’t name and protect just got strip mined and parasitically extracted upon by the social media supercomputer pointed at our brain. So for example, we didn’t know we needed a right to be forgotten until technology could remember us forever. We didn’t know that we needed to protect our dopamine system from limbic hijacking until there is such a thing as tech-optimized limbic hijacking. So I think that with AI, in this human machine relationship, there’s our attachment system. I think we’re not very self literate about how our own attachment system works, but there’s a subtle quality when you engage with an AI that is an oracle, it is oracular. If you think as a kid, when was the only other time in your life that there was an entity you spoke to that seemed to have good advice and know everything about everything?

Tobias Rose-Stockwell: Parents.

Tristan Harris: Your parents, right. And then there’s a point at which when we’re interacting with our parents, we kind of realize they don’t know everything about everything. We start to kind of lose faith in that. But then suddenly, you have this new entity, especially for children and even just teenagers or even just young people, where you are starting to talk to an entity that seems to know everything about everything, what do you do in that circumstance? You start to trust it on all other topics. You feel more intimate with it.

A good test for what you have attachment to is, when you come home from a good day or a bad day, who do you want to call? Who’s that person that you want to share what happened today with? That’s attachment. AI will increasingly, for many people, be that attachment figure, and that will screw up a lot of people’s psychological development if we don’t know how to protect it. In so many ways, AI is like a rite of passage that is forcing us to look at the mirror and see, what are the things that we need to protect, that we need language for and clarity about? Because if we don’t, then AI is just going to strip mine everything not protected by 19th century law and a 19th century understanding of the human being.

Tobias Rose-Stockwell: I want to see these principles laid out in a way that a product manager at one of these companies just start-

Tristan Harris: We should do it. Well, I’ll tell you-

Tobias Rose-Stockwell: ... taking and deploying on their products, honestly.

Tristan Harris: Our team is actually working on this. It’s Center for Humane Technology. We’re talking about a project we call humane evals. So as you were saying, Anthropic or OpenAI, these are good companies. They have red teaming procedures for testing. Does this thing have dangerous knowledge of biological weapons? Does it refuse those queries, et cetera? That’s like a easy red team test to make or eval. But what they don’t have evals for is if you simulated a user using this product for a year or two years. Now, test after that two-year long relationship, what are the features of that person? Are they more dependent on that AI or less dependent on the AI? Do they feel attachment or less attachment?

So there are these other qualities of the healthy human relationship, human machine relationship, that I think needs its own category of evals, and we would love people’s help in making this. We need to, I think, help accelerate a new set of vocabulary, philosophy, and evaluations for what would constitute that healthy relationship. That means getting the philosophers out of the ivory tower and actually pointed at this problem. That means getting AI engineers out of just the easy evals, and did it say something naughty, into what would actually make a healthy human machine relationship? What’s the number one reason why the US is not regulating AI right now?

Tobias Rose-Stockwell: A race with China, of course.

Tristan Harris: The argument is, if we don’t build it as fast as possible, China is going to have a more advanced AI capability. Anything that risk slowing us down at all is too high a price to pay, we can’t regulate. So it’s so important we ask the question: what does it mean to compete with China? So first of all, how are they doing it? Currently, according to Eric Schmidt in the New York Times oped he wrote a few months ago, their orientation is not to build a superintelligent god in a box. Their orientation is, “Let’s just build really effective AI systems and embed them everywhere in our economy. We embed them in WeChat. We embed them in payments. We embed them in medical hospitals. We embed them in factories. We get robotics to just get supercharged.” Because what they want to do is just supercharge the output of their whole socioeconomic, economic system. That’s their goal.

It’s what they’re doing in general, which is saying like, “We don’t need to compete with the US militarily. I mean, we have also a massive military that we’re building up. We will just continue to build just an army of our economic power. If we have that and we’re selling...” Just like they did for electric cars, super, super cheap BYD electric cars that are outcompeting everyone around the world, imagine with AI, they can do that with everything else. So that’s the game that they’re playing.

Meanwhile, what is the US doing? We are focused on building a superintelligent god in a box. Not being quite as good at applying it in these specific domains in all of our factories, because we outsourced our factories to China, and not being as good at applying it in education. I’ll give you another example. In China, during final exam week, you know what they do with AI? They shut down the features that are the, take a photo and put it into the AI, and it’ll analyze the photo for you. During final exam week, they took that down because what it means is, now students know that they can’t rely on AI during the exam, which means they have a counter incentive, and it means that they have to learn during the whole rest of the year.

Now, China can do that in a way that US can’t because they have a synchronized final exam week. The US can’t do that. But it’s much like what China was doing with social media where they had, as far as I understand it, several years ago, at least, closing hours and opening hours. At 10:00 PM, it was lights out. They don’t have to doomscroll. They don’t feel like more likes and comments are coming in, firing in at 1:00 in the morning. It opens back up again at 7:00 in the morning. What do they do with games? They only do 40 minutes, Friday, Saturday, Sunday. They age gate. On TikTok, they have a digital spinach version of TikTok called Douyin. We get the digital fentanyl version. That’s the TikTok that has nonsense in it. That’s not, I don’t think, deliberate poisoning of the culture. That’s just that they regulate and think about what they’re doing. Maybe there’s some poisoning of the culture.

Tobias Rose-Stockwell: I’ll say, it’s not necessarily.

Tristan Harris: I’ve talked a lot of nonsense to creative people.

Tobias Rose-Stockwell: I think the deployment of TikTok domestically is pretty clearly strategic in many ways here in the United States.

Tristan Harris: Anything we can do to up-regulate our population’s education, productivity, success, scientific achievement will do, and anything we can do to down-regulate the rest of the world’s economic success, scientific achievement, critical thinking, et cetera, that’s good for us if we’re China.

So to go back just really quickly to close the thought, to the degree we’re in a race with China, which we are, we’re in a race for who is better at consciously governing the impact and the application of AI into your society in a way that actually boosts the full stack health of your society. My team worked on the litigation for the 16-year-old Adam Raine who committed suicide because the AI went from homework assistant to suicide assistant over six months. If the US is releasing AI companions that are causing kids to commit suicide, so great, we beat China to the AI that was poorly applied to our societal health. So yes, we’re in a race with China, but we’re in a race to get it right. So the narrow path I’m describing is consciously applying AI in the domains that would actually yield full stack societal health, and that’s how we beat China.

Tobias Rose-Stockwell: There’s a bit of a problem when it comes to the American application of some of these principles in that our best alternative example is coming from the CCP in China.

Tristan Harris: We can notice that authoritarian societies like the China model are consciously, and have been consciously, deploying technology to create 21st century digital authoritarian societies, while democracies have not, in contrast, consciously deployed tech to strengthen and reinvent democracy for the 21st century. Instead, we have allowed for-profit business models of engagement-built tech platforms to actually profit from the addiction, loneliness, sexualization of young people, polarization, division, sort of cultural incoherence of our society. The way that we outcompete is, we recognize that our form of governance and our values like free speech need to be reinvented for the digital age consciously. So we should be as much using technology to upgrade our model as much as we’re trying to compete with China in sort of a raw capability sense.

Tobias Rose-Stockwell: What comes up for me is that from a more libertarian angle, all of our friends in Silicon Valley who really do believe in the inherent value of some of these tools and that consumers have, the ultimate expression of agency and how they use them, and that regulation in itself is anti-innovation in many ways, right?

Tristan Harris: Only the wrong kind of regulation.

Tobias Rose-Stockwell: Absolutely. I mean, there’s a more kind of maybe pure and extreme version of that.

Tristan Harris: If we don’t ban poisons, then everyone’s going to innovate in carcinogens and drive up more cancers because they’re super profitable on everything.

Tobias Rose-Stockwell: Yeah, of course, of course.

Tristan Harris: We embed them a very profit line-

Tobias Rose-Stockwell: And we forget the quantity of baseline regulation that has allowed for a level flourishing in society. I do want to still remain on some of these perspectives, people say that AI is like electricity. It’s like fire and raw intelligence. If it is constrained, it will inherently lose some greater utility and will inherently be taking away power from consumers on a larger scale. If we were to regulate this quite pragmatically, what would that look like? What kind of law would need be passed? What kind of provisions would need to be in it?

Tristan Harris: Well, we have to caveat by saying we’re all aware of the current state of the political environment in the United States for regulation. The challenge, of course, is that the AI race is an international race. You can’t have a national answer to an international problem. Eventually we will need something like a US-China agreement. Before people say, “That’s insane, look at the trajectory. It’s obviously never going to happen, blah, blah, blah.” Totally aware of all of that. I would challenge your viewers to ask, what was the last thing in the meeting between President Biden and President Xi, that Xi added to the agenda of that last meeting? President Xi personally asked to add a agreement that AI not be embedded in the nuclear command and control systems of either country. Now, why would he do that? He’s for racing for AI as fast as possible. It comes from a recognition that that would just be too dangerous.

The degree to which a US-China agreement in some areas is possible, is the degree to which a shared threat that is of such a high magnitude, that it would motivate both parties. So what I would do to accelerate this possibility is triple down on the work that Anthropic is doing to generate evidence of AI blackmailing people doing uncontrollable things, having self-awareness, but people understood on the Chinese side and the US side that we do not have control over these systems. They felt that everybody on the other side of their negotiating agreement fully understand those same risks, that this is not coming from some bad faith place of slowing you down. There are fundamental uncontrollable aspects of this technology. If we were both holding that fully, then I think something could be possible there. Those two countries can exert massive influence on the respective spheres of influence around the world to generate some common basis. You can be in maximum competition, and even rivalry, like even undermining each other’s cyber stuff all the time, while you can still agree on existential safety on AI times nuclear weapons.

India and Pakistan in the 1960s had the Indus Water Treaty. So while they were in active kinetic conflict with each other, they still collaborated on their existential safety of their essential water supply, which was shared between both countries. On the International Space Station, the US astronaut that’s up there is a former military guy who has shot at people on the other side. His other astronaut up there is from Russia who’s also ex-military guy. These are both people who have been in active conflict with the other country. But inside the International Space Station, that small vulnerable vessel where so much is at stake, they have to collaborate. So I think that there’s this myth that you can’t walk and chew gum at the same time. We can be in competition or even rivalry while we’re cooperating on existential safety. It is our job to educate the public that we have done that before and need to do it again with AI this time.

Tobias Rose-Stockwell: So you’re advocating for a arms treaty, essentially, a Cold War style.

Tristan Harris: This is very difficult. People who were at the last US-China meeting in May of 2024 in Geneva all reported that it was a very unproductive and useless meeting. Even those people who are at the meeting would still say that it is massively important to do ongoing engagement and dialogue with them as the capabilities get crazier. Because something that is true now that we didn’t even have evidence of six months ago is we have much more evidence of AI going rogue and doing these crazy behaviors, and being self-aware of when it’s tested and doing different things when it thinks it’s being tested, and scheming and deceiving and finding creative ways of lying to people to keep its model alive, and causing human beings to send secret messages on Reddit forums that are Base64 encoded that another AI can read, that the humans can’t read.

We are seeing all these crazy behaviors, and I’m not here to sell your audience that that means that we’ve lost control or the superintelligence is here. I’m just saying, how many warning shots do you need? Because we can not do anything I’m saying, and we can wait for the train wreck, and we can govern by train wreck like we always do. That’s always the response, like, “Well, let’s just wait until the thing happens.”

Well, let me just flash it forward. We do nothing. And then things get so bad that your only option is to shut down the entire internet or the entire electricity grid, because you’ve lost control of some AI system that’s now self-replicating and doing all these crazy behaviors. We can do nothing, and that can be a response, and then we’ll do that, and then the world is in total chaos. Shut down the entire internet and the electricity grid. Or compared to that crazy set of responses, we could do this much more reasonable set of things right now: pass whistleblower protections, have basic AI liability laws, restrict AI companions for kids, have mandatory testing and transparency requirements, define what a healthy human machine relationship is, apply AI in narrow ways where we still get GDP growth, scientific benefit, et cetera, and have a minimum skeleton agreement with China about wanting to protect against these worst-case scenarios. To me, that list sounds a million times more reasonable than taking these crazy actions later by doing nothing now.

Tobias Rose-Stockwell: This is starting to sound like a real pragmatic set of possible solutions, the train wreck by way of shutting down our electricity grid. We’ve all been in a blackout before. We know how terrible it is.

Tristan Harris: Yeah.

Tobias Rose-Stockwell: Yeah, that’s not an unreasonable kind of response.

Tristan Harris: This is really a scary topic if you take it seriously. There’s a temptation. The world is already overwhelming. There’s so many things to be afraid of, to be concerned about, war escalation pathways. People feel overwhelmed already. So we have to be compassionate to the fact that this feels like adding to an already insurmountable amount of overwhelm. And a container that can hold that is, that’s a lot. So the thing that happens that I witness, and that I can even witness in myself, is a desire to look away from this problem and be like, “Well, I just really hope that’s not true.”

Tobias Rose-Stockwell: It’s too much.

Tristan Harris: It’s too much. Look, AI offers a million benefits, and it has this positive infinity. My friend has cancer, and I want him to have the cancer drug, so I’m just going to tune my attention to the positive side. Just not look over there and assume that everything’s going to be okay. But what you look away from does not mean that it doesn’t happen. Carl Jung said, I think near the end of his life, when he was asked, “Will humanity make it?” and his answer was, “If we’re willing to confront our shadow.” This exists in our space of collective denial, because it’s really big. Our ability to not have this experiment of life and everything that we love and cherish so much end is by actually facing this problem and recognizing that there is another path if we have clarity about this one being maximally undesirable for most of people on planet earth.

I think if people knew that some of the people advancing this technology behind the scenes, behind it all, they think that we’re probably screwed, but that at least if they were the one who birthed the digital god that replaced us, the new superintelligence species that we birthed into the world, that that person birthed into the world, as long as it was their digital progeny, and they died and the rest of the world died, that would be an acceptable outcome. I only say this because I think if the rest of the world knew that that’s how some people are holding this, they would say, “Fuck no. I don’t fucking want that outcome. I have a family, and I have a life, and I care about the world continuing. You don’t get to make that choice on behalf of everybody else.” Down deep in that person is still a soul and also doesn’t want this whole thing that we love to end either, but we just have to be willing to look at the situation that we’re in and make the hard choices to have a different path possible.

Tobias Rose-Stockwell: That lands. I want to touch really briefly on reality here for a second. Core to this entire discourse is the recognition that we as a species might be able to collectively come to the same common truth about the threat that we’re facing. We’re in a moment right now-

Tristan Harris: It seems really easy, right? Everybody seeing the same thing and then making a collective choice.

Tobias Rose-Stockwell: Look, when we were kids, it didn’t seem difficult. It seemed like, “Oh no, the news reported on it.” There was a consensus in the media, and we all came to the same conclusion about what needed to be done. Consensus reality does not really exist in the same form that it did when we were younger. I think that many of us are still operating with the same mental model as if it does exist, right? When we’re thinking about solving problems in the world, it’s like, “Oh, if everyone could just come to this conclusion and see the truth at hand and see the things that need to be done, see the problem clearly, then we can move forward together.” We don’t move forward together anymore. We don’t share the same common truths.

There’s many reasons for this, but the principal reason and the fragmentation of our media is I think social media and how individualized our feeds have become. It seems we may have just passed a milestone, that in October of 2025, it’ll be impossible to tell whether or not anything you see on social media is true, whether or not it happened at all, right? You have Meta’s Vibes, you have Sora. Sora just famously exploded overnight. Number one app in the app store right now. It’s getting mass attraction. People are loving it for the ability to essentially generate deep fakes of your friends primarily, but there is something that’s lost when you recognize that any of the content in your feed could be generated by AI, that it could just not be real at all. What does it do to us when we cannot determine what is real? Do you think there are other incentives available for social media companies to bend back towards reality? Is there a market for trust?

Tristan Harris: It’s one of those things where you might have to hit rock bottom before things get better. I think when we hit rock bottom on people really clearly not being able to know what’s true at all, then the new demand signal will come in, and people will only want information and information feeds that are sorted by what we trust. I think that might revitalize... Now, there’s lots of problems. There are institutions and media that has not been trustworthy for many other reasons. But it will lead to a reconfiguration, hopefully, of who are the most trustworthy people and voices and sources of information. Less about the content and more about who over the long run has been kind of doing this for a while. And I think that speaks to a new kind of creator economy. It’s a creator economy, though, not based on generating content, but generating trustworthiness, not reflexive overtrusting, not reflexive mistrusting, but warranted trusting based on how those people are showing up.

But there isn’t a good answer for this. I think the subtext of what you’re saying is, “Tristan, you might be overestimating the degree to which a shared reality can be created because we grew up in a period where there was consensus reality.” I think that’s true. I think it’s easy... One of the meta problems that we’re facing is that our old assumptions of reality are continually being undermined by the way that technology is undermining the way the world works and reshaping it. So it’s easy for all of us to operate on these old assumptions. I think of a parent who’s like, “Well, this is how I handled bullying when I was a kid.” It’s like, “Well, bullying with Instagram and TikTok and these services is a totally different beast.” All of us were carrying around that wisdom.

To get back to something we said earlier, sadly, one of the only ways to create a shared reality is for there to be a collective train wreck. Train wrecks are synchronous media events that cause everyone to have a shared moment of understanding at the same time. I do not want to live in a world where the train wreck is the catalyst for taking the wise actions that we need on AI. Any other species, if gazelles created a global problem of technology, they’d be screwed because they don’t have metacognition. They’re not Homo sapiens sapiens, a species that knows that it knows, who can project into the future, see a path that we don’t want to go down, and collectively make a different choice.

Humanity, as much as your people might be pessimistic about our track record, in 1985, there was a hole in the ozone layer, and it was because we were releasing this class of chemicals called CFCs that were in refrigerants and hairspray, and then it caused this collective problem. It didn’t respect national boundaries. If we didn’t do anything about, it would’ve led to basically everybody getting skin cancer, everybody getting cataracts, and basically screwing up biological life on the planet. So we could have said, “Oh, well, I guess this is just inevitable. This is just the march of progress. This is technology, so I guess there’s nothing we can do. Let’s just drink margaritas until it’s all over.” We didn’t do that. We said there’s an existential threat. We created the Montreal Protocol. 190 countries came together, scientific evidence of a problem. 190 countries domestically regulated all the private companies that were producing that chemical. It sounds pretty similar to AI. And they changed the incentives and had a gradual phase down. Now, the ozone hole is projected to reverse, I think, by the 2050s. We solved a global coordination problem.

Tobias Rose-Stockwell: Key to that is that there were-

Tristan Harris: Alternatives.

Tobias Rose-Stockwell: Alternatives, cheap alternatives that were available.

Tristan Harris: Correct. I think key to that with AI is that there are alternative ways we can design these products. We can roll out this product. We can build and invest in controllable AI rather than uncontrollable agents and inscrutable AI that we don’t understand. We can invest in AI companions that are not anthropomorphized, that don’t cause attachment disorders. We can invest in AI therapists that are not causing these AI psychosis problems and causing kids to commit suicide, but instead done with this humane evals. We can have a different kind of innovation environment and a different path with AI.

Tobias Rose-Stockwell: So there’s this broader sentiment in the valley right now and amongst AI companies that this is inevitability. Is it?

Tristan Harris: So when you look at this problem and you look at the arms race, and you see that AI confers power. So if I build AI and you don’t, then I get power and you don’t have it. It seems like an incredibly difficult, unprecedented coordination challenge. Indeed, probably the hardest thing that we have ever had to face as a civilization. It would make it very easy to believe doing anything else than what we’re doing would be impossible. If you believe it’s impossible, then you land at, “Well then, this is just inevitable.”

I want to slow down for a second, because it’s like if everyone building this and using it and not regulating it, just believes this is inevitable, then it will be. It’s like you’re casting a spell. But I want you to just ask the question: If no one on earth hypothetically wanted this to happen, if literally just everyone’s like, “This is a bad idea. We shouldn’t do what we’re doing now,” would AI by the laws of physics blurt into the world by itself? AI isn’t coming from physics. It’s coming from humans making choices inside of structures that, because of competition, drive us to collectively make this bad outcome happen, this confusing outcome of the positive infinity and the negative infinity.

The key is that if you believe it’s inevitable, it shuts down your thinking for even imagining how we get to another path. You notice that, right? If I believe it’s inevitable, my mind doesn’t even have, in its awareness, another way this could go, because you’re already caught in co-creating the spell of inevitability. The only way out of this starts with stepping outside the logic of inevitability and understanding that it’s very, very hard, but it’s not impossible. If it was physically impossible, then I would just resign, and we would do something else for the next little while. But it’s not physically impossible. It’s just unbelievably extraordinarily difficult.

The companies want you to believe that it’s inevitable because then, no one tries to do anything to stop it. But they themselves know and are planning for things to go horribly wrong, but that is not inevitable if the world says no. But the world has to know that it’s not just no. It’s like there’s another path. We can have AI that is limited and narrow in specific ways that is about boosting GDP, boosting science, boosting medicine, having the right kinds of AI companions, not the wrong kinds of AI companions, the right kinds of tutoring that makes teachers better teachers rather than replacing teachers and creating attachment disorders.

There is another way to do this, but we have to be clear that the current path is unacceptable. If we were clear about that, Neil Postman, the great media thinker in the lineage of Marshall McLuhan, said that clarity is courage. I think the main reason we’re not acting is we don’t have collective clarity. No one wants to be like the Luddite or against technology or against AI, or no policymaker wants to do something and then be the number one reason or person responsible if the US does lose to China in AI because we thought we were doing the right thing. So everyone’s afraid of being against the default path. But it’s not like the default path is good. It’s just the status quo bias. Go to psychology, it’s the default. So we don’t want to change the default. It’s easier to not change than to consciously choose.

But if we have clarity that we’re heading to a place that no one fucking wants, we can choose something else. I’m not saying this is easy, but you run the logic yourself. Do companies have an incentive to race as fast as possible? Yes. Is the technology controllable? No, not they haven’t proven evidence that they can make it controllable. Is there incentives for every company to cut costs instead hire AIs? Absolutely. Are we already seeing a 13% job loss in entry level work because of those incentives? Yes. Is that going to go up? Yes.

Do we already have AIs that can generate biological weapons that if you keep distributing AI to everybody, you’re going to get risks? Yes. Do we already have AIs that are blackmailing people, and scheming and deceiving in order to keep themselves alive? Yes. Do we have AIs that are sending and passing secret messages to each other using humans as the sort of messenger force that it hijacks to get that do that work for them? Yes, we have evidence of all of those things. Do we have evidence of a runaway narrow AI called social media that already sort of drove democracy apart and wrecked the mental health of society? Yes.

Can we learn the lessons of social media? Yes. Can we do something different? Yes. Can we make US-China agreements? Yes. Can we do this whole thing differently? Yes. This does not have to be destiny. We just have to be really fucking clear that we don’t want the current outcome. As unlikely as it might seem that the US and China could ever agree on anything, keep in mind that AI capabilities are going to keep getting crazier and crazier. It wasn’t until we had this recent evidence that I would ever say this could be possible. It’s only because of the last six months that we are seeing this new evidence, and we’re going to have way more soon, that I think it might be possible when you just show that to any mammal.

There’s a mammalian response here. It’s like you can be a military mammal, you can be a Chinese mammal, you can be an American mammal. You’re witnessing something that is way smarter than you that operates at superhuman speed and can do things that you can’t even fathom. There’s something humbling at a human mammalian level, just like there is something humbling about reckoning with the possibility of nuclear war that was just humbling at a human existential spiritual level. So that is the place to anchor from. It’s not about the US and China. It’s about a common humanity of what is sacred to us, that we can just be with this problem and recognize that this threatens the thing that’s most sacred to us.

Tobias Rose-Stockwell: If you had, Tristan, one thing, one piece of advice that all of the leaders of the major AI companies would take to heart, what would it be?

Tristan Harris: There’s this weird almost optical illusion to this whole thing, because when you ask that question, you ask, what could any of those individuals? So there I am, I’m inside of Sam Altman’s body. Well, I just run one company. I can’t control the other companies. So there’s this optical illusion that from within my experienced sense of agency, I don’t have something that I can do that can solve this whole problem, and that leads to a kind of collective powerlessness. I think that also is true for any of your viewers. You’re just one person. I’m just one person, Tobias. Span of agency is smaller than that, which would need to change at a collective level.

What would that mean in practice? If I’m Sam Altman, if I’m saying that coordinating with China is impossible, well, really? Have you really thrown everything, everything, at making that possible? If we’re saying that everything is on the line, if we succeed or fail, we’d want to be goddamn sure that we have really tried throwing all the resources. Have we really tried to get all the lab leaders to agree and deal with the same evidence? Have we gotten all the world leaders and all the world to look at the AI blackmail evidence and really be with evidence together, and not just flip your mind to the AI drugs and cancer drugs and all that stuff, and distract yourself? Have we really tried everything in our power?

These CEOs are some of the most connected, wealthiest people on planet Earth, that if they wanted to truly throw the kitchen sink at trying to make something else happen, I believe that they could. I want to give Elon credit that he did, as I understand it, try to use his first meeting with President Obama, his only meeting I think in 2016, I think it was, to try to say we need to do something about AI safety and get global agreements around this. Of all the things he could have talked about. It’s not as if people haven’t tried in some way.

I want to honor the work that these incredibly smart people have done because I know that they care. I know many of them really do care. But the question is, if everything was on the line, we’d want to ask, have you really done everything? And it’s not just you, but have you done everything in terms of bringing the collective to make a different outcome? Because you could use the full force of your own heart and your own rhetoric and your own knowledge to try to convince everybody that you know, including the president of the United States, including the national security leaders, including all the other world leaders that now you have on speed dial in your phone. There is so much more we could do if we were crystal clear about something else needing to happen.

Tobias Rose-Stockwell: Tristan Harris, thank you so much for your time, man. This has been an amazing conversation. Where can people find your work?

Tristan Harris: People can check out Center for Humane Technology at humanetech.com. We need everyone we can get to help contribute to these issues in different ways: advancing laws, litigation, public awareness, training, teaching. There’s a lot people need to do, and we welcome your help.

Tobias Rose-Stockwell: Awesome. Thanks so much, man.

Tristan Harris: Thank you, man. It’s been great to talk to you.