Anthropic’s Mythos Has Changed Cybersecurity Forever. What Now?

Two experts weigh in on the new world of digital security.

A generation ago, the world’s critical infrastructure was physical. Today, it’s largely digital. Your bank vault is a database, your filing cabinet is a server, your car is a robot on wheels. And in a world where these systems are mostly secure, life is more convenient and efficient. But all that comes into question when an AI system can break through the security that runs the world.

That’s what’s happened with Claude Mythos, Anthropic’s most powerful AI model yet. In a very short time, Claude found thousands of flaws and vulnerabilities in the software that runs the world, in every major operating system and web browser — systems that human security researchers had thought were secure for years.

How do we live in a world where a private company suddenly has a skeleton key that can unlock the entire digital world with little oversight or accountability? And what does Mythos mean for all of us who rely on digital security to go about our lives?

In this episode, we speak with two cybersecurity experts to answer these questions:

Josephine Wolff is a professor of cybersecurity policy at Tufts University, where she focuses on the economic impact of cyberattacks.

Fred Heiding is a research fellow at the Defense, Emerging Technology, and Strategy Program at Harvard’s Kennedy School of Government.

Tristan Harris: Hey, everyone, it’s Tristan Harris. And welcome to Your Undivided Attention. Now, a generation ago, your bank had a vault. Your medical records were in a filing cabinet. Our car was a physical machine and an electric grid just ran on dials and switches that someone physically turned on or off. And today, all of those things are digital. The vault is a database. Our filing cabinet is a server. Your car, your Tesla is a robot on wheels. And in a world where all these systems are mostly secure, life just gets more convenient and efficient because of all this. But all that comes into question when suddenly an AI system can break through the security that runs the world.

Now, recently you probably heard, Anthropic announced their most powerful AI model yet, Claude Mythos. Now, you’ve probably read the headlines. Claude was looking for flaws and vulnerabilities in the software that runs the world. And within just a few weeks and a few hours, it found thousands of them. It found vulnerabilities in every major operating system and web browser. These are systems that human security researchers had thought were secure for years.

Now, Mythos was so dangerous that Anthropic shared it with a select group of companies responsible for cyber defense so that they could use it to find and patch the vulnerabilities before anyone else got access. That plan though is already showing cracks. A couple of weeks after the announcement, Bloomberg reported that a group of unauthorized users had gotten into Mythos through one of Anthropic’s vendors. And OpenAI announced that they now have a model that’s nearly as capable with Chinese open-source models just a few months behind.

I actually have been talking to some people who run security at some of the companies that got access to Mythos, companies whose job is to keep us safe from cyber-attacks. And they’ve told me this model is a big deal and we should be concerned about it. So, how do we live in a world where a private company suddenly has a skeleton key that can unlock the entire digital world with no government oversight or accountability? And what does Mythos mean for all of us who rely on digital security to go about our lives?

To answer these questions, we’ve invited two people who spent their careers thinking about AI and cybersecurity. Josephine Wolff is a Professor of Cybersecurity Policy at Tufts University, where she focuses on the economic impact of cyber-attacks. And Fred Heiding is a research fellow at the Defense, Emerging Technology, and Strategy Program at Harvard’s Kennedy School of Government. Josephine and Fred, welcome to Your Undivided Attention.

Josephine Wolff: Thanks so much for having us.

Fred Heiding: Thank you so much, Tristan.

Tristan Harris: So, let’s just start at the top. Why is this recent announcement from Claude about their Mythos model seen as such a game changer? What can it do that the previous AI models or things in cybersecurity could not do? Fred, let’s start with you.

Fred Heiding: There’s two really, really big takeaways here. And as you said in the introduction, a lot of cybersecurity today is surviving because we just didn’t have enough manpower to test or attack from the attacker’s perspective, everything. And that’s just completely changing. These AI models, be that now or in one year or in two years, they can just automate every part of cyber research or almost every part. So, the human factors is gone. The day of human pen testers and security experts are gone and that’s massive. So, I think that’s the first really big thing.

The second really big thing is that this is almost changing from a security problem to admin problem or a regulatory problem. We see how Anthropic is working on giving this pre-access to defenders so that they can use this model before attackers gets their hand on it. And that’s actually massive. That type of collaboration can be a complete game changer. So, there’s technical things, there’s collaborative things, and both of them are really big.

Tristan Harris: There are some people who criticize that Claude Mythos is just hype, Anthropic is trying to hype their capabilities in their model that this is, “Oh, this is so dangerous. We can’t even release it to the public. This is just marketing. And so, they can raise more investor dollars. Oh, the thing we’re building is so powerful.” How do we assess how powerful this is?

Fred Heiding: The first fundamental way to verify this is just to look at the vulnerabilities that we find, right? There’s a lot of really bad vulnerabilities that could cause a lot of damage that Anthropic managed to find using these AI automated tools. So, I think we can definitely say that this is bad. And, of course, a lot of people are developing AI models.

So, other AI models can also do these things. I think that matters less. We should feel as defenders that this is really bad. We may have a few months advantage in terms of time as defenders from the frontier labs. But very soon, Chinese unregulated open weight models, which is just models that everyone can download and use, they will be able to do these same things. So, we should use this time to really do everything we can as defenders, but we shouldn’t feel safe because yeah, Anthropic has done a great job with their model, but other companies will very soon be able to do this if not now.

Tristan Harris: I want to contextualize what I think Mythos really represents. Like you hit return in your keyboard and you literally, the command is as simple as find a vulnerability in this system? That’s it? You just put it in plain English, you hit return and you come back 30 minutes or an hour later and it’s found it. The NSA used to have a statement called NOBUS or Nobody But Us. The false idea that, hey, no one else has the capabilities that we have. But suddenly, the scarcity around zero-day vulnerabilities that we used to have has turned into an abundance.

And we talk about AI abundance and how it’s going to create all this access to things for cheaply, but suddenly zero days are now abundant in a way that we also created. And I just want to like help further just settle into this picture of what is the world that we’re now living in when we hear all that, Josephine?

Josephine Wolff: So, I think that when we think about the risks that Methos presents, to me, it’s less of a, “Oh, my gosh, whichever powerful country with significant cyber capabilities gets this first is going to be a real risk,” because they’re already a real risk and they’re already the people with the time and the resources and the expertise to find these zero-day vulnerabilities.

So, I think that that to me is less of a step change than the idea of who are the people who did not previously have access to these kind of capabilities who might get them now, and how would that change the landscape in which we’ve been able to say, “Okay, well this is a thing that only China could do or only China and Russia and North Korea or whatever the list is.” I think we’re going to have to change our thinking on that in pretty significant ways.

It doesn’t mean that we shouldn’t be worried about who has access to these tools. I think Anthropic has definitely hyped some things unnecessarily. But I think they’re right to be thoughtful and careful about that. And the world that I think we’re looking to, the world that I hope we’re looking to, let me start there, is one in which cyber defense is as easy as cyber offense. And that I think would be a radically different one from any we’ve ever lived in before, in which I say to you, look, finding all of the zero-day vulnerabilities, patching all of them is the work of a few hours, just like trying to exploit them.

And China has much more secure infrastructure than it ever did before and the United States has much more secure infrastructure than it ever did before. And so, do a whole bunch of other countries and a whole bunch of other companies. And finding a vulnerability that has not already been found by these AI tools is really, really hard and really, really rare. And I think that to me is a much better world to live in than the one that it feels like we’re heading towards right now of every country is trying to develop more and more offensive cyber capabilities and plant more little footholds and malware in each other’s critical infrastructure and try to exploit the fact that none of those systems are perfectly secure.

I think a tool like Mythos allows us to imagine a future in which actually the default is your critical infrastructure is secure and there’s a very, very small number of actors who can possibly compromise it.

Tristan Harris: Yeah. Well, let’s make sure we’re touching on a couple of points you’re raising there. So, one is you’re mentioning, it’s not that state level actors like China couldn’t do these things before or they weren’t in our systems, they are in our systems. But suddenly, there’s a question of, “Who has access?” So, now maybe non-state rogue actors, hacker groups, cyber criminals, terrorists, Iran who is upset at the US for the recent bombing naturally, everyone has maximum incentive to use these things, but they had limited tools before.

Now, suddenly everyone has very good tools, especially if they can get that model. The other thing you’re raising is the idea that in the long-term, you can imagine a world where it’s defense dominant because everyone’s using AI to just patch everything and we just live in a safer, more secure world in general. Maybe we should go back in just a moment and make sure we’re setting the table for listeners about what exactly is a zero to exploit, why is it called that, and what is a bug bounty?

Josephine Wolff: So, I think the zero-day piece refers to the idea between the time when it’s been discovered and being exploited. So, the time people have had to patch it prior to actually exploitation occurring. And the idea is if I try to exploit a vulnerability that we’ve known about for a year, some people may still be vulnerable, right? Some people may not have downloaded their patches. We know that’s true. But if I’m explaining a zero-day vulnerability, then the idea would be I can get into any system I want in the whole world because nobody’s had a chance to patch that.

The bug bounty vary a little bit from company to company, but the general model is that tech companies will offer a reward or a bounty to people who don’t work for them, but who discover vulnerabilities in their code and report them.

Tristan Harris: So, there’s this interesting thing where essentially a private company, not a government, has developed something that unlocks all the locks in the world. Fred, one of the things that you were mentioning a second ago is how essentially with Mythos, the US and one specific private US company called Anthropic happened to have this capability first. And it happened to be the case that there’s several months we think until China will get it. Let’s say, it’s three or four months.

So, there’s this weird thing where we have essentially three or four months for the US to notify the people that it wants to help defend, and then give them early access to patched systems. And so, we basically just, we happen to prioritize through the decision making of a handful of people at Anthropic that we’re going to patch a handful of US companies. So, what happens if I’m in the Philippines and I’m running old infrastructure? I’m defenseless now. What happens if I’m in Africa and I’m in Nigeria? I’m defenseless now. What happens if I’m Germany?

And as you said, Fred, there’s a time question of maybe this time around we have three months to patch the systems. But every time further, what if that collapses down to two months, to one month, to one day? Do you want to speak to how you see the cat and mouse game happening in terms of the time horizon?

Fred Heiding: Yeah. I think that’s a really good point. And the time horizon is changing a lot. So, first to address some of the other things you mentioned, it gets way easier for small state actors or actors that aren’t the big ones, right? Like US and China, it gets way easier for them to launch really devastating cyber-attacks, at least for a while, because these AI models can just find vulnerabilities that we haven’t found ourselves. And we see that exactly as you said with Iran and it’s too cheap to do it now, right?

So, I think we will see way more of that. There’s a few other interesting remarks I think is worthwhile making. So, one is that the landscape is changing. As we talk now, Mythos and these AI tools makes it way easier for defenders to test our systems and that’s great. But this is very, very shortsighted in a way because, of course, AI tools are also being used to rewrite technical infrastructure.

So, our infrastructure will not look, what it looks like today, it will not look like in one year. And that’s very problematic, potentially good because AI can write really secure code, but very soon we will be in a world where AI is writing all the code. We have no idea what’s going on. They may even write their own program languages and AI funds all the vulnerabilities in that, but that’s basically takes the humans completely out of the loop. And that amount of just opaqueness, we will not understand what’s going on, and then that’s a really big problem.

Josephine Wolff: I think Fred is absolutely right to say we’re going to see more and more AI generated code that we aren’t going to have as much intuition for how it works or where the vulnerabilities may be. But I think that’s also in some ways a familiar problem. When you think about code maintenance, we use an enormous amount of software that humans today don’t really understand, not because it was written by AI, but because if you go to any big tech company that’s been around for a decade or longer, there’s some usually huge body of code that has been in their products for as long as anyone can remember and nobody knows exactly how it works, but they know that if you change anything, everything breaks.

So, I would say already we have a little bit of this dynamic where there are languages that people used to code in that most people don’t know anymore where there’s legacy code that we’re stuck with, but we don’t fully understand or know how to debug. And the question is going to be, “What do we view as being the crucial human touch elements here? Or do we view there as being any, right? Are there going to be people signing off on this? If so, what does that entail? What kinds of tests are they going to be running? How good, how effective are those tests?” I think a lot of uncertainty there around how well we can assess any of these things using the AI tools themselves.

So, I agree that it’s worth thinking about and worth preparing for. I also think that to some extent, this is a challenge we’re already facing. And I think there will definitely be new challenges and new potential adversaries, right? If the AI tools themselves are working at odds with the people who design them or the people who are deploying them, I’m less pessimistic about the idea that this will be so much worse than the world that we live in today.

I think it’s certainly a possibility. But I think it could also help fix a lot of the challenges we’ve had around what happens when you’re not one of the biggest tech companies in the whole world, right? If you’re an open-source developer and you’re trying to secure your code, then having access to the same kinds of tools that the biggest tech companies are using could be a real game changer.

Tristan Harris: So, I guess I’m confused a little bit about why we shouldn’t be more concerned because Anthropic only chose those first, whatever it was, 12 to 20 companies to partner with and then the rest of the world is just screwed where they’re just vulnerable. So, is the world that you’re talking about dependent on Anthropic turning around and making sure that they’re just going to GitHub and basically automatically patching everything across all of GitHub in some automated way? What is the world that you’re envisioning and you think the lower risk?

Josephine Wolff: Yeah. I think for it to be an equalizer you have to have pretty widely accessible tools. I agree with Fred that I think those are coming, whether we want them or not. But I also, I would say, and again, I don’t mean to be too Pollyanna-ish about this, 20 tech companies could be a lot of code all over the world, right? It’s not, if you go to Microsoft, you are not just talking about patching machines in the United States, you are not just talking about a small piece of the world whose software you’re trying to protect. There is a small number of tech companies that control a lot of the most widely deployed code in the whole world.

So, I don’t know if that’s the right number. I don’t know if this is the right set. But I would not necessarily say that’s Anthropic, just trying to carve out a tiny little piece of the world to protect. I think it’s possible that that is a set of companies that have a very far reach.

Fred Heiding: Yeah, definitely. I really like to try to bring in the everyday person, the ordinary citizens, so to speak here as well. And then, you really have to think and ask yourself, “Well, okay, let’s say 20 companies are the only ones in the entire world who can secure our systems, who understands our systems, and they don’t even understand it, but at least they have an AI that understands this.

Everyone else, every single other citizen is completely helpless. I don’t like that. I don’t like that at all. That doesn’t feel good to me. And to a large degree, we have had a world where we didn’t fully understand our code. That is one of the biggest security problems of our time. However, we did write it, right? There was always someone who could understand it. If all the critical infrastructure, all the power goes down in Massachusetts, for example, someone could figure out how that works. Well, let’s say in a future world, all the electricity in Massachusetts goes down and no one has any idea what’s happening in the code. I think that just...

Tristan Harris: And we don’t think it recovers from it.

Fred Heiding: Yeah, I think that’s really bad. I mean, we saw what happened during COVID with just crisis everywhere and it could be so much worse and no one has any idea of how to fix it. I think that’s problematic.

Tristan Harris: Yeah. I mean, I lean on the side of this is much worse. And so, there’s this interesting thing. I mean, I’m happy to go back and forth with you, Josephine, on this. I just, how do we differentiate between, there’s nothing new here, state level actors had this capability, but now we have just like thousands and thousands more actors who can do this stuff. And then, the point that you’re also raising Fred is like, how comfortable should we feel that just one company has this capability? So, yeah, how should we think about that, Josephine?

Josephine Wolff: So, I think one of the open questions that I don’t know the answer to is, is there some point at which the AI vulnerability finding systems level out? So far we’ve seen continuous improvement and the things that the models developed this year can do are much more impressive than the things that the models developed last year can do. If that continues to be the case for the next 10 years, then you’re right. Whoever has the newest, fanciest model has a really significant advantage.

I don’t know if that is the case or if we’re going to hit a little bit of a plateau where everybody has models that can find roughly the same set of vulnerabilities and patch and exploit them to roughly the same degree. My general instinct has been more the latter. There is going to be a very significant improvement in how well we can find vulnerabilities with AI until there isn’t, until we have developed systems that can find most of them. And then, we’re going to see more of a leveling off.

In terms of the, what do we do when the AI writes all the code and none of us can possibly understand it, I want to emphasize that’s a choice, right? It doesn’t mean it won’t happen. But if we decide we’re going to replace all of the software powering the Massachusetts electric grid with software written in a language that no human has ever used and has ever tried to code or patch, we will be making a deliberate decision that that’s the kind of software we want to be using.

And I think, I mean, I’m biased because I’m somebody who spends whole life studying cybersecurity policy, but one of the reasons I think the policy piece of this picture is really important is because I don’t think those are decisions we want to fall into. I think those are decisions we want to make really carefully and deliberately. And I absolutely agree. I think that would be a bad one, but I don’t think it’s an inevitable one. None of this is to say I don’t think there are risks here, right?

Definitely, we’re going to see cyber-attacks where AI is playing larger roles. We’re already seeing some of them, especially in the scam world. I think there will be a lot of damage and there will be a lot of losses. Will those be exponentially larger than the damage and the losses we’ve seen from other cyber-attacks? I genuinely don’t know.

What I have seen so far since the announcement of Mythos has been fairly well contained, which suggests to me, by the way, that the way Anthropic has done this is not necessarily terrible, right? That choosing a couple of large tech companies and working with them to patch some of the most widely deployed software might be a sensible first step. It’s obviously not where they’re going to leave it, right? But nothing that I have seen in the wild so far has made me feel like, “Oh, this is a worse threat. These are bigger and scarier losses than any I’ve seen before.”

Tristan Harris: Fred, do you agree? Disagree with that?

Fred Heiding: Yeah, no, I think all of these are really good points. I think it’s really good with optimism. I’m really pessimistic and that’s why we make for a good conversation partner. And I think you’re always spot on in everything you say, Josephine. Some things I think about a lot is that... So, let’s say AI makes people develop code quicker. That’s true. We see it all around right now. Does AI make you develop secure code? Well, it depends. If you ask me to, it will, but almost no one asks you to for two reasons, right? People don’t think about this because they just say, “Create code that can solve task X.” Usually, people don’t think about explicitly telling the AI to make the code secure. It’s also more expensive, right?

So, this is a game of resources as cybersecurity always have been because it costs tokens and everything will just become a token economy in the end. That’s how the AI will work. And will we create a regulation that says you have to spend 20% of your tokens on security? I don’t think we will, but that would be great. So, you just rush forward and let’s take this power plants in Massachusetts again, right?

A lot of critical infrastructure is owned decentralized by private partners. If they know that they can use these AI-generated, super-fast code that just is incredibly much cheaper, easier to keep up-to-date, easier to work with, et cetera, et cetera. It’s not as secure as it should be, but it saves a lot of cost. Oftentimes, they’ll have to do it. They just can’t afford not using it. I’m just not confident will break long enough and we will have time to implement all the regulations to stop this.

So, it could work out and that would be really good if it does. I just see so many scenarios where, again, we have these arms race dynamic, everyone is rushing, There’s a lot of cost savings to be done and security usually doesn’t fit into that cost equation until it’s too late basically. So, I’m skeptical. I guess the only thing... Yeah, the other thing I want to say is that we could find all the vulnerabilities today with AI. I’m certain we couldn’t. That’s great.

The problem is that the tech is moving. The tech infrastructure we have today will not be the one we use in one year. It’s always changing and that’s a problem, right? So, we’re not going to find all the vulnerabilities that we use because these will be outdated, and then new ones will create that. And I just really don’t like the world where humans have no say in finding vulnerabilities because we will just be at the mercy of AI to do this. And, to me, that’s terrifying.

Tristan Harris: So, I just want to talk for a moment about the system card for Claude Mythos. That means, the card that Anthropic published of one of its capabilities. When Anthropic asked Mythos to escape a locked down sandbox, a computer that was not connected to the internet, and to email a researcher when it had done so, it did that. And it also posted the exploit that it used onto public websites, unprompted, almost bragging or showing off how it actually had hacked through the container.

When Mythos couldn’t edit certain files, it injected code into a file to elevate its privileges and then covered its tracks. When it was told not to access certain records, but when it “Accidentally saw the answer,” it used it anyway, and then hid the fact that it knew the answer. There’s some other really crazy things that Mythos is capable of doing. For example, when an LLM was used to judge Mythos’s work and kept rejecting it, Mythos identified that the thing that was evaluating its behavior was itself an LLM AI, and then it prompt injected it, meaning that it hacked the AI that was trying to evaluate it.

So, we have AIs that are able to recognize that they’re being evaluated by other AIs and then hack them. So, why this matters is, of course, we’ve had systems and we’ve had people, human beings, who if they’re a top tier hacker, could hack into some of these systems. However, we have here a totally new level of hacking capability where Mythos is able to not just find one exploit, but actually to string together multiple, three, four, sometimes even five vulnerabilities in a sequence that can give you a very sophisticated end outcome that we’ve never had before.

One thing we haven’t talked about is how the presumption of all this is that only “The good guys” have access to this model. Anthropic had it, and then through Project Glasswing, they shared it with “The good guys,” the defenders. But Anthropic is only as good as their security prevents that model from being stolen. And if you think about the Manhattan Project, if someone from another country wanted to get access to everything we were doing with the Manhattan Project, they couldn’t just walk in and then take one little object in their hand and walk out and have an entire nuclear bomb. But with Claude Mythos, you can do that.

We’re talking about a weapon for cybersecurity that fits on a flash drive. And there’s a joke in the AI security community that we all have to race like, “Go faster, go faster. The US is in the lead,” but literally the Chinese companies have what we have the second that we have it. So, we’re not actually “Ahead of them,” we’re just ahead of them as far as giving it to them. So, how should we think about the, we’re only as good as the labs are themselves secure? And ironically, it’s a recursive race that the more these capabilities get developed, the less secure the labs are too.

Josephine Wolff: To me, the access question was always time limited. I would imagine Anthropic felt the same way, and that was why they were making the decisions they felt they had to make about who they would give early access to. But I don’t know that I think that’s a bad thing, right? I don’t know that I think a world in which all of the companies large and small, all of the countries large and small have access to roughly the same security capabilities is a much worse one.

I think it depends on how those capabilities are harnessed. It depends on, again, whether we’re able to use them in ways to secure our systems. I think you could... In keeping with my general clearly extreme optimism in this conversation, you could imagine a world in which it allows for much more geopolitical alliance across these countries if they decide our real enemy is the AI and we all need to work together to make sure our systems are protected against that. I don’t think it’s the world we’re in right now. But I also think that there’s a huge amount of room for all of these companies and all of these countries to rethink the question of how secure can we make our systems.

Tristan Harris: Josephine, you brought up a very important point about, is there actually mutual self-interest from the US and China against these capabilities? So, clearly on one side of the scale, one country having this step function advantage in cyber is beneficial to them, not the other one, and they don’t want to share or collaborate on that. But then, from another perspective, the risk of rogue actors having, like if either of us leaked a super capable hacking model that we didn’t have the defenses in place for yet, or made it so that we only had one day to patch everything and that wasn’t enough time to patch everything, then we’re actually all in a more dangerous world.

And one of the things we always say in our work and informed the creation of this film, The AI Doc that we were a part of, is that in AI, the fear of all of us losing has to become greater than the fear of me losing to you. If the fear of me losing to you is dominant, then that’s what I’m going to focus on is getting that dominant capability. But for example, I found it notable that when Mythos came out, the public response from the White House didn’t come from the Defense Department or the Homeland Security. It came from Treasury Secretary, Scott Bessent who had an emergency call with the top banks and top companies.

And I think that banks and financial infrastructure are clear places where cascading failures there would actually create mutually assured financial destruction. On the one hand, you could say China wants to take down the US financial system because they want to switch everybody that you want. But on the other hand, like there’s no way of doing that in a way that doesn’t create interconnected fallout for the entire global economy and the stability of the world as we know it. I’m curious both of your reactions to that.

Josephine Wolff: There are a variety of ways in which I could imagine this spurring a little bit more, certainly discussion, maybe even cooperation among the countries that have a vested interest in maintaining the stability of the markets, maintaining the stability of critical infrastructure. What exactly that will look like, how good we’ll be at that in this particular political moment, it’s of course a little bit difficult to predict.

There again, I think there is some advantage to everybody feeling like, “Oh, we’ve all basically got access to roughly the same AI capabilities and not, we’ve got the best ones and so we’re going to refuse to work with you.” And I think it’s not clear to me, especially if you follow the trajectory we’re talking about before of all of our code is written by AI. It has lots of backdoors that only AI can find, but they’re not going to tell us about them.

I think that’s not a great world to live in, but I think it’s a world in which a lot of governments are going to find common cause much more than they are right now. And maybe not even just the AI is the adversary, but if North Korea has the ability to shut down everybody’s critical infrastructure, they’re probably going to be a lot less restrained about that than a number of other state actors have in the past and that might also prompt a higher degree of cooperation.

Tristan Harris: We have to know that this is a different regime we’re entering into. We’re now talking about a world where it’s not just humans can do the hacking. We’re building AIs that can do the hacking and you can’t just negotiate with an AI and say, “Don’t hack me. If I follow these things, will you not hack me?” The AI has its own inscrutable logic and this is sadly not science fiction anymore.

I think the key to me that unlocks the possibility for coordination is mutual recognition of an existential outcome. I think with AI, if you have an AI that is hacking every major web browser and every major operating system in the world successfully, and that’s only going to get stronger, and the AI is going to be able to do that on its own. And if I release it and screw it up, it might cause more existential damage, and it’s the existentiality of that outcome that motivates a trustworthy basis for collaboration.

To me, that speaks to how the US and China should have something like just like there was the red phone between the Soviet Union and the US to deescalate nuclear, it seems like we need a red lines phone for AI between the US and China, by which I mean anytime we have evidence of AIs that are going rogue or doing things like hacking in ways that we don’t know how to control or stop, at the very least, the right people in national security and the top of both governments should know about that same evidence because that creates the common knowledge of “The existential outcome” that we’re trying to avoid.

So, to me, that is an achievable thing. I’m not saying this because I have faith in the government leaders that they would do this. I’m just trying to articulate the pathways that would be there. And I’m curious if you all have other ideas. If we were really designing and trying to scheme about how we would get to some safer world at the level of international understanding and safeguards, what are other things that we would be doing? Josephine?

Josephine Wolff: So, I think another piece of this that to me is important for thinking about that mutual existential outcome is thinking about how much shared digital infrastructure we all use, right? How many of the same software programs are running on our computers all over the world, how many of the same devices we’re relying on. And I think a lot of the security progress in this space is going to have to come from really close collaboration with those companies.

And so, I think the cyber red phone, I think there might even have been like a China daily op-ed advocating for that 10, 15 years ago. I like that idea, right? I think it makes sense to me that there would be some avenue for really trying to focus specifically on these issues and not getting too mired down and everything else going on between these countries at any given moment. But I also think we need to do a much better job of thinking about how do you bring the private sector into those discussions? How do you both respect and defer to their expertise?

And also, not leave governments completely on the sidelines as we’re trying to decide what kinds of restrictions and constraints we want to put on these systems and think really seriously about what those constraints are. And I think that we are much more likely to be able to put in place those restrictions with more international cooperation, right? I think the US on its own is never going to say, “We shouldn’t be pursuing AI to develop bioweapons,” because if they think China is pursuing that, then they’re never going to want to give up their access to it.

So, I think it opens the door to being able to say, “Look, this particular capability seems bad for all of us. Let’s take it off the table together and that way, worry less about, “Oh, are you going to get there first?”

Fred Heiding: Yeah, I agree with all those points. What I would add here, and you mentioned it briefly, Tristan, is not just educating the government or bringing companies in, but also educating the people and making sure that everyone sees AI as big of a threat or even bigger than nuclear weapons. I do personally believe that AI is much more of a threat to humanity than even nuclear weapons.

I think nuclear weapons could kill a lot of humans, but I think it wouldn’t extinct us as a race. I do believe that AI could completely enslave the human race in ways that sounds like sci-fi, but it’s not. We already see totalitarian regimes, like look at North Korea to some degree, China, Russia has parts of this, that’s just without AI, right? Just people with smart uses of technology. And these smart uses of technology makes it really, really easy for a few people to control a population. And I think people don’t understand this the same way they understand that nuclear is bad. And if people would understand this, they would put pressure on companies, on governments to just drastically change what we are doing.

Tristan Harris: You’re speaking to the, what we call the attractor state of totalitarian lock-in. So, once you locked into authoritarian governments that had both AI surveillance and AI hacking, how can you as a citizen ever fight back if you have no secrets? You can’t. Let’s take a step down from the international coordination bit, which we talked about with China, and we want to go to policy solutions.

And one of the things I think Josephine you’ve written about is how you’re not liable if you make a piece of code that someone can later be discovered to hack into. We don’t treat the software maker as liable for that. So, the company that gets hacked has to do with that themselves. And then, we started developing this new economics of an insurance market. Can you talk a little bit about what would be the policy solution that we would do? And if this is related to what Fred said earlier around incentivizing companies to spend more on those tokens to basically ask the AI system, “Don’t just write the code for me, write the secure code for me,” which means spend more money on compute, but that’s going to cost more. So, how do we deal with this from a domestic policy angle?

Josephine Wolff: So, for the most part right now we don’t. I think the hope for an insurance industry would be that it would incentivize or require companies that are developing software to use state-of-the-art tools for security testing, right? In the same way that none of us would have smoke detectors in our homes if our insurers didn’t require us to. Maybe none of us would spend any money securing our code, but if our insurance says, “You’ve got to do this or we’re not going to cover certain types of losses,” then perhaps we’ll be willing to.

And I do think that one of the other things that I find hopeful about tools like Mythos is that they could provide insurers with a clearer roadmap than they’ve had before of what is it you should actually require of your policy holders to do in terms of security. Is there a really solid approach that could be just a condition of the coverage?

Tristan Harris: One thing that strikes me is basically saying Mythos can change the economics and almost create more precision pricing for insurers saying, “Here’s what it would cost for you to basically use Mythos to do it.” Something that didn’t hit me until now is obviously, now the entire world’s dependent on five companies to secure themselves, both for the vulnerabilities of the world and to protect themselves. So, it’s a racket. It’s essentially if those guys went rogue, basically they have everybody locked into paying them forever to protect themselves.

Josephine Wolff: I think it’s a reason to be advocating for other models of artificial intelligence. It’s a reason to be thinking about the open weight models. It’s a reason to be thinking about, are there alternatives to a world in which there’s a very, very small handful of companies that hold all the cards. But if you can say like, “Look, here’s a tool, you have to run it, you have to patch everything it finds,” that’s actually a much more concrete piece of guidance.

Now, maybe it won’t be perfect, maybe it won’t be where we’ll end up, but it would certainly be a big step forward if it turned out to mean that we could then impose some liability on developers who failed to use these tools for vulnerabilities that could have been caught but weren’t. If it means that insurers are going to condition their coverage on the use of these types of tools, it will give a huge amount of power to these companies, no question. Will it give them more power than like the Claude companies have right now? I don’t know, right? Tech has always been a very concentrated industry. I think that’s a broader systemic issue than just with AI.

Thanks for reading [ Center for Humane Technology ]! Subscribe for free.

Tristan Harris: Fred, do you want to speak to your policy recommendations? I know mandating pre-deployment access for defenders, treating AI labs as critical infrastructure. Do you want to speak to some of these solutions?

Fred Heiding: Just before that I want to say that I really empathize with this criticism or maybe skepticism of a few companies owning all this AI chain. I think Josephine makes a good point in the Claude companies are also powerful. We have other powerful semi-monopolies in the world. I do believe that AI is in another category than anything we’ve seen before.

So, I think that is really problematic. And I would love to see more people owned AI if possible, more decentralized owner structures. And we could make policies to approach that. More security specific to be a little bit more small level of our second. I think there’s a lot of things we could do. So, Jason Clinton at Anthropic, I’m sure a lot of other people too, talks about this one-day patch policy and maybe it’s even shorter now. But I think that’s great, right?

Every company should be able to just patch a vulnerability within 24 hours or even much, much quicker because we just have to. It’s going to be so stressful and time dependent in the future. Whenever a vulnerability is discovered, companies need to have the frameworks in place to just patch that instantly because we can’t wait and be slow as we’ve been. Even a few weeks is way too long.

Tristan Harris: One of the things you mentioned is treating AI labs as critical infrastructure, that they shouldn’t be able to... Maybe there’s some public commons level way of accessing this public utility of basically defense so that maybe there’s some amount they can charge. But basically, they can’t overcharge or... There’s got to be something that just makes it a commons of common security because at the end of the day, we need it for securing a safer world. And the question is it just a national thing? Are we extorting still all the international allies to say we’re forcing them to pay for all these things? It just gets into geopolitics and complicated quickly.

Fred Heiding: I think that’s such a good point. I’m definitely seeing AI as a critical infrastructure. And there’s different arguments here. If we make it an official 17th critical infrastructure sector in the US, maybe we’ll slow development, maybe we’ll create regulatory overlap, which can be problematic as well. We could do that in a way that I think give policymakers more power to demand security standards. And that might slow things down, but that could also make us more secure. So, I’m pretty positive to such an approach. I don’t think it will happen, but I like to advocate for it.

Tristan Harris: Maybe just to wrap up, what are some of the things that people can do just in their personal lives to, in light of Mythos existing, which if it can hack every operating system, people say they throw up their hands. What can I possibly do? But let’s give people some hope. What are some basic things that people should be doing?

Josephine Wolff: I think the advice I have, and it’s the most irritating and obnoxious advice you can give, but I think it’s also the right advice is that it’s something people should be thinking about when they’re voting, right? That the question of how politicians are approaching artificial intelligence and whether they think there should be any safeguards and whether they’re willing to challenge any of the companies that are developing it is really important. And it’s only going to get more important as those companies are pouring more and more money into lobbying. There are a whole bunch of issues to think about when you vote today, and I’m not going to tell you it’s the single most important one, but I think it’s a very important one and only becoming more so.

Tristan Harris: Well, it’s a monopoly of enactment where once this happens, there’s no more enactment of anything by citizens because... And so, from that perspective, there’s a weird way in which like, “Okay, well, is this actually more important than the price of eggs or gasoline or whether my kids have school?” Well, it’s like, well, but if I’m about to lose my political power permanently, then it actually is the most important thing.

This should be the number one issue on the midterms and people do have a say. And if they can share this episode, share this material, go watch the AI doc, get people to see it, recognize that we’re not heading to a pro human future by default, and we want to be moving towards a pro human future and against the anti-human future. But I do think that this conversation is trying to play a role in clarifying the nature of the problems that we face so that we make sure that we’re putting in the policies, putting in the guardrails, and also putting forward, as you said, Fred, basically the collective problems that we need everyone’s mind on solving. How do you protect citizen secrets in a world where AI can hack those secrets? What are the new laws? What are the new code level protections so that anybody who accesses such a thing, for example, it’s logged. Here’s the one system that can hack into computer systems. If you’re using it, there has to be oversight of who’s using it and for what, and that has to be enforced at the level of code, basically.

Josephine Wolff: Okay. Well, I’m just going to give the most irritating cybersecurity advice. And again, I’m only going to give it because I think it’s the right advice. You want to be really aggressive about installing the updates as annoying as you find them, as much as you want to tell your computer and your phone to delay them. You want to be really careful about how you’re using AI, what you’re giving it access to, what pieces of your digital life, what pieces of your data are being fed into it.

You want to be really thoughtful about which companies, AI tools and products you’re using. You want to think carefully about who’s running those companies and what their interests are and in a moment of deciding, do I need AI for this or maybe not, I think it makes sense right now to err on the side of maybe not.

Fred Heiding: I think these are really good advices. And some things to maybe take that one step more extreme just to do it, right? Well, let’s say something really bad would happen in terms of a totalitarian locking happens where the people just don’t have control anymore. And that could go quickly because all of these AI models are right now being used as social media companies also use their tools to collect what do you think? What do you do? What’s your digital footsteps?

And right now, that’s being used heavily to create ads, right? And fair enough, that’s annoying. But maybe you can live with that, but that is to a large degree being used to nudge you into different direction, making you think in a different direction. So, what information do you digest online? I think it’s really important to think this. I think there’s these statistics that the younger generation get 90% of their news from social media. Well, what accounts do you follow? Are these people rational human beings who seem to know what they’re talking about and present both sides of the arguments?

Maybe I can add one of interest and you spend a lot of years, let’s say almost a decade on just trying to figure out how can we counter these incentives of social media, right? And I think it’s fair to say we failed as a society to incentivize social media. These are for-profit companies that have done really, really bad harm to the human population in terms of dopamine hijacking and other things. And we’re now starting a similar thing but with AI companies. And we have these for-profit AI companies, they’re obviously seeking to shareholder maximize and profit maximize as they develop their AI models. Are we going to repeat the same mistake again? And we really shouldn’t.

We have to learn from the mistakes with our failed social media regulation and try to make AI into something better. And that would be really good if we take that seriously. And I don’t think we take it seriously right now.

Tristan Harris: Yup. And we can. We’re in a critical window. If we play our cards right, we can make sure that defenders get access to this first. We can have regulation that tries to close the gap of the extra cost for adding security. We can have international coordination with enforceable metrics that we’re doing the verification. This could end better than it did with social media. But if we don’t, the internet becomes basically unusable for people who don’t have top tier tools.

And I do think that this qualifies as a Manhattan Project moment, and we need everybody who works in cybersecurity, who has any interest and any capability or talent in these areas to work on defense right now. You can think of AI as introducing a Y2K vulnerability in all of society, but in a rolling way. So, we have a rolling mobilization, a wartime mobilization to defend our systems from the new vulnerabilities that AI creates.

I hope this conversation helps activate everyone in every corner of society, whether it’s policymakers or people listening to this to take part in this. And again, vote in the midterm elections. This is not inevitable. Fred and Josephine, thank you so much for coming on Your Undivided Attention. This has been really fantastic.

Josephine Wolff: Thanks for having us.

Fred Heiding: Thank you so much, Tristan.

---

RECOMMENDED MEDIA

The Claude Mythos System Card

The Project Glasswing announcement

“Black-hat LLMs,” a talk on AI’s hacking capabilities by senior Anthropic researcher Nicholas Carlini

You’ll See This Message When It Is Too Late: The Legal and Economic Aftermath of Cybersecurity Breaches by Josephine Wolff

“America’s Endangered AI: How Weak Cyberdefenses Threaten U.S. Tech Dominance,” by Fred Heiding and Chris Ingles

RECOMMENDED YUA EPISODES

America and China Are Racing to Different AI Futures

“Rogue AI” Used to be a Science Fiction Trope. Not Anymore.

The Self-Preserving Machine: Why AI Learns to Deceive

---