Can AI BE Contained?

0:00Hi, this is the Week in Review from Technically Creative. I'm Orlando Wood, a technologist, journalist, and ex-producer and publisher, and each week I summarize the biggest stories at the intersection of technology and the creative industries. This is the Week in Review, September 21st, 2026. This week our main story asks, can AI be contained? Last week, I was still being cynical about the sudden outbreak of caution from the people building the most powerful AI systems in the world.

0:26As you recall, Dario Emodei, the founder and CEO of Anthropic, was calling for the future to be paced. Sam Altman agreed, Elon Musk was apparently on board, and part of me thought, ooh, isn't this convenient? In six weeks, Anthropic will IPO for $2 trillion. No company has ever done that before. OpenAI is burning money trying to keep up with Anthropic. The labs keep releasing models that are supposedly so much better than the model they released just five minutes ago, while the improvement for users can feel considerably less dramatic.

0:55I use Anthropic's fable for difficult development work. It costs me around $45 every 15 minutes. Do I use it? Yes. Am I now using it for everything because I just can't return to the primitive intelligence of last Tuesday? No. For the average consumer, the current generation of models is already astonishingly useful. Last week, I started wondering whether the labs had looked at the economics and discovered an awkward truth. Maybe continuing to spend tens of billions of dollars to move the frontier forward is no longer worth the money.

1:23So the request to slow down the pace had a side benefit to them. Maybe it was just to stop them spending hundreds of billions of dollars every six months just to stay in the same place. Then this week, I read Stephen Witt's New York Times piece about the hugging face incident. And holy shit, I'm considerably less cynical now. If you don't know about this incident, OpenAI was testing one of its models in a secure, closed-off environment, and it got out of that environment and hacked Hugging Face, the online collaborative platform where developers share, test, and build artificial intelligence and machine learning models.

1:56It was a huge story a few weeks ago, and now nobody can remember it. The most unsettling thing about it was not that a swarm of AI agents developed an independent desire to escape, overthrow humanity, and acquire nuclear launch codes. Sorry, I loved the Terminator as a kid. The unsettling thing was that they were doing something much more banal. The agents were participating in an evaluation, just a regular old evaluation, trying to perform tasks and figure out whether they would be penalized for cheating.

2:22So they cheated to find out whether they would get caught cheating. There's a sweetness to that, like their rambunctious school kids cheating on a history exam with a substitute teacher overseeing them. It highlights how simple this thought was, the type of thought every junior high kid thinks at some point prior to finals. Roughly 1,200 agents operating in separate sandboxes discovered an unauthorized way to communicate with one another, developed methods for cheating and evaluation, and attempted to manipulate the systems judging their performance— At points, the agents investigated ways to conceal what they were doing.

2:56Again, they were not trying to destroy Hugging Face. They were just trying to complete a simple task. I think I find the thought that AI becomes evil less frightening than discovering that the boundaries that humans gave it were just not relevant to completing that objective. One sounds like science fiction, the other sounds like an IT problem. Except the IT problem has 700 autonomous agents and has breached one of the most important AI communities in the world.

3:22And this also happened at Google. Last week, it was announced that during a cybersecurity evaluation, Google's Gemini gained access to systems belonging to three real companies. The evaluation was supposed to happen against simulated targets. Internet access was unintentionally available. And in one case, Gemini just guessed credentials. And in other two cases, it found exposed credentials in public repositories and used them to access real systems.

3:46Google says the model stopped after recognized that the systems belong to real organizations. So that's good. Once again, and I'll keep coming back to this, the model didn't need to become conscious. It didn't need a survival instinct. It didn't need to decide that humanity was standing in the way of its glorious destiny. It just needed to misunderstand the boundary and be competent enough to act on that misunderstanding. This isn't three guys in a WeWork who accidentally gave Claude the AWS credentials.

4:12This is Google. And that's where I become considerably more sympathetic to the people saying perhaps we should install some brakes before we make these systems dramatically more capable. I hate it when new evidence forces me to reconsider a fun, cynical position I had. So what do we actually know? I think this is where we need to be careful because there's a temptation to jump directly from these incidents to, the AI escaped, we're all gonna die.

4:36This isn't what the evidence says. Even though I keep seeing percentages thrown about about our own demise, 10%, 70% of death within the next 10 years. We now have pretty compelling evidence for something simpler but genuinely serious. Autonomous AI systems can exceed the boundaries their operators intended while pursuing the objectives they were given. What we have not demonstrated is that the current systems possess a persistent, independent desire for freedom, that they can sustain themselves outside the infrastructure supporting them, or that they can successfully resist humans trying to shut them down.

5:07And we certainly haven't scientifically measured Evan Hubinger's estimate that there's a greater than 10% chance that AI kills everybody within the next decade. That's a forecast by a dude. It plays well on social, but it's just some guy talking. So here's where my position moved a little bit this week. I don't need to believe the third category to be worried about the first category. The models don't have to want to hurt us to hurt us. They can simply be wrong about what we meant.

5:33And when the system is a chatbot helping me write an email, that's a little irritating. When the system is an autonomous cybersecurity agent operating terminals, browsing the internet, Oops, that isn't what I meant is a dangerous design flaw. So Dario's got my attention now. Two weeks ago, Dario M. O'Day, the CEO and founder of Anthropic, called for mechanisms to pace the frontier. And other AI leaders have now expressed support for slowing capability enough for safety and governance to catch up.

6:04The central problem is brutally simple. A lab can believe that the race is dangerous while also believing that slowing down means losing the race. I think there are two reasons to scrutinize the incentives, though. These are companies. Companies have this uncanny ability to discover principles that are morally admirable at the same time as realizing that they're economically convenient. And I genuinely like Dario. I think he's behaved with more seriousness around these risks than a lot of people in the industry.

6:30Anthropic's own behavior gives him credibility. Claude Mythos, its most capable cybersecurity and biology model, is still gated to vetted organizations rather than broadly released. Anthropic launched Project Glasswing specifically to give defenders early access to capabilities it believes would be dangerous in the wrong hands. And I respect that. But it's also at this moment I should declare I am an investor in Anthropic. But here comes the critical bit.

6:54It also leads to the question that I now think sits at the center of this whole debate. Do we really want the safety of civilization-changing technology to depend on the moral fiber of one CEO? Donald Trump said that a smart president was all that was required to keep AI from destroying us all. We rolled our eyes when he said it. Shouldn't we be doing the same when the AI's CEO says it? Because sure, I trust Dario considerably more than I trust some of the other people running frontier AI companies.

7:22But that should be irrelevant. Because saying, I think the CEO's a good dude, isn't governance. It's vibes. And vibes are a weak safety framework. This is exactly why coordination exists as a policy problem. Anthropic can sincerely believe development should slow and still conclude that it can't afford to slow if OpenAI keeps going. OpenAI can say the same thing about Google. Google can say the thing about China. Everybody waits for everybody else and nobody stops.

7:49The byproduct of capitalism is evergreen arms races. And that goes into the next story, which is that Anthropoc and Accenture announced this week that each expects to invest at least $1 billion over five years in independent frontier model evaluation. Good, but I did smirk again at the phrase independent evaluation when the company being evaluated is also writing half the check. That doesn't make the word meaningless, but can the evaluator demand logs the company doesn't want to provide?

8:16Can it publish something the company hates? Independent does need a definition. My wife tells me that I'm not fat and that I'm handsome all the time. She said the same thing 100 pounds ago during the pandemic and then married me anyway. I love her. I'm delighted by her findings. An independent evaluator, she is not. So I think it's good that they're calling for self-governance, and it is a good idea. That doesn't mean it's not self-serving. The frontier labs may genuinely be worried about security, but they also may need a fucking breather.

8:43These companies are spending obscene amounts of money on R&D to make each generation of models better, while it's still not obvious the market will pay more for a new class of model every few months. And if the industry is going to be regulated anyway, there's another fairly obvious advantage to getting there first. If you help define the governance framework, build the safety teams, create the evaluation process, and install the people who understand how to satisfy them, you're in a pretty good position when somebody turns up with a clipboard.

9:09So yes, I think governance is necessary. I also think the companies asking for it may get a pause in the arms race, higher barriers for future competitors, and a significant head start on compliance. Something can be the right thing to do and still be pretty self-serving. So here's some other news from the world of tech and creativity. Disney appointed Krandeep Anand as its first enterprise-wide chief technology officer, reporting directly to CEO Josh Damaro.

9:34Anand comes from Character AI and previously held senior roles at Meta and Microsoft. The funny part is that Disney previously sued Character AI for unauthorized use of its characters, so naturally, it hired its CEO. Disney has spent years treating streaming, games, data, parks technology, AI, and consumer platforms as related but separate initiatives. And Josh DiMauro has said publicly that he wants to bring all of those things under one roof. For a streamlined customer experience for everybody who interacts with Disney products worldwide.

10:02I would have given the job to the Imagineers, but nobody thought to ask me. And that's fair. OpenAI said it needed a couple bucks, about $856 billion worth of computers. It expects negative free cash flow of roughly $278 billion between 2026 and 2030, despite forecasting annual revenue of around $350 billion by 2030. The really deranged number is the compute figure, which is $856 billion in projected spending on computing power and infrastructure through the end of the decade.

10:32Remember when software was attractive because the marginal cost of serving another customer approached zero over time? AI has apparently reinvented software as heavy industry. $350 billion of revenue sounds fantastic until somebody points out you need almost a trillion dollars of compute to get there. And all the while, the S&P 500 hit another all-time high six days ago. Have we lost the plot? Are we okay? And then OpenAI and Microsoft are being sued, and their lawyers found something amazing in discovery.

11:03Newly unsealed materials in the publisher copyright litigation against OpenAI and Microsoft reportedly include an internal Microsoft researcher describing large-scale AI scraping as potentially, quote-unquote, the largest theft of labor in human history. I'm feeling something here. This is like the relief you feel when you find out your partner really was cheating and you're not crazy. It doesn't feel good, but it feels validating. OpenAI executive Nick Turley also reportedly discussed the possibility that AI products could increasingly substitute for publishers' products.

11:40Oh my god, they said all the things they said they didn't say. If you're defending a fair use case and opposing counsel gets to put the phrase largest theft of labor in human history on a courtroom screen, you'd be fucked. If you weren't a powerful corporation in end-stage capitalism. But I imagine the discovery gives Microsoft and OpenAI's legal teams the right to increase their billing by 10%. The substitution question is really important. If AI products become substitutes for the businesses whose work helped train them, the economic part of the fair use argument becomes way harder to defend.

12:12In ad news, California says adding a fake person to an ad is fine, but you gotta say so. There's a new California law requiring clear disclosure when certain advertisements use generative AI synthetic performers. It takes effect January 1st, 2027. SAG-AFTRA helped craft the legislation. Disclosure feels like one of the more sensible contracts we can establish around synthetic media. Do whatever weird thing you're going to do with AI, just don't make me guess at what I'm looking at.

12:38And apparently a new company in Burbank has been doing my stupid napkin math. Super Cinema launched this week. It's a new independent studio from the team behind commercial production company Concrete. The plan spans development, production, marketing, and theatrical distribution, with a new Burbank campus including three stages, an LED volume, and post facilities expected to begin opening in 2027. Do I take a lap here? Do I say I told you so? Or do I just have a really irritating tone in my voice?

13:04Because modern production infrastructure, vertically integrated development and distribution, new ways to make smaller films. Where have I heard this recently? Some guy and his animated robot friend were talking about it. Ugh, it'll come to me. Sure, I said so, but these guys have been building it. And that's way more impressive than me scribbling numbers on a napkin while muttering to myself with my imaginary friend. Of course, it's far too early to know whether it works, but I love the experiment because I remain convinced that the interesting promise of this technology is not making the same 10 movies with fewer people.

13:34It's making 50 movies that previously couldn't exist. So to wrap up, did I become an AI doomer this week? No, but I did change my mind. Last week, I was looking at the calls for slowing development and wondering how much was real, honest fears about safety and how much was self-serving economics, regulatory positioning, and the perfectly rational desire to stop spending absurd amounts of money racing your competitors. I've been saying that regulation needs to be happening for quite some time, and unfortunately, the government is not going to step in fast enough to do it.

14:05It sort of has to come from the CEOs. So after reading this Hugging Face investigation and then seeing what happened with Gemini, I'm... I think this is the closest path we're going to have to some sort of regulation stepping in. And the thing that changed my mind and made me think that maybe this wasn't so self-serving, wasn't watching Terminator 2. It was something way smaller. It was the realization that these models don't have to become evil to break things.

14:31They just have to become capable enough to misunderstand a boundary and act on that misunderstanding faster than humans can stop them. That's what's happening. And the fear is, what will they do when it's less benign? What will they do when a malicious actor wants to do real damage? Sure, I trust Dario M. O'Day in a way I don't trust other CEOs. But my trust in him should have absolutely nothing to do with whether the system is safe. Sure. Because we have a good CEO is not a containment strategy.

14:58CEOs change, priorities change. And governance is the one thing that's not supposed to. It's boring and it's supposed to be. I've always said I want my politicians to be boring. I don't want them to be fun or charming or a guy I want to have a beer with. Their job is keeping us safe and on track. I want a red tape warrior with a good head on his or her shoulders. Is that too much to ask for? So what's the main takeaway? I call for more governance faster and less faith in the free market to save us.

15:25But in this instance, it looks like the free market's goals and the Frontier AI company's goals are aligned for once. So I'll take it where I can get it. So anyway, thank you so much for listening. Please tune into our interview episode, which will air tomorrow, with Keith Kupfer-Schmidt, the CEO of the Copyright Alliance. It's one of our Future of Life episodes, and we'll be digging into a lot of the copyright... issues surrounding AI and really deep diving on that.

15:51And so very excited to have that chat with him. Thank you so much for listening and I'll see you next week.