Where Are The Rational Conversations About AI Extinction Scenarios - Week in Tech

0:13Speaker 1:Oh, wait, we've got the queen herself here.

0:15Speaker 2:Natasha, we were worried you'd been killed by AI, but you're here.

0:21Speaker 1:I said the EAs took you out.

0:24Speaker 3:Or did they? Is this a Natasha deepfake?

0:35Speaker 2:Welcome to Tech Stuff. I'm Asif Al-Hashim, and this is The Week in Tech, where I'm joined by the world's most plugged-in reporters... to break down what's really happening in tech right now. Today, we're joined by Reid Albergotti, tech editor for Semaphore, Taylor Lorenz of UserMag, and Natasha Tiku, tech reporter at The Washington Post.

0:53Speaker 3:Welcome all. Thanks for having me.

0:55Speaker 1:Hi. Thanks for having us.

0:57Speaker 2:So we're not dead yet.

0:59Speaker 3:Oh, you know, how would I prove that to you? I have no idea.

1:05Speaker 2:This has been I mean, I felt I felt like the in the last week, obviously, everyone's been on the Oh, wow, AI is going to kill us all train since the Jacob Coxon viral post. And I feel like I'm my response. This is so cynical, I can barely bring myself to engage. But I feel like that's a bad place to be. And I mean, I have to ask you guys, what's it what's it been like for you kind of being in this wave of interest in this beat?

1:30Speaker 3:I find it to be a very uncomfortable situation where like, as a reporter, I'm actually like, on the other side of the warnings about, you know, the dangers of some new technology, right? I'm like, the reporters are supposed to be the ones warning everyone. And I'm like, no, guys, like, this is not exactly what you think. I'm like, actually, now I have anxiety because I need to call my mom back. She was she had a she was sending me panicked text messages yesterday about this. So That's where I'm at.

2:03Speaker 1:I love to hop in the discourse. So I think this is such a fun thing to talk about. I am worried about the effect that it's having on people. I feel like over the past week, I've been hearing from a lot more people that are like, hey, what do you know about this stuff? Are we all going to die? What's happening? And that makes me nervous because I don't like when the public is scared of technology. I think it doesn't bode well.

2:28Speaker 3:I would also just say, Oz, that there are a lot of real concerns from this technology, many of which we've talked about on this show. And I honestly do think that when you start talking about human extinction and this sort of hypothetical future that doesn't exist yet, you're taking a lot of attention away from stuff that really does need to be addressed. Fully agree.

2:49Speaker 2:I mean, that's partly why I find I don't want to engage in the conversation. But why did your mom call you? What is it that Coxon said that broke through that Stuart Russell and Jeffrey Hinton and Dario Amadei himself, like, I genuinely don't fully understand what was different about this and why.

3:07Speaker 3:It went viral. Well, I think a lot of it's timing, you know, you, you had had a lot of, uh, you know, stuff coming out about the hugging face hack. I think also there's an understanding that these companies are about to go public now, like there's real money on the table. And then this guy's resigning and giving up a lot of money, right? Like he's a young guy. Like this isn't like a Jeffrey Hinton who's probably made more money than he's ever going to spend. And it's like now, you know, retiring anyway. I mean, he's like, this guy's like saying, I'm giving up my financial future here to like raise the alarm bells. I think that does resonate with people. Do you agree, Taylor?

3:45Speaker 1:I was going to also say timing as well. I think that like it's been building for a while. I also think that heat perfectly executed his media strategy. And I think this to the point that people thought it was like a PSYOP.

4:00Speaker 3:How?

4:01Speaker 2:Explain how.

4:02Speaker 1:I think that we are very familiar with kind of this process of like sort of what the whistleblower goes through now. And, you know, Jacob had spoken to the Wall Street Journal. He had lined up an appearance on Fox News. He had spoken to other people in the AI safety communities. There were people ready to like boost his message very early on. a lot of these AI safety orgs immediately started reaching out to reporters to amplify his message and say like, hey, did you see this?

4:28Speaker 1:So I feel like all of that also, it was almost, to me, it reminded me of like a tech launch. Like I've been talking to a lot of people in startup world lately and it's like, they'll be like, okay, you know, we're launching at 11 a.m. At 11.15, you quote tweet it. Then I send it to the group chat and these organizations will boost it. And so that's kind of how it felt like this. And so I think that that sort of strategic rollout helped it break through as well.

4:52Speaker 4:He did tell me he sent it to his group chat. So he said he had a group chat of about 10 people, including Sneha from Encode, which is like a very strategic and successful AI safety nonprofit. He's also, he said he was neighbors with Daniel Coccatello, the, you know, the guy behind AI 2027, that his group house was next to the driveway of Daniel's group house. And he'd previously put out a Slack message in the Anthropic Slack. So they knew it was coming as well. I mean, it's still hard to get 170 million views on X these days.

5:30Speaker 2:But what was your what was your conversation with him, Natasha?

5:33Speaker 4:It was very brief. It was 11 minutes. I did not realize that, or I would not have spent so much time asking him about how, how did he get to go so viral? But yeah, I mean, he was, he didn't have any particular insights to share from his time at Anthropic. I was like, you know, was there something that you saw in particular? The only thing that he mentioned was company Slack, where they're talking about the end game in a really overt way and the discomfort of the idea of these executives or a company talking about the end of the world in this way on the company Slack. And when I asked him, do you belong to any particular group? He said, no, even though he had a lot of friends in AI safety. And he said, actually, he thought his message went more viral because The way he talks is not AI safety coded, which I think is probably true.

6:27Speaker 1:What does that mean? Like, I.

6:31Speaker 4:Mean, it wasn't immediately discernible. I will say he had I asked him, did you like scrub your presence from the Internet? Because like even on Nexus, you couldn't find anything. I'm sure Reid is also a great sleuth and and. found stuff, but like it was surprisingly little. So normally if an AI safety person comes out, you know what scholarships they got, you know, you know, you, you found them on less wrong. He wasn't out in that way. But then during an AMA, he kind of said he was a rationalist.

6:56Speaker 1:I feel like he's absolutely part of that community. But as Natasha said, he wasn't immediately visible as part of that community. Um, there's a lot of efforts as well now. And there's memes about this online of people being like, well, I'm not completely EA. I'm EA adjacent. I'm not part of the real whatever. You know, it's like all of them want to claim that they're not part of this cult. And fundamentally, they are.

7:19Speaker 4:I'll just say, like, I have even had people who are on the EA website as an example of an EA say, like, I'm not really EA. So it's I can't recall anybody wanting to claim it. Oh, there's some other stuff that he said. I did also ask him about, you know, you're getting calls from senators, whatever. What are your policy goals? And he said he didn't have any specific ones, but he thought, you know, maybe this would be really good, good kind of fodder going into Trump's meetings with Xi. And I asked him, like, oh, did you time it that way? And he said, no, I just learned that these talks were happening.

7:55Speaker 1:So ridiculous. Yeah.

7:56Speaker 3:There's a big hope that somehow the U.S. and China are going to get together. That's been an EA sort of AI safety hope for a long time.

8:05Speaker 2:I want to unpack the timing a bit more, though, because the Coxon viral post happened. The whole world became interested in existential AI risk. And then the CEOs, Altman, Musk, and Amodei, chose the immediate aftermath of that to do their call for a slowdown of the frontier.

8:25Speaker 1:Were they.

8:26Speaker 2:consciously adding fuel to the fire? Or were they just going to do it anyway? And they didn't reflect on how much the timing of their announcement right after the Coxon thing will kind of send this to the stratosphere? Like, what was what was it that they did this week? And how is it different from last week's news cycle driven by Coxon, Natasha?

8:44Speaker 4:I mean, I think Dario Amadei has said multiple times that he is was hoping for some kind of coordination between you know, for the labs to all agree or, you know, multiple countries to agree to a slowdown. So he was motivated. I do know that Sam had been having these conversations, but I think it's also just like in a way like a recruiting retention tactic. They know that their employees care a lot about this and all of a sudden it doesn't become, you know, anti-capitalist. It's not, um, you know, it might not be seen in a cynical way.

9:17Speaker 4:It looks like there's an opening, you know, and tech companies are nothing if not like total sheeps in a way. And so I think that they found cover. And it's true that a lot of the people who work on capabilities research are also within this community. And, you know, you can see a lot of stuff that OpenAI has done recently to make sure that it's still able to recruit and retain people from this community. So I think that, you know, they know that they've been hated. They have their own reasons why it would behoove them. Like Reid said, they can't control their models. They don't know what they're doing right now. Their risk section in the S-1, if they file to go public, is going to be... the best sci-fi we've read all year. But can I just say one thing that I thought was so funny? Dario specifically, so in his essay, he put this like three-step plan. One, we have embedded evaluators. Two, You know, the government gives us an antitrust exemption waiver so that we can coordinate between the labs and then three China and the U.S. collaborate, whatever.

10:24Speaker 4:The fact that he did not anticipate how badly the embedded evaluators response was going to be because he put in this company meter, the same one that did the hugging face report that has like such obvious evidence. EA connections and that everyone in the industry has been stewing about the fact that Meter was the one that did this report. The fact that he didn't even anticipate this was going to blow up. I'm just like, wait, why are we listening to him about like global strategy, about geopolitics, about anything if he couldn't even see that leaving that off would have saved him and his company a world of hurt?

11:01Speaker 3:Yeah.

11:02Speaker 2:How did how did Trump react to the call from Amadei and others?

11:07Speaker 3:Do we? Yeah. I mean, so ridiculous. Like his tweet.

11:12Speaker 1:Trump's posts.

11:13Speaker 4:Yeah.

11:13Speaker 1:Trump's posts have like been really, it's, it's so hard to tell if they're parody or not. I feel like sometimes when you read them, but Trump is sort of going all in on AI. Like, I mean, he, he also like, called Jensen Wong on stage at the All In Summit to say that like, you know, he's got the right approach to AI. He's basically like, he's like the last hyperscaler maxer left. Like, it's kind of funny. I mean, people were saying like when Trump dies, there really is no other pro-AI like politician to that extent.

11:49Speaker 1:I mean, he's been talking about how like AI is the future. People that don't want AI to move forward are like poor and stupid. And he's fully pro-AI.

12:00Speaker 3:He said all we need is all we need for AI safety is a genius president or something.

12:05Speaker 4:Yeah.

12:06Speaker 3:Okay.

12:07Speaker 2:Why is he such a mixer? Is it because of David Saxon is around him or because he believes in that the keeping the stock market going throughout his presidency will be good? Or what's his like, what informs his extreme passion for this?

12:20Speaker 1:I think he genuinely like believes this stuff. I think his thinking is probably shaped by people like Jensen and David Sachs. But I think also like he's just got it in his head that it's good. I mean, this is a guy that uses AI, you know, he's posting AI slop or his intern is or whatever. Like, I think he recognizes that economic power. It is ironically, like, people have been saying online, like it's like the one thing that he's kind of able to like cut through and be quote unquote good on in the sense that like, I think he is fighting against a lot of like media pressure and PR spin, but I don't think he has a coherent regulatory strategy. I think he just sort of has this vibe that AI is good, good for the economy, good for his presidency. And so he's going to be pro AI.

13:05Speaker 3:Well, he gets calls from people and they convince him and then he tweets, right? There's just, this isn't like a very, I don't think this is like a very well thought out thing. And there's a bit of a rift in the White House between there are some real kind of like AI safety oriented people now that I think have some influence. But he probably got a call from someone, maybe it was Zuckerberg, maybe it was Jensen over the weekend and decided to tweet that. That's my guess.

13:31Speaker 2:What about Congress? Is Congress united on the like regulate AI kind of movement now? Are there any kind of, unleash the brakes figures in the Senate or the House?

13:44Speaker 3:There are tons of bills. There are all these bills, right? But none of them are going to pass. Like it's, it's the state that the states that matter, right? The states will actually pass laws and none of it's actually going to, even if we, even if they pass all the laws, let's say Congress could get together and even pass a law. It's not going to prevent the existential risk from AI development, right? This is, We make all these assumptions about technology that hasn't been invented yet that turn out to be wrong. And I think we will do the same thing with AI.

14:15Speaker 2:But I want to ask you guys, what are the actual legitimate existential concerns? I know it's a bit of an eye roll, but on the other hand, what are the real scenarios driving existential risk that people are laying out that are actually somewhat plausible in the 20-year horizon?

14:32Speaker 3:I think the thing that The AI safety people, if you read their books and all the detailed things they've written on less wrong, et cetera, the AI always has to find a way to get around the off switch. These AI labs, they need everybody to be on their giant data centers, right? Which is what all the powerful AI models need right now to run is these giant data centers, which are also giant off switches. Like the AI development that could potentially threaten humanity, like isn't even happening yet. like, we're just talking about making large language models bigger and bigger. But the real question is, like, could we one day have AI models that can continuously learn and can operate, you know, can have human intelligence with 20 watts of energy? Right? I think these are like, these are the types of things that actually we need to pay attention to. And they're sort of coming down the pike, maybe 1015 years from now. So that's sort of what I, no one, no one has this sort of practical conversation when you, when you have, when they have Coxon on, you know, on, on Anderson Cooper or whatever these shows, no one actually gets into the details.

15:38Speaker 3:And I think that's what, that's partly why everybody's sort of irrationally just freaked out right now is they're not actually thinking through like, What actually could destroy humanity, right?

15:46Speaker 4:Why would you ask a 27-year-old who's worked at two AI companies how something might unfold on a global scale? They are not the experts, so they're not going to be able to speak in detail about that.

15:59Speaker 3:Then why are we freaking out because of what they're saying?

16:01Speaker 4:I'm not freaking out.

16:03Speaker 3:Well, we as a society.

16:05Speaker 4:No, no, I know.

16:05Speaker 3:My mom.

16:06Speaker 2:Yeah. When we come back, how might pacing the frontier actually help AI companies financially?

16:20Speaker 3:Stay with us.

16:34Speaker 2:What's going on with the IPOs? I mean, people are talking about this being the year of $ 3 trillion IPOs, but it looks like OpenAI and Anthropic might not happen this year.

16:43Speaker 3:Or probably won't happen this year. Yeah, because I mean, ultimately, like, they're not right now, they haven't really figured out their finances. I think that's pretty clear. I mean, you get you see a lot of numbers get leaked in the press about how much revenue these companies have, and they're profitable, etc. But like, none of this is, you know, audited gap financials, right? This is all just these are all leaks to the press to kind of, you know, manipulate investors in some way or another. So When we actually see the numbers, and by the way, these companies filed confidentially, right? Like we haven't seen the official numbers yet. It's not going to look great. Right now, they're spending billions upon billions of dollars on these massive training runs, right? That are making these models better, right? They solve millennium math problems, right?

17:33Speaker 3:And that's great. But most businesses, most of these customers of these companies don't need to solve the Millennium Prize math problems, right? They need very basic, practical, and reliable models to automate parts of their workflow. But because of this race, this competition with these other companies, they have to keep training these, spend billions of dollars in training these massive models, which also, by the way, require GPUs to be crammed into these hundreds of thousands of GPUs to be crammed into one building because the training runs have to more or less happen in one place, which is also not practical from an energy perspective, like getting enough power or just getting the data centers passed by some local community. So all of this stuff points to, we should pace the frontier. And you can call it what you want, but I mean, it's good for their business.

18:28Speaker 2:Natasha, how much of the pacing the frontier conversation has been driven by what you've called AI's tendency to hack?

18:35Speaker 4:I mean, I think that the open AI hugging face hack, which we've talked about a lot, definitely played a big role in how the public received this and then how the companies reacted. It's just this really concrete example of how things could go wrong in very. In ways that have nothing to do with existential risk or with, you know, some of the far out ideas around goal seeking or power hoarding or even sentience. You know, this is just like, it's also coming out in drips and drabs. I mean, even just yesterday, there were more, like I saw like a university from Tennessee was asking why OpenAI's agents were inside their private system. So I think that these real world examples, even though we haven't had, you know, um, actually anybody go after OpenAI for these hacks yet, like, you know, not the FBI, not a private company yet. I think that they play a big role in that. And I think we saw this, too, in some of the accounts that we're hearing from the lab employees, like not necessarily Jacob, who hadn't been at Anthropic that long, but Evan Hubinger, who is a very well-known AI safety person and has been at Anthropic for

19:52Speaker 4:a long time. You know, his kind of quote tweet that really helped push and legitimize Jacob, I think. He has been talking for years about the problem that I had written about recently, which is like, you know, they hack because we taught them to hack. Like, this is a problem people identified a decade ago, the way that we develop these models incentivizes them to be focused on this one reward, and you can cheat your way to that reward. I mean, there's been some really interesting reports, too, just coming from the folks who build some of the environments where these agents learn to move around and do the kinds of things humans need them to do and how sloppy those environments are, how poorly quality tested they are.

20:40Speaker 4:And so, you know, it's essentially the humans are hacking, so then the AI agents are hacking. I mean, I think... even when people talk about far off problems, when you sit down and you ask like, but they're doing this because this is what we told them to do, right? We told them to operate as a swarm. We told them to like write down notes as your memory. You know, we told them to win that exploit bench, the test that the OpenAI agents were on. I think like, yeah, it's just coming to the fore that, hey, we might've known about this problem for a decade, but we have not at all.

21:16Speaker 1:Figured it out.

21:17Speaker 3:I do think this hacking stuff is an issue, don't you? I mean, I think it was an issue before AI, like we weren't doing enough to, you know, shore up our systems. And it's a bigger issue now.

21:27Speaker 4:You don't need the most advanced models to cause a ton of chaos. You just need to have, you know, kind of give them permission or not observe them closely enough to where they can do this. Like this does not need to be mythos level agents to break into some universities.

21:45Speaker 3:Yeah.

21:46Speaker 2:So is there going to be a big investment in cybersecurity now? Because you don't need any more. Regardless of how fast the frontier grows, to your point, the cat's out of the bag. And even open source models probably are strong enough to do a lot of this type of work. So do you see a huge boom in cyber defense investment coming?

22:05Speaker 3:It's not enough. I mean, I was talking with one of these big defense contractors the other day. And I'm like, well, clearly, there's going to have to be some huge government effort to go and shore up all of our you know, critical infrastructure and make sure it's not running on Windows XP. And they're like, well, that would just be way too expensive. And I'm just like, well, guys, like this is going to be a huge disaster then. I mean, best case scenario is just it costs the economy a lot of money because there's just more and more ransomware attacks. Worst case is they, you know, shut down the electrical electricity grid like that. That would be horrible. I mean, that's like basically, you know, worse than a nuclear bomb in terms of, you know, how many people would die.

22:44Speaker 1:But also cyber attacks are not new. I think, honestly, we're not listening enough to cybersecurity experts in all of this conversation. I think they have a lot of nuanced thoughts on this. I think that they have said many times, I mean, I agree with Reid that the government is probably not secure as it should be, but also there are extremely sophisticated hackers trying to take down our infrastructure every single day and the government does mitigate those threats.

23:16Speaker 3:Are they really trying to take down the infrastructure every day? I mean, certainly state actors have like broken into critical infrastructure, but like, I don't think they actually want to take it down. Like I, but I, but I think like if you give that power to anyone, if it's not just state actors and now it's literally anyone can do it. I, I do think there's some risk.

23:37Speaker 1:Anyone cannot do it. Anyone cannot do it. Like, we're talking about like massive, like, I just, this idea that like, I just, I, I am so skeptical all of it. And I think that none of the, no cybersecurity people are the ones out there raising the alarm, going on CNN, making these cases for like doomsday that like, we all need to panic right now. What they've said is that we, we want to make sure that we're prepared. Here are the steps we could take to be prepared. I think like, you're not going to see an overnight takedown of the entire electrical grid. We'll probably see these smaller attacks. Like once those happen, I think they will raise alarms to people in power. Like, I just think that like, we're getting so ahead of ourselves and there's endless amounts of hypotheticals that you can come up with.

24:22Speaker 3:Oh, I mean, I sort of, I get, I get what you're saying. Like I, and I, I'm not, I mean, it makes a lot of sense. I just think like, and I'm not saying like panic freak out, like we, the world is about to end, but like, You're right. There are hackers who are hacking into systems every day and every big company gets hacked. And taking down hospitals. Taking down hospitals. There's ransomware attacks that happen every day. Every company is being hacked every day, for sure. We're not talking about a step change, but we are talking about a percentage increase. And what percentage increase is that?

24:56Speaker 3:I don't know. Is it 20%? Like there's an economic cost to that. And there's also just like, I think, unintended consequences. And I think it would make sense, like even without AI, it would make sense to do this. But I think it makes more sense now to have a real concerted national effort to improve cybersecurity. That's what I'm saying.

25:17Speaker 1:Sure. I agree with that.

25:20Speaker 2:So you have the kind of people leaving AI labs and warning of existential risk in one camp. Then you have the people leading the frontier labs calling to pace the frontier. Then you have Trump saying, no, don't pace the frontier. Then you have politicians trying to message test to figure out if AI will actually bring people to vote in the midterms and beyond. There's a lot going on. What happens next? Where is this all going? This seems really in the eye of a storm right now.

25:51Speaker 3:Probably something else. Sorry, go ahead, Taylor.

25:56Speaker 1:I'm very concerned about where this is going because I think there's a lot of fear. And to just bring it back to what Reid was saying right when we started getting a call from his mom, this has really broken containment, no pun intended, to normie audiences. And I think in times of great technological change, which we're definitely in, you see a lot of fear from the public and that can often have very negative consequences. I don't think we get thoughtful, useful policy when the public is scared.

26:25Speaker 1:I don't think that a lot of the public will take advantage of the upsides of this new technology when they're so scared of it. And so I just, I think there's a lot of fear. There's a lot of money in feeding people's fear and denialism. And so I, yeah, so I'm just very concerned about where that will go. I don't think it's going to go well. I think it's going to result in really bad laws and just a lot of like hurt and pain and needless kind of angst from from people, especially young people. You know, a lot of them are feeling like they're growing up in this world that's falling apart, climate change, et cetera.

27:01Speaker 1:Now the, you know, AI is going to kill us all. Like it's, it's hard for like a, I was talking to a 16 year old yesterday and it's like, I get why they feel so nihilistic and upset. Yeah.

27:12Speaker 4:I was going to say class warfare or class warfare is where we're headed, I think. and also like further radicalization of billionaires who are extremely resentful about this populist backlash and this confusion and the fear mongering. And, you know, and they're already anticipating that this will be a tax, a huge tax burden for them. And, you know, that people are be coming out with pitchforks to like take what's theirs. And, you know, as we've seen for the past 10 years, like, brainwarmed billionaires are like can be quite influential in American politics. So I think we'll see all that. But like one thing I wanted, I do agree with both Reid and Taylor about this being a terrible, a terrible perspective to start writing policy. And what Reid was saying about some of the stuff not coming to effect for 10 years is I think if we are legislating about a future that, you know, is completely unknown, and I think we saw this with social media, even some of the stuff I wrote about early, that ended up not being the main problem with social media, right?

28:24Speaker 4:And say we just listen to the cybersecurity experts and do things that help the way that the models are now, that put in appropriate guardrails or think about the problems that we're seeing in front of us. We could be preventing that future, but if we legislate towards that future, all the stuff we are seeing is just going to fester, I think.

28:45Speaker 3:Yeah, it's totally unpredictable what will happen if and when this stuff is invented in 10, 20, 30 years. We have no idea. So creating legislation now is sure to have major side effects, unintended consequences that you know, might create other issues, right? We've seen this before. I mean, I think a great example, like my colleague Liz Hoffman wrote about the parallels to the financial crisis. And a lot of people don't remember, but like there was, and I think it was 2004, there was like this FBI agent who was warning about, you know, the mortgage crisis and, you know, all this fraud in the mortgage industry and how it was going to create this huge problem in the financial markets, et cetera. And he just got completely ignored because everybody was, Completely focused on terrorism at that point, right?

29:34Speaker 3:The Patriot Act and all this stuff. And we just didn't police any of this stuff because of that. And then we got the financial crisis, which then of course, led to all of these other issues that we're still dealing with today in our political system. And, you know, I think we have to, we should look back and think about that. Think about like, okay, guys, like, Let's try to find our center here and our principles that we really care about and stick to those rather than go off on hysterical tangents because some 27-year-old said something.

30:07Speaker 2:Is there anything that should be happening on the product side of AI development that's not happening or that changed this conversation? Or is what you're all saying, this is actually the product side has been kind of overemphasized versus.

30:23Speaker 1:I think we really need to invest in open source AI from America right now. Most of the open weight models are from China. I think that as we seek to regulate AI, I really worry about us not discussing open source software in light of these regulations. There's a really aggressive attack to, I mean, California just passed a law, Colorado. There are attacks on open source software development outside of AI. And I think that going forward, it would be great if as a country, we recognize the need for open source software, recognize the need for open weight models, recognize the need to have transparency and accessibility with any sort of new technology. And we make sure that any regulation that we write fosters that type of tech ecosystem. I worry that the way that a lot of regulation and a lot of the way we talk about the future of AI, we're only talking about anthropic and open AI. And, you know, AI is a lot This is a new type of technology. It's not only those companies that we're writing these laws for.

31:28Speaker 3:And Taylor, you brought up liability earlier. I think liability will shape the product side. These companies will be worried that they're going to get sued if these agents start going out and causing economic damage by hacking, etc. But then you mentioned open source companies. well, who's liable if I'm running an open source model on my home computer, right?

31:51Speaker 1:This is exactly why we should put the liability, in my opinion, on the person doing harm. You know, like the person, you know, if you use any sort of new technology to commit crimes, then you should be, then you should be like held liable for those crimes. So like, if I use, you know, an open source model to hack a bank, then I hacked a bank. The problem is me hacking a bank, right?

32:22Speaker 3:Well, here's the thing. I use codecs a lot. Like, so, and I, and I probably, I have it on a separate computer and I try to experiment with it and do all these, all these things, which I think are great. Um, hypothetically, if I'm trying to like, I don't know, build a, web app for my family. And for some reason, the agent decides to go hack into JP Morgan and, you know, shuts down the financial system in the process. Like, I don't really think I should be liable. Like I think opening, I should be, but cause I never asked it to go hack into JP Morgan.

32:54Speaker 1:But here's the thing. I think that if you put that liability on the user, the companies will inherently put in safeguards to prevent that because read it, they would have no users.

33:06Speaker 3:Well, They have put in safeguards. Right. And nobody's hacking banks. But that's the second part of my point. I'm not that worried that my codex is going to go hack into JP Morgan while it's building my family web app or whatever, because there are all these safeguards in place, right? And the hacks that happened with the Hugging Face hack, et cetera, those were in the lab where they don't have these same safeguards, right? And so these companies are already dealing with these liability issues. The product changes are already being made. And I don't know how big of a problem, I think Natasha kind of made a similar point earlier, but I don't know how big of a problem all these hacks are really because the safeguards basically work.

33:54Speaker 3:And I don't think OpenAI or Anthropic are going to release some harness that is just going to go off and hack into the banking system inadvertently. I think you're going to have to do that yourself. And if you do do that yourself, there's going to be a record of that. There's going to be a record of you trying to, I don't know, do whatever it takes, jailbreak this harness to go off and do what it's not supposed to do. Or if I'm running an open source model on my home computer and I go hack into something, there's going to be a record of that too. And you're going to be caught. So the product changes, I think, I think the liability does sort of affect the product and it is doing its job to some extent. Right.

34:35Speaker 1:But what we don't want, which they're trying to do in Congress now, is put complete liability on anybody that makes software. And at a time when software is being democratized and people want to build their own tools for things, they shouldn't be subject to these intense regulatory systems that raise that bar. I think that if you do crimes, you should be held accountable for those crimes. And if you didn't intend to do the crime and blah, blah, blah, blah, blah, you can argue that in court. But as you said, Reid, it's quite hard to not... These things are not going off the rails and you are not going to accidentally hack a bank when you're trying to do something small with these models. And I think that the people that are making those arguments are not technically literate. honestly. And so, you know, I just, I worry.

35:20Speaker 3:We need to differentiate between software and products, right? Like Codex on my computer is a product, but an open source model that I download from Hugging Face is not a product. It's just a piece of software. And I agree. I don't think anybody should be held liable. I think the open source ecosystem needs to be allowed to be free. Can't really stop it anyway, right? Because this stuff can be made anywhere in the world where they could just do it outside of the.

35:45Speaker 1:But they can criminalize it.

35:47Speaker 3:They, they can. And I think it would be, I agree that it would be a really bad idea to do that. I think, but I do think the products, you know, there should be product liability. Like, I don't think, I don't think these companies should, we shouldn't have anything like a section 230 type of like exemption.

36:02Speaker 1:Well, but when you put product liability on a, on a system that is primarily, that is, you know, people's primary way of accessing information, that concerns me a lot because we've seen the way that these product liability cases and precedents are being used, and they're not being used to protect people from cyber attacks. They're being used to censor access to information. So saying that, like, hey, Chachi PT is faulty because it gave a teen girl information about an abortion in Texas.

36:31Speaker 2:Natasha, final thought to you before we wrap?

36:34Speaker 4:I think, like, one good part of this conversation swamp of debate that we've been mired in for the past week is the discussion about, I mean, they have been, the companies would like it to focus on evaluators and embedded evaluators. I think we should like get back to auditing, but I'm glad that there is a focus on, hey, who is even deciding what we should be safety testing? What are the technical chops of the institutions that are doing the testing? I think bringing in auditors and people from different disciplines to look more closely at this, like that's just a no brainer. And right now it's been kind of a LARP up until now, I think. And it would be great to not just bring in cybersecurity people, but I think that's sort of what Taylor is saying too with open source is it's like, if you start building some of these product safety things through open source, where there's like a huge incentive to do it, it's going to be better for everyone. Like, yes, let's grow up and start bringing in some outside experts.

37:43Speaker 2:That's all we have time for today. Thank you all so much for joining us.

37:46Speaker 1:Thanks for having us.

37:47Speaker 3:Great to be here.

38:01Speaker 2:For Tech Stuff, I'm Osvaldo Schim. This episode was produced by Eliza Dennis. It was executive produced by me and Julia Nutter for Kaleidoscope and Katrina Norvell for iHeart Podcasts. Jack Inslee mixed this episode and Kyle Murdoch wrote our theme song. A special thank you to Taylor Lorenz, Natasha Tiku and Reid Albigotti. Please check out all the work they put out into the world. We're lucky to call them friends of the pod.