678: Entropy Ain't Easy

0:11Chris:Hello, friends, and welcome back to your weekly Linux talk show. My name is Chris.

0:16Wes:My name is Wes.

0:17Brent:And my name is Brent.

0:18Chris:Hello, gentlemen. Coming up on the show this week, Greg KH has been busy. Seven Linux kernels landed on Saturday. Almost all of them were in LTS. We'll break down what changed and what releases just might fit your home lab best. And then why generating truly random numbers is so dang hard and how it's gone so bad in the past. We'll talk about that and we'll round it out with some boosts, some picks, and a lot more. So before we get there, let's say time appropriate greetings to our virtual lug. Hello, Mumble Room.

0:50Chris:Hello, guys.

0:51Mumble:Hello, Brent.

0:52Chris:And hello up there in the quiet listening and everybody watching live over at jblav.tv, making it a Tuesday on a Sunday. And good morning to our friends at Defined Networking. Check out Defined.net slash unplugged and sign up for Managed Nebula 100 hosts for free, no credit card required. Nebula is one of my favorite technologies, and Manage Nebula makes it so straightforward, and now they give you a lighthouse to get going real quick. Picture a Mesh VPN completely and totally under your control with a design that is intelligent and low resources from the beginning.

1:28Chris:Something designed to connect some of the most important and sensitive data around the world is now available to you and, through the magic of cryptography, is very straightforward to set up. You get strong encryption, direct device-to-vice connections, and you can self-host Lighthouse Nodes, giving you complete control over the network that you depend on. That's huge. So go get started. 100 hosts for free. No credit card required at Defined.net slash unplugged.

1:55Chris:Go check it out. We love it. Defined.net slash unplugged. You know, we just have a really quick housekeeping this week. We wanted to say thank you. We meant to say thank you last week when we were remote at Brent's place. But it dawned on us afterwards, once again, a big thank you to everybody who helped us get those headsets. It's almost been, what, two years now?

2:17Brent:Those things have been just solid. I don't think they've never ever caused us a technical issue we've had to solve. Everything else has, but never those headsets.

2:25Chris:It's crazy. It's crazy what a reduction in complexity and gear that was for us, too.

2:31Wes:Really? Yeah. I mean, at that point, it's them and a mixer of some kind that can handle enough inputs.

2:36Chris:Yeah.

2:37Wes:And then a computer.

2:37Chris:And without them, it's big microphones and stands and separate headphones and wires for all of that to work. And it's nuts. It's nuts. So thank you so much. We just wanted to take a moment. And it's like every time we use them, we're like, oh, so, so freaking grateful, the members and the listeners, too. People contributed from all around.

2:57Brent:I want to thank you two boys And the Mumble Room for doing a nice little send off Of my beloved studio That was nice That thing has done quite a few episodes So I want to say thanks for your One last hurrah in there It was really great.

3:15Chris:Seven kernels on a Saturday. Greg KH shipped a bunch of stable kernels in one window. I'm hoping he's taking a summer vacation after this. Six of them were LTS point releases and some big numbers in here, boys. Check this out. 6.12.100. 6.1.180. Get ready for this. It's still kicking. Linux 5.10.262.

3:42Wes:Yeah, those are basically point release numbers, right? so those get bumped up as Greg releases new ones so you can kind of tell how long he's been doing those.

3:50Chris:Yeah.

3:51Brent:I'm glad he's keeping track because we're not.

3:54Chris:Linus also tagged Linux 7.2 RC5 over the weekend, so he called it quite the massive RC5. This really kind of brings it up to a total of seven. So we got these six stable point releases, and then the new 7.2, which kills AppleTalk, is reaching RC5. And it has quite a bit of changes in it because there's been a lot of AI-assisted patching that's been hitting the kernel recently. It's a lot. What do you say, Wes? You want to break some of it down for us?

4:21Wes:Yeah, I guess just take it to handle on what kernels we're talking about here. Well, sort of before the big push, Greg released 7.1.5. 7.2 is not out, right? So we're doing the 7.1 thing.

4:30Chris:Yes, thank you.

4:31Wes:And then, of course, as you said, Linus released 7.2 RC5. We're expecting 7.2 to come out maybe sometimes around August 16th, somewhere in there. Obviously, depending on how many RCs Linus deems necessary. And then, so that was July 26, and then on July 30, yeah, six LTS point releases. So we got, we'll go from the newest and then kind of get older. We got 618.41.

4:58Chris:Wow.

4:58Wes:That's the newest, quote unquote, active LTS, right? 618.

5:02Chris:Okay.

5:02Wes:That has an end of life coming up in December 2028. Then we had, let's see here, 612.100, Debian 13 Trixie and RHEL 10 base. That EOL is also December 2028.

5:16Chris:That's, I mean, 2028's not that far away, but pretty good. So these are going to, this will be the kernel that ends up in people's RHEL 10. So this will be pretty widely used kernel 612-100.

5:24Wes:And it's the first time we start seeing the involvement of the civil infrastructure platforms SuperLTS that's going to try and keep this thing alive until December 2035.

5:38Chris:I'm sorry. What did you say?

5:41Wes:Yeah, 2035.

5:43Chris:Wow, yeah, yeah. That's incredible.

5:45Wes:So basically, it's a Linux Foundation project, the civil infrastructure platform, with a bunch of industry folks, right? Like Siemens, Toshiba, Hitachi, Mux, so people that.

5:53Chris:Build like- I hope Google's in there with Android.

5:54Wes:Right? Long-term embedded industrial systems. And so it kind of exists to extend kernel.org, which often does it for like roughly two years. But they're trying to offer stuff more like 10 plus, 20, you know?

6:08Chris:So if you want to run 612-100, you can say that you're running a super LTS kernel.

6:13Wes:Yeah, that's right. Okay, we also got 66147. That has an end of life of 2027. That's getting pretty close, December as well. 61180, that's a Debian 12 bookworm base. That AOL is December 2027 as well.

6:29Chris:Also coming up.

6:30Wes:And would you believe it? We are still pushing updates to the 5 series kernel. Yes, we're on 7 now. that's like two giant numbers ago uh, so this is 5 15 213 that's the ubuntu 2204 lts base that's coming up soon though end of life december 2026 as far as obviously ubuntu does their own work but yeah in terms of kernel.org.

6:52Chris:Right right then they're on their own after that.

6:54Wes:And then the last but the least i don't know that's up to you uh 5 10 262 that's the debian 11 era kernel.org end of life December 2026 coming right up but this one is also under that super long term support to January 2031.

7:11Chris:When you see that, when you see they get the super long-term support, I think the obvious giveaway here is like these have ended up in some sort of, widely deployed corporate system, some image that's on a bunch of consumer devices or control devices, something like that.

7:25Wes:Yeah, and things shipped that like aren't easy to update or maybe are certified. And so you need, if you change the kernel, you got to do a whole new certification. You might not be able to do updates in place.

7:34Chris:But a lot of these drop off in 2027. A couple of drop off in December of this year. Two of them drop off. So like Greg's list, I don't know if he's responsible for the super long term, but like Greg's list does shorten a little bit in a year or two.

7:50Wes:And that's part of sort of the two-year thing, right? It's like the kernel.org team has to maintain stable kernels that like, you know, actual people pull from. And they work on whatever the latest kernel is all the time. It's just, you know, you have to handle who's going to actually support this, which is where it is nice to see the companies involved in wanting to run ancient kernels stepping up to sort of take on some of that ownership.

8:10Chris:So, Brent, I think last time we were looking at some of the patch sets that were hitting 7.2, there was a lot of stuff in video, a lot of video drivers. What are we seeing that got your attention in 7.2 RC5? The new freshness.

8:23Brent:There is new freshness. There's AMD GPU compute stuff in there. that's a 42x regressions though on some of the 7.0 point releases so stable diffusion xl via comfy ui went from a nine second to uh 388 seconds.

8:42Chris:That's a pretty significant regression.

8:45Brent:Okay so.

8:46Chris:All right so maybe 7.2 don't don't use it on amg compute systems just yet.

8:50Brent:I mean some of those are fixed upstream in 7.0.13 but not every single distro has picked them up yet so So take your chances or do some research, I would say.

9:00Chris:I also saw a fair amount of pent-up networking work. Wes had a stat here that over a third of the patches that landed were networking alone. And again, I think, Brent, that's more of like the AI-assisted code stuff only sitting in the networking branch. It just, to me, seems like... We're going to need, like, a spring cleaning next year or something.

9:20Brent:How much are you hoping to clear out? As much as we cleared out?

9:22Chris:Well, if we're willing to get rid of AppleTalk, then where is the line, boys?

9:26Wes:I think you're right. We're probably going to see more of that as basically every subsystem in the kernel gets more and more pressure. The stuff that is no longer load-bearing is just the desire to cut it is going to increase.

9:38Chris:But to your point about, like, that massive 42x regression compared to Linux 7.0, So that stuff stings, but most people that are using that stuff in a production workload where they're using the actual AMD compute infrastructure, I doubt they're going to install 7.2. And AMD does seem to be actively contributing upstream, so I imagine they're going to be working on that.

9:59Wes:I think part of the story here is kind of like what and why do you choose a particular kernel? How do you go about doing that, right? So it's like some of these things are, yeah, you hit it if you are upgrading, but do you need to upgrade for hardware enablement? Do you need some sort of new feature? Are you really looking for a potential performance improvement?

10:16Chris:There's another wrinkle to it as well, as sometimes the drivers or system support do get backported to these older releases. So you might see that they started contributing drivers at the 7.0 series, but sometimes, like the NVIDIA stuff... IOMMU stuff, that stuff gets backported. And so the older kernels can sometimes end up with that. So I think you could think about it in tiers here, right? If you're looking through an LTS lens and you're just looking at long-term stable kernels, you want to set up a box and forget about it for as long as possible.

10:49Chris:I mean, you still need to update, but you won't have to redo your kernel. Then that's, I think, right off the bat, that's Linux 6.18. It is the current, quote-unquote, bleeding-edge LTS. It has some of the newer AMD, NVIDIA stuff backported. And it will last until 2028. That gets you a pretty long runway.

11:06Wes:Yeah, I suppose that comes down to like what are you supporting and how many updates are you willing to do? How long can you go without changing kernels?

11:13Chris:Yeah. Or like you said, is it just doing stupid file serving and maybe DNS? And it's like something you run on a Raspberry Pi or it's a low-end networking box. Then you don't need 6.18. Something like 6.12 will be around until possibly 2035. So talk about a box for networking infrastructure you could just set and forget and just do quarterly updates on. And you're not going to have to worry about a new kernel. So 6.12 could potentially be around until 2035. That's the one for those machines.

11:44Wes:It's funny it feels old, right? 6.12 kind of sounds a little old. But, I mean, it was a kernel we really talked about a lot on the show. It was a big one. Preempt RT stuff. I mean, there's just a bunch of stuff. And I think it was pretty widely deployed, which is why it's getting that long-term support.

11:59Chris:Right. Preempt RT. Right. Also, the note in here that it's considered one of the most battle-tested kernels of all of them. And that's their opinion. So, okay. Let's go with that. So, 6.12 seems like a real winner if you don't need super modern hardware and you're cool with the setup and forget it kernel. Then after that, like, I think it sort of just drops off and it's, okay, well, you're just going to go with whatever distro you want. This is really if you're even making an active decision, right?

12:24Chris:Right, because Debian's going to manage their kernel release. Ubuntu will manage theirs. There's going to be stuff that they're just going to take care of for you, and you may end up on a different kernel at some point when you just upgrade the distro. But if you are actively making a choice, and I'd like to know how many of you really are actively making a choice, which kernel do you actually pick? Like, I'd like to know how many of you are actually doing that. I mean, we do.

12:45Wes:Yeah, we definitely do. I mean, we're running this end kernel. And then back in the day when this was a KDE Neon box here in the studio, you were doing the mainline tool to just kind of get the freshest kernels.

12:54Chris:And on FakeNez, we're doing the LTS kernel. We're doing one of the LTS kernels just because we hardly ever update that box and we don't want it to break when we do ZFS updates. And so we did go to the LTS gear for the rolling arch-based server. And so I wonder how many people out there are, A, actively choosing their kernel. I bet it's a minority. And I wonder, B, which kernels they're choosing. So boostin, boost.jupiterbroadcasting.com and let us know.

13:17Wes:Yeah, I like that. It turns out you can have a rolling distro with a stable kernel or a stable distro with a rolling kernel.

13:23Chris:I like it. I mean, on our desktop boxes, we just go full rolling with everything. But on the server, it's not that crazy, I think, to have an LTS kernel and a rolling user land. I mean, we've been proving it for how many years on that fake NAS box, abusing it, doing rando updates.

13:40Brent:Isn't it just lots of luck, though? I don't know.

13:42Wes:I think the cottonwood is structural. It helps at this point.

13:46Chris:It's really about, well, for us, right, it was down to we wanted rock solid ZFS support. And sometimes ZFS doesn't keep up with the current kernel. And if you load the current kernel, you can sometimes not have a loadable ZFS. And so for us, it just made a lot of sense. I think if we were doing Bcache FS or ButterFS on that machine.

14:04Wes:Probably wouldn't be a big deal.

14:05Chris:It probably wouldn't matter. But then if we were trying to use an NVIDIA driver or an AMD driver.

14:09Wes:Then it could be.

14:10Chris:It could be. So again, I'd like to know how you pick and what you pick. Let us know. But let's talk about 7.2 because I feel like we've been building to this for weeks, especially since we learned about the death of AppleTalk coming at 7.2. So what stands out to you, Wes?

14:23Wes:Yeah, there's a lot of exciting stuff. A few highlights for today, though. The cache-aware task scheduler.

14:30Chris:Yeah, buddy.

14:31Wes:The scheduler now knows which cores share a cache. It groups tasks that share data onto the same cache domain, so you stop paying cache miss penalties across cores. Now, we should be clear, this really only counts if you are on a system like Epic or Xeon that has multiple cache domains. You're not going to see it on a single cache domain device. But if you have that, this is basically like a free performance when you don't do anything except upgrade your kernel and things get faster.

14:56Chris:See all those Xeon CPUs, boys?

14:58Brent:Sweet.

14:59Chris:I'm going to go dust off that old Mac Pro upstairs that's like 15 years old that's got Xeons in it. And I'm going to... Actually, I do have an active Xeon system in production that I think about it.

15:08Wes:Oh. So I love it.

15:09Chris:I love it.

15:09Wes:Well, time to update.

15:10Chris:Mm-hmm.

15:11Wes:Or it will be in August.

15:12Chris:Yeah.

15:14Wes:Okay. Not the only work on the scheduler, though.

15:16Chris:Okay.

15:16Wes:And scheduler stuff. I mean, not only is it one of the trickiest areas, I think, of the whole kernel, but it can tend to really, you know, it can have regressions pretty easily. It can be very noticeable, or it can have a lot of performance impact. So another change is a fairer GPU scheduler. The kernel's DRM scheduler used to default to high priority for GPU jobs. Now it defaults to fair. So GPU time gets distributed more evenly when you've got multiple things competing for it.

15:42Wes:And if you're running GPU pass-through to a VM or something like that, while also running all your fancy desktop compositing as you do, this should help. if you've just got like one gpu you know churning away at like a ml task or just rendering one thing specifically you probably won't notice but again these are hardware specific but that even makes them you know a little more challenging to do yeah, So it's impressive we're seeing the changes.

16:05Chris:Okay, you got anything for a guy that's just got a regular old computer that doesn't have like Xeons or, you know, multiple Goopoos?

16:14Wes:Well, we do have memory reclaiming. How about that?

16:17Chris:All right. I like that.

16:19Wes:The kernel's page reclaim got a lot smarter in 7.2. MGLRU improvements are showing 30 to 100% higher throughput on things like MongoDB, which is kind of just a test workload.

16:30Chris:All right.

16:31Wes:And your page allocator isn't stuck and always handing out 2MB anymore. It can hand out 16K, 32K, 64K pages, smaller bits. It means less overhead on every memory lookup. And that means instead of having to allocate more stuff than you actually need, programs can request and get more reasonable amounts.

16:50Chris:I can't believe that wasn't a thing already.

16:52Wes:Yeah. There's definitely some of that in here.

16:55Chris:You're telling me the huge page allocator in Linux has been stuck at 2 megabytes forever?

17:00Wes:Yeah.

17:02Chris:Wow. Okay. That's a big one. That's a big one. All right. You got me.

17:06Wes:And it should mean container startup and teardown in particular should be noticeably faster.

17:11Chris:All right. All right. You got me on board, Wes.

17:13Wes:And maybe you don't want to be using it, but if you do...

17:16Chris:What?

17:17Wes:Your pal swap also got faster.

17:19Chris:Oh, you know, you never know. Sometimes you got to swap.

17:21Wes:Yeah. Back in the old days, the bad old days, swap used to route different memory types through different code paths. That's gone, cleaned up, unified. Less bookkeeping, less bloat. and at the end of the day that just makes it faster.

17:34Chris:I like to put my swap on a ram disc so i'm glad to see that performance has been improved.

17:39Wes:Now it's not all good news necessarily there are some regressions here um that, you know that's being worked on though but rfs in particular is in a bit of active surgery um large folio supports on by default now okay and they've improved things there was a previous direct io regression that got a fix a 59 throughput fix no less.

18:01Chris:Jeez, okay.

18:02Wes:But, as a result of that, you know, so you had to do that to get the new fix. You had to get the performance improvement. We got large folio support that speeds things up. Right. But, it did also introduce regression itself. In particular, if you're running KVM guess with RAM backed by a ButterFS file, so it's like a particular setup, it can cause a deadlock.

18:22Chris:Hmm.

18:23Wes:But they're working on the patch. I don't know yet if that's going to land in 7.2 ultimately. I hope so. But, we haven't gotten there yet.

18:30Chris:And then they will backport it to Linux 6.12.

18:35Wes:And perhaps in a way where if this is only in the RC, then perhaps then it's a way that you just never have a possibility of getting it at all.

18:43Brent:I'm like working through all my VMs and like, is this going to hit me? And I think actually one of them does.

18:49Chris:Yeah, I could see that getting me. What you need is a super fast external storage in a different file system. Maybe some new stack for the universal serial bus could enable this, Wes? Maybe a new iteration?

19:03Wes:Oh, maybe. Yeah, we are seeing USB 4 streaming support coming. There's not hardware really, I think, out there a ton of it yet that you could actually use. So this is the case where the kernel sort of leads the way with, people doing upstream development, right? Where before the hardware exists, it's in the kernel. So then when the hardware does exist, you just plug it in and the kernel supports it.

19:21Brent:That's so great.

19:23Wes:Also stuff that isn't super used yet and isn't a little more experimental, but fun to watch because we've been paying attention is sub scheduler support in the sketch X system that's, that enables you to write your own scheduler.

19:34Chris:What's the difference between a scheduled task and a sub scheduled task, Wes?

19:38Wes:Well, like you can, you have a finer grain on, you know, because, and you can change, you can basically have tasks that have their own schedulers, right? So you can have, instead of one scheduler that has to know about how to schedule everything for every type of task, it can then have its own subscheduler that can schedule just that type of task.

19:56Chris:That's awesome.

19:56Wes:And then you can optimize it more, like maybe specifically you're trying to run a database server, right? And the database does all kinds of stuff that other kinds of applications don't do. And so you might need a scheduler that's aware, like, oh, give the database priority on this kind of thing.

20:08Chris:Dang it. You're going to make me run 7.2.

20:10Wes:Well, that was the idea.

20:11Chris:Dang it.

20:12Wes:I don't know why. I just like the number 7.2. It's got a good round shape in my brain.

20:16Chris:Yeah, it does.

20:17Wes:It's got a nice color, so we should all be running it.

20:20Chris:There has been. I mean, I was teasing earlier, But there has been a bit of cleanup in the 7.2 kernel release as well.

20:26Brent:Quite a bit of cleanup. You mentioned the networking stack.

20:30Chris:Mm-hmm.

20:30Brent:And you mentioned a third of the patch of RC5 is networking alone. So mostly driver-side stuff. And Linus himself said, nothing in there strikes me as particularly strange or scary. So don't worry, guys.

20:47Chris:Oh, man.

20:49Brent:Also, there's a, hey, you remember the Sega Dreamcast? Yeah.

20:52Chris:Did you guys play that? Yeah. That was like a peak late nineties, really.

20:56Brent:Oh, I played the heck out of that. Yeah. That was 1998.

20:58Chris:Wow.

20:59Brent:Well, the Dreamcast is still getting driver fixes. So there's a mouse driver null pointer dereference.

21:05Wes:It had drivers at all?

21:07Chris:And a mouse? Okay.

21:08Brent:I mean, we should investigate maybe. So there was a null pointer dereference since 2017 and that's been fixed.

21:16Chris:That's got to be an LLM fix.

21:18Wes:For sure.

21:18Chris:That's got to be.

21:21Wes:I mean, it's kind of interesting almost that this happened. And are people on the list being like, why do we still support the Dreamcast?

21:27Brent:You can't have AppleTalk.

21:28Wes:But the Dreamcast is totally fine.

21:30Chris:I like to think there's a couple of load-bearing Dreamcasts out there running the LAMP stack. That's what I like.

21:36Wes:Look, it's been hosting my personal website since 1998.

21:40Chris:Or it's like some kernel developer's email server from like, yeah, 99 or something like that. Like in 2000, he gave up on it and just turned into a Linux mail server. And they're still maintaining it.

21:51Brent:Well, they might be running off disks, so GDROMs, those disks, mount properly now in the Gamecast, just in case you're having that problem. But don't forget, i486 is also gone from that.

22:06Chris:Wow, right.

22:08Brent:Yeah.

22:08Wes:Hey, 14,000 lines removed, though.

22:11Brent:It's true. That's quite a few. That's 80 whole files. And Linus says here, zero real reason for anybody to waste one second of development effort on this. It launched in 1989, supported for 37 years in total, which is 18 years after Intel stopped making them. 6.12 LTS still supports it, though, till 2036. So don't worry, Chris.

22:38Chris:That 6.12 is the MVP kernel, guys.

22:43Wes:Yeah, you want your Apple talk, you want your 486.

22:48Chris:Honestly, if we're still going in 2035, 2036, hand to sky, I hope, we should load up 612 and get a 486 box that we can still get booting going.

23:00Brent:Yeah.

23:00Chris:That would be pretty great. And the Dreamcast, right? All right. I mean, this is pretty solid, right? We've got a lot of kernel releases here. I mean, a lot of times you're not even going to have to think about this. We like to think about it. We like to pick. But I think a lot of us don't actually have to think about it that much. We hope that maybe you're the kind of person that does give it some thought. Seven kernels. Two of them are finally aging out at the end of this year.

23:22Chris:And one that does kind of break your GPU performance, so be careful. But the clear MVPs for home labs, if you need modern graphics hardware, is 618. and if you just want to set it and forget it forever, is 6.12. I mean, 6.12 is getting the fixes that we talked about for the performance regressions. It's getting support until 2035, and it's going to support 486. Probably also has AppleTalk in there. I mean, probably has AppleTalk, right?

23:47Wes:It does.

23:48Chris:Oh, my goodness. What an MVP of a kernel.

23:50Brent:Are we calling it the legacy kernel?

23:54Chris:I think we should call it the MVP super kernel. I mean, that's what they call it. It's the MVP super.

23:58Brent:Let's do it.

23:59Chris:We got all kinds of links in the show notes. if you would like to nerd out and read more. And there is some good reading in there. And you know where to find those show notes over at linuxunplugged.com.

24:17Brent:Well, we got a really nice email into the show this week, or more accurately, into my personal email. And here's the title. Serious Security Advisory for Your Cold Card. And I thought this was a scam at first because I've been getting a lot of really, quite sophisticated spam into my email. And it's been very entertaining to watch. But nope, this one's real. And it got me a little nervous about what the heck is happening out there. Do you guys have any idea why I got this email?

24:48Chris:This is really a tragic story. As of this weekend, almost 2,000 addresses have been drained of nearly $90 million worth of Bitcoin, affecting cold card wallets, initially the cold card Mark III devices, and then later expanded to include certain Mark IV, Mark V, and cold card Q firmware versions, basically everything but the most current emergency firmware that was released over the weekend. And it comes down to an issue in how the cold card generated random numbers.

25:19Chris:You know, we're going to get into this. Randomness has been a real challenge and a challenge for Linux, too, that has gone wrong in catastrophe kind of ways. So in this case, the cold card had a firmware bug that sent the seed generation down the wrong path. Instead of using the hardware that was built in to derive a seed phrase or a key that had low entropy, it was falling back to like a Python script. In fact, a micro Python script that roughly generated an entropy of 40 bits on the Mark III and 72 bits on the later devices.

25:54Wes:Yeah, it's essentially the micro Python runtime provides sort of like a random, a basic random generated. It'd be fine if you were using it for, like, you know, an offset in a cron job or something like that. But it is not meant to be cryptographically secure.

26:06Chris:Yeah, it doesn't generate complex enough cryptography that it can't be figured out. So the hardware was designed to generate strong randomness. The hardware was present. It was working, but it just wasn't being used, which is really a tragedy. And so because people can now mathematically derive the... the seeds, they are able to essentially restore wallets on remote systems and drain the cold cards without ever having your cold card connected to the internet or your computer active.

26:35Wes:Yeah. So essentially without the hardware part that was supposed to be like the secure random entropy, the fallback random number generator was seeded from the chip's UID, which is like a unique ID on the chip that has a limited possible amount of numbers, and a boot time timer state. And neither of those are truly secret. And a lot of them, it turns out to be kind of closer to a predictable range than something random. And then so if you can kind of constrain the space of things that the initial secrets generated from, then you can explore that space and the rest of it's all just deterministic.

27:07Wes:And then you can generate a whole bunch of private keys at once and then just go check them.

27:11Chris:Yeah. And before the computing machine became easily accessible to us, analysis confirmed in the past that the hardware worked. That's what people really focused on in their audits is that the hardware did what the hardware said it did. But they didn't test if the actual path to activate it was working until LLM-assisted security reviews came along and discovered it.

27:30Wes:Yeah, because the code to use the hardware is in the binary. Like it's built in, it's available. It's just never called.

27:38Chris:Yeah, yeah. So the code exists, it works, but they failed to verify it was actually being used. And so as a result, attackers have been able to access people's funds. And that's a tragedy because people thought they were doing the best thing they could for themselves. They were keeping them offline and all of that. And it got us thinking about some of the previous times in Linux where random number generation has gone horribly wrong. It's not necessarily a hardware story itself. It's really about the difficulty, of generating true randomness on deterministic systems, right?

28:12Chris:Because if you give a computer the same input, you're going to get the same output every single time. So the limitation then is how do you generate something that is truly uniquely random on a deterministic system? And it has to be something that's sound and secure and hopefully as future-proof as possible. So when something like this happens, well, you don't get rugged. And that is more difficult than it seems. And it's something that Linux has been working on since the very early days, like since 1994.

28:39Wes:Yeah. It's essentially cryptographic security is essentially a bet, right? It's like not a law of physics. It's not a guarantee. You are just trying to do everything you can to have a giant pool of randomness to pull from. and then keep that hidden and secret and intact across the entire chain. And there really is a chain. Like it's not just, you know, give me number and done. It takes a lot of setup. So kind of how it works today, the very first step is some of the most important and that's getting quote unquote unpredictable inputs. And this is stuff that comes from a large space that's really hard to guess.

29:12Wes:So Linux, the kernel is continuously collecting small little timing variations and environmental noise and pieces of the system that are sensitive to things like that. So when does a hardware interrupt fire? Which one fired?

29:23Chris:It can even be like noise from the disk, user input, just things that the system generates that are unique.

29:30Wes:Totally. CPU timing, bootloader provided seeds you can use if you want to try, to bootstrap randomness in a VM. You can also use hardware things either like included by your CPU manufacturer or an add-on device or something like that. And part of the idea here is that no, you're not trying to make sure to have every single source needing to be perfect on its own, you can try to mix them together so that even if some don't have as much randomness in there as you think, you're still getting a good final randomness on the other side.

30:00Wes:So after you're able to find a bunch of different sources, you mix that into your entropy pool. And you need to be careful because the most dangerous time is when you boot something up fresh. Because there hasn't been a lot of time for environmental noise to get into the system. There hasn't been that you haven't really done anything yet. It's all kind of starting code cold from the very non-random initial firmware and like, you know, first code that runs on the system.

30:23Wes:And so a lot of work over the years has been around like, do we need to block if there's not enough randomness or not? What does it mean to have enough randomness or not?

30:30Chris:And Intel has tried to solve it with their RAND kind of built-in stuff that's trying to generate some noise. But, of course, that thing's a black box. You can't use it as the one source of truth.

30:38Wes:Yes. And that's where, like, the kernel folks have said, like, it's OK to use, we think, but we're not going to rely on it in terms of being the only option. We're going to make sure we mix it into a bunch of other stuff. So even if it is compromised, that limits the damage.

30:49Chris:Yeah, I would say, in a way, the random generator in Linux is a story of learning that we need multiple sources and we need to make sure that we don't trust any one particular source. We have a clip from Jason Donfeld, and he took over the Linux random generation system from Theodore So, who originally created it in 94. And this was some of Jason's observations when he started working on it. And, you know, he's a developer and he tells it like it is. But I think if you look through that, it's a fair assessment.

31:19Clips:It's in Random.c, which is really old code. It dates back, as far as I could find, to version 1.3.30. I mean, it's really ancient code. There's a copyright date on it from 1994. So I think that even the code was living someplace else before there. But actually, considering when it was written, it's not so bad. it's held up remarkably well despite the knowledge that was around in that time, and I think for all intents and purposes it's okay, it's held up it's been done, with some reasonable intelligence but yeah, Still, over time, there's been a lot of complexity added to it.

32:02Chris:But that complexity is kind of necessary in order to generate randomness that doesn't have one single point of failure as well.

32:07Wes:Yeah, I think his point, too, is sort of identifying, like, what's the essential complexity and what's sort of the complexity that's just crept in over years of maintenance? And how do we keep what we need, but also keep it simple enough to understand?

32:19Chris:And so on Linux, as an end user, you perceive this as there's dev random and there's dev u random. And when I was reading about this, one of the things that I thought was really fascinating is early on, DevRandom, or maybe it was DevURandom, I can't remember, but one of them would essentially let an application ask for a random number. but wouldn't force the application to wait for the correct answer. And so applications would query, but this thing would take a minute to generate, and then they would be like, okay, got it, when they didn't actually have like a truly random number generated. And so later on, they had to create an API, that would force the application.

32:55Chris:It's just called get random. It's the get random API on the Linux kernel. They had to create this API that would essentially manage all of that for the application so they could, in a sense, force the application to wait for true randomness to be generated. but then for compatibility reasons they had to like allow the api to also keep that working in the background and just allow the application to move on and think it had everything it needed like it's it's this funny kind of line they have to walk every time they try to make a change to the system so i imagine complexity does seep in.

33:21Wes:Yeah and i think part of it was just being confused around like the intentions like you random versus random was like never super clear but what the new interface is able to do and get random is it has um underneath there's crng ready and so the kernel now tracks whether its random number generator has been seeded with enough quote-unquote real entropy to be trusted and that's the gate you want to wait for it doesn't really matter if you've generated a bunch after that which is sort of where the you random versus random didn't ultimately make sense but what does matter is especially at early boot has the pool itself had enough mixed in entropy yet and that you probably do want to wait for and that's where it becomes important to like, you know not roll your own stuff to rely on standard apis to rely on trusted audited libraries that are using the correct code paths because that can and does change over the years.

34:11Chris:There's been improvements. Is there anything else you want to touch on before we talk about when it went horribly wrong?

34:16Wes:No, I guess just the rough idea is, you grab random bits, you mix them into a pool, you wait until that pool has sufficient entropy, then you use deterministic pseudo-random number generator stuff like cha-cha 20 in the current kernel and that sort of stretches that value. it can't add more stuff, but it can turn like initial secret seeds into big long streams of random numbers. And that's what you actually pull from. And they sort of generate ephemeral keys and then generate stuff and then throw them away. And there's a whole process around that, but it's not adding new entropy. It's just using the entropy you have to get as many random bits as you actually need for user space.

34:54Chris:Okay, so when you don't get enough random bits, you have situations like the cold card or a situation that lasted multiple years, CVE 2008-0166, also known as Debian's OpenSSL bug. I remember this one. And it started very innocently. A Debian package maintainer was trying to silence a memory analysis tool, a Valgrant warning about OpenSSL utilizing uninitialized memory. And it seemed like, okay, well, this is an easy two-line fix. Clean that up.

35:26Wes:And normally that's good, right? Because why are you reading from memory that hasn't been initialized? It can't possibly have a value you want.

35:32Chris:Right. So they remove those lines of code. They ship it. Everything seems to be fine. The issue was is that they had basically really cut down the entropy of the keys that were getting generated for things like OpenSSL, which you can imagine what that impacts. In fact, it didn't just impact OpenSSL. It was like everything that needed a cert. So SSH, all your X509 keys, like everything got affected by this. And the worst part was it was out in the wild for two years before researchers discovered it.

36:04Clips:So this bug, which affected OpenSSL, the random number generator, was introduced two years ago due to a Debian-specific patch. So it's a Tebian-specific bug, and remained undiscovered for two years, almost exactly two years until Luciano discovered this, and it was published on May of 2008.

36:39Chris:Think about the scope of this for a moment, right? You've got to reissue and revoke every key that was created from 2006 to 2008. There were, I mean, just... Maybe millions, could have been in the millions that were affected by that. It was a massive, massive issue.

36:58Wes:Yeah, it also affected a chunk of live Tor relay nodes at the time, which isn't great. Under the hood, what was happening is that OpenSSL was actually intentionally using uninitialized memory as an unpredictability source. The idea is on different systems, other processes, whatever has been in that system, how it came up in the hardware, all of that will be different. And so it's one of the very few times you might actually legitimately read from uninitialized memory.

37:25Wes:Valgrind doesn't know that. And there hadn't been any infrastructure or communication or clearly comments around it or whatever to, like, make it clear. And so when you remove that, it wasn't like there was no other sources, but it was essentially only the process ID that was left in the pool of entropy. And so that's like a number between 1 and, like, 32K at the time. And that's not enough, right? Right. You'll start having. And so what how they found it is that they found real world collisions out there.

37:50Brent:Right.

37:50Wes:And that's the only way you can, because if you just look at output from this random number generator, you can do statistical tests on it. You can look at just sampling that one value won't tell you. It doesn't tell you how big of a space it came from you. The only way to do it is by looking at huge samples of the outputs in the real world, effectively. And so it's it's really hard to do, especially for a developer making a change. And it's kind of the thing that tends to favor the attacker.

38:16Chris:It strikes me, too, that if an automation tool led to a mistake like this today, we would probably spend 90 percent of the discussion having an existential panic about automation tools and 10 percent of the discussion about the actual bug. But back then, it wasn't that wasn't the conversation at all. And it was so it was so massive. And there was a lot of downstream issues as well. And it's reminiscent of the cold card issue. When you dig through it, you know, it looks like this person that made this commitment, I don't know for sure, but it looks like maybe they were a Python person and they didn't understand the C code and they didn't really know what they were looking at.

38:52Chris:They thought they applied a fix that solved an error that was happening when they were compiling the firmware. They would get this compiler error over and over again, and the guy went in, or the gal, I don't know who it was, went in and made the change that ultimately bypassed the hardware and used the MicroPython. And they resolved the build error and considered it fixed.

39:13Wes:And all the tests were making sure that the build flag was there to say enable the hardware RNG, and it was there. It was just supposed to be a one, and it was a zero. And there was no check on the value. There was only ever a check that the var existed.

39:28Chris:So I just sat there for 20 months just waiting for somebody to come along with a tool to find it.

39:34Wes:And it is a case, right, if you did add your own passphrase thing on top, if you had brought in your own sources of randomness.

39:40Chris:Yeah, which you can with a cold card. You can provide your own entropy. But if you didn't. You have to do that, right? And if you're buying a device like that, you assume it's...

39:47Wes:Part of the value was like, well, this has a hardware source. That's part of why I'm buying it.

39:51Brent:It's so sad that the exact thing they were promising is the thing they didn't do.

39:57Chris:Yes.

39:57Brent:Unintentionally.

39:58Chris:And it was for people that had all the best intentions. But it makes me realize, you said at the beginning, or towards the middle there, Wes, you said, you know, don't reinvent the wheel. Try to use things that exist. In a crazy twist of fate, people that rolled on a Linux box and used the Linux random number generator are better off. And I think there's a lesson in that. And it's why Linux is so important to us as a society is that it is a general technology platform where there are people with mutual incentives that are improving the system and making it and testing it.

40:32Chris:And every time we go outside of that and we recreate something, you take on risk. And this time it's a type of risk that was at the firmware provider level when a lot of people were thinking about physical security and air gapping and 9-volt batteries and all this kind of stuff. And we're not really focused on the actual threat.

40:51Wes:And it also speaks to open source is not enough. Maybe that will help in our new era of automated reviews. But the code was out there, right? It wasn't that it was, obviously the hardware is somewhat hidden, but the code was there.

41:02Chris:You know, it wasn't like a truly open source license. And I think if it had been like a GPL license, maybe more people would have built off of it and discovered this. But because they didn't want people doing that, they did like some sort of like read only you can view license kind of crap. And this is also one of the side effects of that. And it just goes back to like, that's why Linux is so important, is we have solved some of these problems and they continue to be monitored and maintained and it's done right.

41:27Chris:And it really comes back to like we invented it once and we've continued to iterate on it. And what started back in 1994 as kind of a simple system has developed into this truly robust multi-source of noise and entropy, truly great as we can get on these determined systems, random number generator. And that's just one of the many gifts of Linux is that these things like networking and disk IO and random number generation and cryptographic signals, all this stuff is handled by Linux for us. Such a gift.

42:00Wes:We get a world-class product for free. You don't have to go buy a proprietary, cryptographic enhanced private OS from Microsoft or from Qualcomm or for whatever.

42:10Chris:And just trust that they got that aspect right.

42:12Wes:Right. No, you just use the standard kernel that is open source and anyone can download.

42:25Brent:Oh my goodness, gents, there are some boosts here. We're going to start with our baller, Greg the Lawyer. Greg came in with 133,333, sets.

42:42Chris:Whoa! Thank you, Greg. Appreciate that.

42:55Brent:Greg is also adhering by the rules of the show. One of the boosts is a row of sticks.

43:00Chris:Was that a row?

43:00Brent:And it said, well, the other one is actually.

43:03Wes:It is now.

43:05Brent:We've said that, I think, twice. He wants to reward early movers on Buzz. Not for me, but as a Mac user since 1984, I appreciate those early adopters.

43:16Chris:Aw, thank you.

43:16Wes:Aw, sweet.

43:17Chris:Yeah, Greg came coming in with a live boost here because we got low support for last week's episode and we realized we probably should have waited a year to cover Buzz, and so thank you.

43:26Brent:Greg.

43:27Wes:And thanks for hanging with us, audience.

43:29Chris:Also a few Satoshis in there to feed those chickens. I do have, I wonder how I could get those public in a way that wouldn't drain my internet. But I do have a few cameras set up on those chickens. But do I want to dedicate, one to two megabits of my precious Starlink upload link? I don't know. But if I can think of a way.

43:47Brent:Isn't that why you got the mini?

43:50Chris:Well, no, I do the backup. Oh. The connecting backup.

43:53Brent:We've been planning this for months, really.

43:54Chris:And then if I have to switch to it for production, maybe I pause the streams or something.

43:57Wes:What about a system where people, it's like you boost and then the stream is opened for a certain amount of time?

44:02Chris:Opens the Ngrok tunnel to your knees.

44:04Wes:Yeah, so you get like 10, you know, send in a couple sets and you get like 10 minutes of chicken camp.

44:08Chris:That's great. They are the best chickens. You would not believe. Like Labradors. Like they get on our lap and they just snuggle. It's ridiculous. And Boss, the rooster, he's got a routine now in the evening around sunset. He comes in and watches TV on the couch with us.

44:22Brent:What's his favorite show?

44:24Chris:I mean, mostly it's Magnum, but last night he was watching YouTube. He doesn't mind. He just likes the colors. Nice. Turd Ferguson comes. And thank you very much, Greg. You basically brought up the whole trend for the episode. Turd Ferguson comes in with a row of McDucks. That's 22,222 Satoshis. Oh, it's a live boost. The cold card mess has me tired. We trusted the hardware because it was offline, but the bug was already baked in and waiting fun times ahead.

44:58Wes:Yeah, pretty brutal. And then it can just sort of disappear out from under you. You don't have to do anything. It's just moved in a transaction, and the next time you check, it's gone.

45:05Chris:You're like Brent and you take a few days, you go check, and it's gone.

45:09Brent:I did check this morning because I thought that was a responsible thing to do. Not gone yet. Don't try anything on me. But I've got some homework to do after the show.

45:17Wes:Can you describe your gold card more to me?

45:19Chris:You know, it makes me think like, yeah, what else? That is a good point.

45:25Brent:Oh, that's a good question.

45:26Chris:We're just getting this thing rolling, right? Because what do you think the coincidence is that it happened a few days after Kimmy K3 with Open Weights came out publicly? And it's one of the only really current top tier models that doesn't restrict you from doing cybersecurity stuff. And then three days later, this exploit's found.

45:44Wes:I think it depends on how long it took to do some of the pre-work. But it's a good question.

45:50Chris:We may find they were cooking for months. Or we may find that they slammed it in 10 minutes with K3. Because the reason why I bring that up is now people have tried it, and it finds it in 5, 10 minutes. It's just immediate.

46:00Wes:Yeah, I think the main question is from finding it, how long does it take to generate all of the private keys?

46:05Chris:It's a massive sweep and all of that. Yeah, yeah, for sure.

46:08Wes:And I assume there was some staging before you actually submit all the transactions. but who knows that's a great question.

46:14Chris:I'm sure we'll be i mean that's the thing about the blockchain people are already putting it all.

46:18Wes:Together well e scott boosts in with 2371 sats, what what what what happened with nextcloud i have two separate instances running i even moved slash data off my windows 7 ntfs share to a proper zfs mirror with backups also this is a star trek.

46:41Chris:So a quick version is I had like a cold stop will not be using xCloud ever again moment just because the data it wiped out was some of the most valuable data possible. And so what happened is I went to view a text file just using the viewer like it's in the web interface and you click on a .txt file, and it opened and the instant it opened, it wiped the document, it put some escape characters in the top left corner and then it immediately wrote that to the file system.

47:11Chris:and it should have just been a document viewer. It shouldn't have been writing, and it did it instantly. And Wes, you looked it up, and it turned out to be some sort of in-browser issue that is very rare but happened on the client-side rendering. And I realized for what I need, there was just too much going on there. And the fact that by just viewing my most important document, which is what I used NextCloud for, was for its privacy, totally offline, off the Internet, by just viewing that document, I wiped it out.

47:38Chris:And that was when I was like, I don't need any of this. And I decided at that moment I was going to stop using it, which has been a real pain in my butt. But when something fails that catastrophically on you, sometimes it just leaves a bad taste in your mouth. And that's after it worked for me for a long, long time.

47:57Wes:And it doesn't mean it still couldn't be a great CalDev suite. There's lots of roles it still makes sense for, especially if you have robust backups and it's just one place to look at them.

48:04Chris:It was just sort of the last straw, honestly, because it's been a lot to upgrade and manage over the years. and I sort of have rolled back what I use it for to some degree. And so that was sort of the last thing I was really, really relying on it for and it bit me.

48:14Wes:And made you reconsider. Do I even need it?

48:16Chris:Yeah.

48:17Wes:I did look it up. 2371 is a well-known year in Star Trek. So we got a few things. Well, third season DS9 begins then. It's also the year USS Voyager launched on its ill-fated mission. And it's Star Trek Generations is set in that year when Captain Kirk meets Captain Picard.

48:35Chris:That was such a great time for Star Trek. That was such a great time. Thank you, sir. Appreciate the boost.

48:42Wes:Thank you, E-Scott.

48:43Brent:Unicorn, 1999 came in with 6,000 sats. At some points in open source and with some projects who had to sign off that you were the author of the code and had not used any source that was also not clean. But now with AI, that doesn't really matter. It seems the who owns AI code question is not completely solved.

49:13Chris:Wait, why is that? Why is that?

49:16Wes:Well, there's still active legal debates around it as one. So, like, for instance, some courts have said that pure, like, if all you did was one-shot code, it can't be copyrighted at all. So I think the question of, you know, exactly the law has not caught up with the current tool set is sort of one aspect of it. Like we have some cases that mostly dealt with code in the era of browser tabs and nothing that's really dealt with the case of you're doing project management on an agentic harness and then you edited some parts of it or you reviewed it.

49:47Wes:So I think that's where some camps concerns are coming from, just in that it is still an open legal question.

49:53Chris:But it seems like if my self-driving car runs into Brent and smashes his van up, I'm legally responsible for that. It doesn't matter if the AI degenerated that.

50:03Wes:Yeah, but that's a legal sort of – it's a separate set of law essentially, right? Like this was a judge making a case about if you could copyright pure AI-generated content.

50:13Chris:Oh, copyright, copyright.

50:13Wes:Yes. So like when you make something – not everything is copyrightable, but if it is a copyrightable expression, not just the idea but the expression, you get to sign copyright. to it that's how you add a license to your code files but they're saying if all you did was prompt a single ai to generate code that cannot currently be.

50:31Chris:Copyrighted i just think in the context of most most not all but most free software project this is really ant humping because who's responsible is the person who submits it and this is sort of full stop, i mean this anything else is just sort of ant humping now when you're working on a giant international project maybe that's really where things matter like that i think but that's my opinion.

50:51Brent:There were some explorations of laws in europe though that recently suggested that open source software might be responsible if people use it in certain contexts so, that's an interesting angle unicorn 99 here wants to clarify though i'm not anti-ai i'm anti-corporate exploitation.

51:10Chris:Boy sure the corporate ai aspect is so obnoxious so tiring i agree with you there it's the worst about it it's the worst thing about it and yet We need their precious GPUs, at least for now. Hey, Mr. Mayhem's back with 16,000 Satoshis. Look at that. Boost the roost, boost the roost, boost the roost. Something broke. It's a fill-in boost. Well, boost the roost indeed. Thank you. Appreciate that. And that is all of the paid boosts, but we do have some member boosts. Would you like to kick that off, Wes?

51:44Wes:Oh, you know how I love a good member boost.

51:47Chris:Hey, Unicorn's back.

51:47Wes:Oh, Unicorn1999 comes in with a member boost.

51:51Chris:Hey!

51:53Wes:I think tech people bringing up the copyright issue is not only crap, but partially how copyright is always used by corporations against us users. Like, it was okay back in the day for ISPs to host Usenet servers full of TV shows, but not okay for people to download and share.

52:08Chris:We don't talk about Usenet. We don't talk about Usenet. Yeah, no, I think it's like we're talking about two different things here, Unicorn. Because everything you just said I completely agree with. I just think there's hypocrisy in folks that, you know, have their RAR stack set up or whatever, or rip DVDs, then now concerned that LLMs are ripping content, right? There's just hypocrisy in that. But yeah, using copyright as a kludge is never good.

52:34Wes:There's also sort of like, we're seeing projects now, which is perhaps reasonable in that they are much more established, projects, but we see projects now that are having concerns over copyright questions that maybe were different than how Linux was treated and being actively developed in the ESCO era, where there were similar sort of copyright questions.

52:51Chris:Mm-hmm, mm-hmm, mm-hmm, mm-hmm, yeah.

52:55Brent:Mm-hmm.

52:57Chris:All right, well, Greg, the lawyer's here with a memboost. Welcome back to the States. I hope the trip was flock-free. You know, I'm not sure. I doubt it, though. I doubt it. He says, I got fed up with Dario's nonsense and switched back to Codex. I figure their model is better because they broke out first. The weird feeling when switching VC-funded frontier models is how I fight in part for open source. Yeah, although if it makes you feel better, the Codex is open source.

53:25Chris:The backend isn't, but the Codex client is open source.

53:28Wes:Amadeus member boosts in. My workflow of cleaning up my own drives is using a great Rust tool called Dust. Find out what takes up space, old hugging face models, old Proton versions from Steam, Podman containers, et cetera. Works great, and I'm using it on every machine I control or manage.

53:48Chris:The Podman containers makes me chocolate.

53:50Wes:Thank you for concluding the very realistic selection of actual disk usage.

53:54Chris:Yeah, yeah, really, yeah.

53:56Wes:And Dust is a Apache 2 license.

53:59Chris:Good to see. Nice little extra pick there.

54:01Brent:We've got Joshua here with a free members boost. I'm a software engineer and always have ideas for side projects, but no time to actually make the most of them. Cloud Code has really changed that, though, but I worry about making my new projects open source because of public backlash, since they're all written with AI. Do you have any advice?

54:26Chris:I would bet Joshua that you would actually find the reverse problem is going to be your issue is that if you open source it and it's popular, you're going to have a lot of people that are contributing back with AI assistant code development. You're going to have a lot more of that than your people that drive by your repo and get upset at you.

54:42Wes:Yeah. Like I think it would be different. Like if you if you vibe something up and go share it on three subreddits at once. And like there's there are versions of that where you can get yourself in trouble. But if all you're doing is sort of putting it out there, I don't think you'll receive too much backlash.

54:56Chris:Anonymous comes in with a member boost. It never occurred to me that a nutcase might not only have a thousand tabs open, but that nutcase might also have them scattered across dozen of windows. Absolute madness! I know, I know.

55:19Wes:Are we, it says anonymous, but I don't know. I kind of smell the PJ booze.

55:27Brent:I don't think I've been called a nutcase before and certainly not for my tab habits.

55:31Wes:Well, I came from the whole squirrel thing.

55:33Chris:Yeah, as a squirrel father, I think you would put yourself in the nutcase category.

55:36Brent:Yeah, that's fair. That's actually kind of a compliment. Thank you, anonymous.

55:40Wes:Well, SensorSmile comes in with our last member boost. I'm finally ready to start playing with AI. However, I'm not sure where to start. I have a local Quen model up with Lama CPP. The built-in chat interface is boring. How do I code with this? Hermes? How do you do long-running agent? A beginner's guide for the tech experienced.

56:04Chris:This is SeniorSmile. You're like, you're starting with Bitcoin by installing a node before you've acquired sats. I mean, not necessarily a bad way to go to truly learn from the ground up, but, you might You might just spend a month with OpenCode and OpenCode Go subscription, and learn that because what they're going to have for you is agents that are, A, optimized for coding, B, work right out of the box with tool calls. And when you're going to do your own Quinn and your Lama CPP, I love it, but you're going to need to use a front-end interface.

56:36Chris:You're going to have to pick one that understands all that stuff, that knows how to work with that stuff. And that, when you're just getting into all of this, feels like a little extra complexity before you've even really figured out what they can even do for you.

56:46Wes:And they could also help like if you do get open code going or whatever.

56:50Chris:Then you can have it set it up for.

56:51Wes:You exactly you can have open code talk to Lama CPP no problem but that's just more stuff you kind of get in that isn't actually playing with AI but you can have the AI sort of bridge the gap and then try out both local and cloud hosted.

57:03Chris:Not too bad there Westpain alright thank you everybody who supported this episode with a boost we had 18 of you stream sats as you listened along and collectively you all stacked 17,625 Satoshis for the show. And you bring it all together with everybody who boosted. We stacked a humble but very grateful 198,570 Satoshis. Not the best hourly rate when you split it four ways, as we do, or five ways, really. I think it was like five in there, but we still appreciate the support.

57:38Chris:It is a lean season. When you hear the dynamic ads, you know we're hurting. But we really appreciate the members and the boost. It's also one of our favorite segments on the show. It's stuff we never really plan to talk about, but often is great conversation. And you can send a boost by going to boost.jupyterbroadcasting.com or jupyterbroadcasting.com slash membership or linuxunplugged.com slash membership to be a member. That's it from us for the boost. Thank you, everybody who supported us.

58:27Chris:And we got a couple of pics before we get out of here. This is one I've been sitting on for a while because I thought maybe we would use it for our Texas trip. But I think I'd like to put it out there and get a sense if the audience likes it and if anybody deploys it before we use it. But I could see this being very useful for us in the future. It's called Trek. I already like the name. It's a self-hostable travel trip planner with real-time collab features, interactive maps, progressive web support, single sign-on, planning budgets, packing lists, and more.

58:59Chris:I mean, this thing, and it looks good. It looks like a top-grade commercial application that you can self-host. And then you can work with your friends, your family, whatever it might be, together to plan a trip. And, like, you know, make your big plans, put your budgets in there, get it all on a calendar. You can subscribe to the calendars. Like, you know what I'm saying? Like, this is a real tool with interactive maps and stuff like that. So you get a journal for every trip.

59:24Chris:You get real-time collaborative planning. You see it all on a map. It has lists and categories. It has – you can put in, like, we're going to go on a vacation. You can have it pre-visualize that. You can track costs per person. You can drag things around and drop them on the calendar. And Wes, it's got an API.

59:41Wes:Ooh, okay. Okay. This looks really nice.

59:47Chris:Yeah, I'm pretty excited about it. Like I said, I haven't had a chance to try it myself, but I think somebody out there in the audience might be willing to give it a go and let us know. It is AGPL3.0, and it is primarily TypeScript.

59:58Wes:It's got a Docker Compose setup, Kubernetes if that's your style. Yeah, okay.

1:00:03Chris:Yeah, yeah.

1:00:04Wes:The real-time collab part sounds quite interesting. We could just be on a call, workshopping a trip, trying things out.

1:00:10Chris:Teasing Brent about how long he has to drive, because we'll just put it right there on the map in front of him.

1:00:14Wes:Right.

1:00:15Brent:It'll probably help me with that part. One of the big questions I have is, will it nix?

1:00:21Chris:Well, of course. It does also have a Docker if you just want to go that way. But from that, you could get to a declarative Podman configuration and about two licks and two shakes of a lamp's tail.

1:00:31Wes:Oh, it includes built-in MCP as well as an API. Great.

1:00:35Chris:I mean, you could see how maybe our agents use this to have a plan and they just tell us when they're done. All right, Brentley, you got a little sneaky pick. I believe that you wanted to pull forward, as they said.

1:00:46Brent:I'm representing this sneaky pick as producer Jeff actually brought this up to me and he said, Brent. This is perfect for me and therefore probably perfect for you. It's a to-do app that is called C'est Fait, which is French, but I'm going to declare that everybody else can probably just say C-Fit.

1:01:07Chris:Oh, yeah. I would have sold it.

1:01:08Brent:You tell me what you should call it.

1:01:09Wes:Can we get the French one more time?

1:01:10Chris:I would have said C'est Fait.

1:01:12Brent:Or C'est Fait.

1:01:13Chris:C'est Fait. I know, because it's got a cat. So I would have gone with cat, you know, because it has a cat logo.

1:01:18Brent:Yeah, yeah. Yeah, that works too. A silent F, right? In French, that really means it's done, which I think it's kind of clever.

1:01:24Chris:Ah, I gotcha. Okay.

1:01:26Brent:But it's built for people who, quote, want keyboard-centric efficiency, which I have been looking for forever. Jeff, you know me so well. It looks like you can use it comfortably from the command line via TUI. You can use a desktop GUI if you want or on the go with a native Android app. Ain't no Electron in here.

1:01:46Chris:It's got a TUI and a graphical app and an Android app.

1:01:49Wes:This is impressive.

1:01:51Brent:I don't know how none of us found this It's also.

1:01:53Wes:It's apparently available in FlatHub already, FreeBSD There's obviously, it's in FDroid and Google Play There's Windows releases maybe There's Mac OS stuff, the only thing I'm not seeing is Nix really, but we can.

1:02:03Chris:Get that done As a long time, and slightly ashamed Todoist user This, this cooks I think this is blowing Todoist away Even at the UI layer.

1:02:14Wes:To find shortcuts on the fly Wow Yeah.

1:02:17Brent:It's nice, there's a little detail here It's 69.5% rust.

1:02:23Chris:All right. You know what?

1:02:24Brent:Does that count?

1:02:26Chris:We'll give it to it. It's over 50%. It's over 50%. Everyday usage. Press the question mark and navigate to the help tab. Configurations in the CLI-2E under the hood. All right. So, PJ, you've been trying this? Like, what's the dealio? What do you think?

1:02:39Mumble:It really did catch my eye. I found it in F-Droid just going to do some updates, and it was right there on the front. Yeah. I really do like it. I like that the Linux desktop app and the phone app, they look very similar. They're not these giant, there's no giant changes between the two.

1:02:56Chris:Right.

1:02:57Mumble:So you don't need to necessarily learn all the key bindings because I never will remember them anyways. And I really like the subtasks. You can tree out things. I dream of an idea of having dependency tasks. This ain't quite that, but the subtasks kind of get you there. So I've been using it.

1:03:13Chris:So it's like a, it's like subtasks isn't quite a project, but you can have a task with a bunch of things you got to do to complete it.

1:03:19Mumble:Yeah, exactly. So I have, I'm kind of using it for goals. Like I have, you know, truck repairs, right. And then a bunch of subtasks under that of the different things I want to do for my truck, you know, house repairs or home things I want to do. And I can have a bunch of subtasks for that. And it's been working out pretty well for that. I like that a lot. Just a dedicated place. I can put the things I want to get done or, oh, I've got 15 minutes to kill. What kind of thing can I do?

1:03:41Mumble:Well, here's a whole list of things I want to try. I want to do.

1:03:44Chris:Right.

1:03:45Mumble:And I'll forget about them otherwise.

1:03:47Chris:How is the self-hosting side set up the server side?

1:03:50Mumble:I'm using NextCloud to sync everything up.

1:03:53Chris:Oh, so you don't have to run a backend in particular? You could just have it sync?

1:03:57Mumble:Whatever CalDev server you want to use, yeah.

1:03:59Chris:Oh, okay. I see. I see. So they include Radical, a self-hosted lightweight feature complete thing you can use, or, yeah, NextCloud. Huh. This is great, Jeff. Nice find. I'm glad Brent, I'm glad he pulled it forward.

1:04:17Brent:Mm-hmm.

1:04:19Chris:I may give this a try afterwards. Especially if I could store it, of course, I don't know. If I could store it in something, we'll see.

1:04:25Wes:Yeah, especially if we can compose it with a few other things. The fact that it's got the Android layer solved, there's a lot of promise.

1:04:31Chris:There's a lot I like about it. And the client's in Flathub, which is nice. And the Android version's in F-Droid.

1:04:37Wes:Which is nice. Okay, so PJ's now game correspondent, right? And I guess he's also like a...

1:04:42Chris:Productivity.

1:04:43Wes:Yeah, productivity expert.

1:04:46Chris:Productivity John. PJ still works for that. Productivity John.

1:04:50Wes:Lifestyle expert.

1:04:52Chris:Yeah. And a few other things. Solar, long hair, floor showers.

1:04:58Wes:Oh, floor showers. He's got that down.

1:04:59Chris:He does have the floor showers down.

1:05:01Brent:Also our on-site Californian expert.

1:05:04Chris:It's true. If you want to know anything about California, you just ask PJ. All right. Well, we'll have links to everything we talked about at linuxunplugged.com slash 678. We're slowly working our way to 700, so we hope you stick with us and join us live. You can make it a Tuesday on a Sunday, 10 a.m. Pacific, 1 p.m. Eastern, over at jblive.tv. I say before we go, we tell people some pro tips. We should always, maybe we should start the show with these.

1:05:28Wes:Instead of burying it all the way at the absolute end of the show?

1:05:30Chris:Right.

1:05:30Brent:We should, like, do a reverse show sometime. We should start from the bottom.

1:05:35Wes:Flip the duck. Surely a bot could flip the duck.

1:05:38Chris:We should flip the show.

1:05:40Brent:All right, next week.

1:05:41Chris:We should think about that. That could be a lot of fun. Don't you think?

1:05:45Wes:Probably.

1:05:45Brent:Oh, yeah.

1:05:46Wes:At least tell us what we like and don't like about doing it.

1:05:48Chris:How do you handle the intro and the outro? Do you do the outro at the beginning? Because that would really screw with new folks. Like, you lose them immediately.

1:05:55Brent:Oh, go for it. Come on. What's one episode?

1:05:57Chris:I don't know. I think if we could, the intro is the only thing. We need some input on that, but I would totally be down for a reverso episode. That could be fun.

1:06:05Brent:No, it works perfectly. You start and you say, about the same bat time, same bat channel, you know, make it a Linux on a Tuesday, on a Sunday, and join us Sunday.

1:06:13Chris:But then we play the outro music in what?

1:06:16Wes:No, it starts with the outro music.

1:06:17Brent:We'll figure it out.

1:06:20Wes:I don't think we will. And then you intro the show at the end.

1:06:22Chris:I think we need help.

1:06:24Brent:Drew will save us.

1:06:25Chris:Oh, yeah, sure. Yeah. All right. Okay, very good.

1:06:28Wes:But in the meantime, JSON. It's in the cloud and in the feed. We put JSON in our XML, but then we also put like text in there in the form of a VTT.

1:06:37Chris:That's true. So you can have it either way.

1:06:38Wes:And we put in a link. It's not the whole MP4 but there's a link to an MP4 embedded within.

1:06:43Chris:Yeah. So you could watch it in video version and you may be surprised there's visuals but because we're such effing pros you can just listen to the audio version and it never detracts from that to such a degree you wouldn't even know there's a video version. How about that? Ha! So yeah, you can find that MP4 in the feed as well and our Mumble Room is always going every single Sunday. So do join us live. It's fun. It's unique. And not a lot of podcasts do it.

1:07:11Chris:And, of course, links to everything we talked about at our website, linuxunplugged.com. And the back catalog and all the other great shows over at jupiterbroadcasting.com. As long as you want to check it out, you might as well go to the calendar. It's over there in a contact page as well. Would you believe it? And last but not least, you can get a message and support the show at boost.jupiterbroadcasting.com. And you members, don't you forget you get a free boost every single episode because we appreciate you so much.

1:07:36Chris:In fact, we appreciate you just for listening. Thank you so much for listening to this week's episode of your Unplugged program. And you know what we're going to do? We're going to work on a show right after this, get cooking on another one, and we're going to see you right back here next Tuesday, as in Sunday.