160: Position Localization

0:00A:Programming Throwdown Episode 160: Position Localization. Take it away, Patrick!

0:21B:Welcome to a nice ending in zero episodes. It's like a 10th thing, 160-16-10s. We've had it—it's our Sweet 16, Sweet Deca 16, I guess. Oh, I was trying to come up. Thank you, that was better. So, I want to talk about something that I feel was always loitered around. I remember it going back a little bit of a nostalgia trip, and then we're going to take it forward. So, I'm going to give why I'm bringing it up, and then—and then I'll sort of get into my monologue here. But I remember going back to okay, not all the way when I was a kid—that's too far—but I remember going back to like the early 2000s and there being portable PCs. So, not like a laptop, I mean. There's kind of been a thing, like it's kind of obvious, but I'm talking about something in sort of like okay, like a Game Boy Advance, Nintendo DS—that form factor. And the reason I'm bringing this up is the Steam Deck. So, the Steam Deck is now sort of generally available just recently. It's been like, I think they're going to be in retail shops in Japan. I saw some cool pictures, and so instead of this hyped-up, you know, hard-to-get big waiting lists—you have to pay overasking—the Steam Deck has come out, and I want to talk about. So, I have one. I hadn't played it a lot, but now I've been playing it more and more, and I'm going to sort of talk about like my journey, but also this nostalgia trip. Of, I remember there used to be Sony Stores, Sony retail stores, and there was a specific mall—you know, I was older, but you know we were in this must-have. I didn't look at it; I should have looked it up, like 2001, 2002, 2003—that time frame. And you would go to the Sony Store, and they would have, you know, digital cameras, and they would have these little like bio-computers that had like little tiny keyboards and like a screen. And I never knew that, you know, before the iPhone, right? Before sort of on-screen capacitive sensing. So, it's like resistive touch. They didn't have a long life there. Wasn't, you know, data over cellular networks wasn't like an everyday person thing, and so it just weren't all that compelling. Maybe if you had like a very specific use case, you know, you would have done it, but for most folks, it wasn't super compelling. And

2:19B:feel like, you know, they kind of ran Windows—maybe you could get one that runs Linux if you were an elite hacker or had some specific thing, but you know they kind of languished around. And then the Steam Deck for some reason like captured my excitement. Maybe in part it's just I have this belief that I'm going to be a video gamer even though I'm not really a video gamer. I'm just a casual gamer, like I play just, you know, iOS games or whatever. I'm not—not big on PC games. I hardly ever sit at my computer and play them. I do have one. It is not technically a gaming PC, but I just have a GPU; it is capable. I just don't sit there and play it very often. But I got the Steam Deck, and I already had a bunch of games—huge appeal to me that, you know, you can have lots of concerns about DRM or whatever. I'm just going to leave that aside for this conversation. But through various Humble Bundles and other things, I picked up lots of, you know, five-to-ten year old video games which was like perfect for the Steam Deck. So, can the Steam Deck play anything? Like what's the connection between Steam and the Steam Deck? So, this is the part that's really cool. So, first, it already comes built in with access to your Steam library. Okay. The second thing is the way they've made it. It's, you know, in contrast to a lot of cell phones; it's very commodity parts. So, it's an x86 processor from AMD with an integrated GPU, and by default, it runs on SteamOS, which is built on top of Linux, but it very easily will let you drop back into Linux. You can attach a keyboard and mouse and like install your own packages. Wow! But you could also install Windows. You can dual boot. I haven't done that; it's a little off the beaten path because they're still working on BIOS changes and stuff, stability. One day, I think I'll do that. For now, I'm not. So what they've done is they have this a library called Proton. And Proton—I'm pretty sure this is great. Proton is sort of like what Wine did, right? Like providing the Windows API calls and doing a translation under the hood. So all of these developers have built all of these games most of them don't run on Linux sort of natively. They run on x86 but they—they make Windows or DirectX calls or whatever, and it can cause issues. And so there's

4:16B:this interface like wrapper that they will run and detect it, and they also provide this mapping from the keys, and they have trackpads. They've done some really cool stuff on the hardware, but on the software side, yeah, it's super compelling. And so all these games just sort of like—they have ones that they verified work, which is a huge lift on their part that they're like trying to go through and figure out ones that work. And there are quite a surprising number that work fully, like they understand the Steam Deck. They don't pop up an on-screen keyboard that you need to type like; they recognize like you can use your gamepad to sort of enter audit. And because now there's a lot of crossplay with PlayStation and, you know, Xbox and PC, a lot of developers have these kinds of things in mind when they're building the games, but it's just really coming to its own. And for me, what I was trying initially was a few games like Dyson Sphere Program. I was really into that at the time. Like, I'm gonna play this on my Steam. It was terrible. I hated it. It was not good. I just needed a mouse and keyboard. People use the trackpad. They have actually a community that owns like all these different customizations so you can just go to the library and people have like reviewed key mappings where like different spots on the little touchpads on the side correspond to different things, and hardcore users are probably hating me because there's an amazing in-depth like customization group who probably spend as much time like optimizing settings and like running plugins to sort of manipulate like under-the-hood things. I'm not that person. They're just like install a game and try it. That's very satisfying for

5:45A:those people, you know, but it's a whole dedicated hobby.

5:48B:to do. Yeah, yeah, like it becomes as much of a hobby as playing the game, right? And some newer games like Elden Ring and stuff supposedly run really well. I'm too cheap to buy it, so like I don't actually own it. It's too new, but it's like—but it's like $10. I'll buy it, and I've got a big backlog, but what I realized was embracing sort of more casual games I had just never played before. So, like everyone raves about Hades. I'd had it on, oh.

6:09A:it's a great.

6:10B:one my Xbox, but like on the Steam, it just works great because you—it's sort of like a short run-time sit and play, but it's not something I have on my phone or my iPad that I'll try to play, like Dead Cells. But just like for me having the gamepad and like there's just a much better experience. How do you

6:29A:carry the Steam Deck? Like it doesn't fit in your pocket, right? So this is the kind of thing you need, like a back very big pockets if those old JNCO jeans from like the '90s cargo pants. Um.

6:38B:So, one of the things they have a really nice carrying case that comes with it, but you're right, it's sort of bulky. So now though, because there's enough of them have been moved and the community is sort of building up around it, you actually get a variety of companies are making at different price points sort of silicone like wrapper things that go around the outside, like rubberized stuff that have like hatches over the top to protect your screen so it won't survive like a drop in your backpack. But if you want to just slide it in and not have your screen get scratched, you could put a scratch protector. They also have little domes that cover the control sticks so they don't get, you know, knocked. Um. So I recently go, and they have like little kickstands on the back so you can have a controller and like kickstand it up like on an airplane or something. So there's a lot of these things building up, and then I recently—there's, you know, been a lot of hype. So the Switch is kind of Nintendo Switch is like in a similar form factor. I think that's been a great, you know, kind of cool thing that it's both the like successor to the Game Boy and Nintendo DS kind of lines, but also with the console from Nintendo—that's pretty cool. In the same vein, in my opinion, almost similar hardware, but and now there's some new ones coming out as well from other companies, and what I just wanted to shout out is like I think it's kind of here like the sort of miniaturized computer that you know can have a web browser, plays all these games, and is much more just yours than sort of the phone ecosystem. And they're going to kind of shrink, and then coming from the other end there's a bunch of big phones put into the same sort of form factor but a little bit smaller where they look like Game Boy Advance or whatever, and they're really just running Android emulators, and you can play retro games, but also Android games. So Dead Cells and things work great, and so they're sort of converging into this like middle ground of sizing that you can just play really, really powerful sort of like games and have this sort of on-the-go experience. I just think it's

8:26A:really cool. Yeah, I used to have back when slider phones were a big deal where you'd slide the keyboard, you know, from under the phone. Um, I used to have a phone—I don't remember what it's called. I think it's called—I don't remember—but it was from Sony, and when you slid it, it was a gamepad. And so you had like a full gamepad. It was a, you know, Android, you know, one or something very old, but that was amazing. I mean, I beat Final Fantasy VII on that. I mean, put like a ton of hours into it eventually. Like all the slider phones, the ribbon connecting the two halves, like fault, you know, fails, but yeah, that was phenomenal. The other thing that I was following for a long time which sounds like the Steam Deck finally sort of cracked this walnut, but I was following Open Pandora, which was a—uh—basically it sounded exactly like the Steam Deck. It ran Lennox, you know. It just the thing is, it was expensive, and there were just always issues with it, and it just didn't have the level of polish for me to dive into. It sounds like the Steam Deck really got it.

9:23B:think for me. Like in part it's not trying to be everything to everyone. He's not trying to say, 'Oh, this is going to be your laptop on the go,' because people just have their phones like—yeah, that wasn't that's not a need anymore. But some people just don't want to have a gaming PC. And I know folks that are like, yeah, I have a video game, but like I—I don't like what would I might spend that money on a PC now go to the r/PCMasterRace subreddit if you want and like, you know, you know indulge in your superiority. But there's a lot of folks who just aren't going to do that. And this is sort of like—I think a good middle ground, right? Like you don't necessarily take it on the go, like, you know, if you're just going to have a few minutes, but you could take it on a trip with you. Um, but you could also just play it on the couch or whatever. I know, I think it's—it's a pretty cool middle ground, and the sort of software stack that they're figuring out is really cool, like all these Windows games being able to run on Linux. I think that's—that's really awesome.

10:13A:Yeah, that is super cool. I'll have to check this out. It looks like it's around $600, five to seven hundred depending on the amount of storage you want.

10:21B:And the great thing is, like, it has really good support for the Micro SD card. So I bought the smallest one, and it isn't a problem for me. I just put a really big Micro SD card—oh, that is cool, although not cheap—is also like I could take it to something else if I needed to or I stopped using it. It's a less of a commitment, so yep.

10:40A:That makes sense. All right.

10:42B:Very cool time for the news. Well, I did not prepare the ordering very well, so I'm the first up. All right. So I know Jason is somewhat of a retro video game enthusiast, so definitely sure he's seen this person before. There's a YouTuber called Summoning Salt who just makes the most like straightforward—if I tell you what it's about, you're gonna be like, 'Whatever.' I'm not that like this sounds so boring,' which is like in-depth exploration, like an hour-long video of how the timed speedrunning records of various video games fall over time. Yep, this guy's amazing. So he gives her a narrative history. He does all this work. I was watching the one about Super Mario Bros. 300. So there's like all these different categories. I know nothing about this. I don't have any desire to do this speedrunning.

11:30A:I'll cover this. Okay? So go for it. Go for it in speedrunning. There's Any%, there's 100%, and then there's another one I forgot the name of—it's effectively Zero%. So Any% means you just have to get to the title, the ending screen, right? Um, 100% means there's a list of things you have to do, which can constitute 100% of the game and it's defined in advance, and you have to get to the title screen. And then I think it's called Mini%, but basically it's a list of things that you're not allowed to do. Like for example, there's actually in Zelda II—I think it's Zelda II or one of the Zelda, maybe all the Zeldas—you can actually beat the game without getting the sword. It's really hard, but it's doable. And so there's like a speedrun of I think Zelda 1 and Zelda 2 where you have to beat the game without getting the sword. That's called Mini%.

12:25B:There this guy has a variety. Some of the ones about like Mario Kart very interesting games you've played sometimes. You watch a run and it looks like 75%. It just looks like very precise, like very cool stuff. And then something will happen where he'll just walk through a wall. You'll be like, 'Wait a minute, like what happened there?' And so there are I guess like there's rules about what's considered like a glitch and what's like an acceptable glitch. And so like oh, if you bypass the whole game, you know, like to Jason saying that doesn't really count. But if you're just like clipping through a wall, then that's okay. But what ends up happening is inadvertently they talk a lot about video game programming. Like, 'Oh, you know, Mario runs at this speed in pixels,' which is a fractional amount because that's what they decided. So there's this like sub-pixel counter. And so what someone's doing is they're trying to control this hidden variable, get it into a certain spot and then go do something so that they get the highest probability of like this thing, which is supposed to be random, like reliably occurring. And you just get into this very intertwined combination of like how and the 100% one—this one that I'll have in the show notes—or if you just, you know, you can kind of look up. It's brutal because it's many hours long because you have to play every level, every mini-boss, every everything. And sometimes you get to the very last level and just look, there's something that's out of your control. It's just randomly the movement of this thing, and you can get a really bad random number generator (RNG) and like ruin a—you'd have a world record run, get to three hours in about perfect frame inputs, and then all of a sudden it's just trashed because, you know, you drew.

14:01A:Bad. Yeah, it's unfreaking believable. There's my favorite of all of this is—I was watching so, you know, I'm a big fan of the Final Fantasy games. You know, I love the story. And I was, you know, when I was younger and I had the time, I would read a ton of fiction books, and it really like kind of touch that cord. It's kind of, you know what it did? It connected the fiction books I was reading to video games because up until then there was Pac-Man, and then there were, you know, fiction books which had this really deep plot and story but were totally non-interactive, and they occupied like two different worlds. And Final Fantasy, as primitive as it was, and like the Super Nintendo—whatever or the Nintendo—it like connected those two worlds, which is really powerful for me. So um so I wanted to watch the Final Fantasy IV, which is in the U.S. is called Final Fantasy II that speedrun. And I watched the Any%, which means you just have to get to the ending screen however you can. Um and so I'm watching this person's like yeah playing this like perfect game and doing things like you'd only do if you do what was coming ahead and everything, right? But then he gets to, you know, this um tower where this Ruby gets stolen, which is about like a fifth of the way through the game, and the video is almost over. And I'm just scratching my head here. And basically what he found is there's a spell called Warp, and Warp takes you up a level in a dungeon. So if you're, you know, on like two levels down in an underground dungeon, it'll take you to one level. It doesn't take you out completely; that's the exit spell. It'll take you like a little bit closer to the exit. And if you're going up a tower, you know, it takes you down, takes you a little bit closer to the dearest town, right? But there's this one part of the game where the developers kind of forgot that they had to think about this Warp spell, and when you cast Warp, it just takes you into like random memory where you don't belong. And he's walking around, and as he's walking around, like every tile is just random, like.

15:59A:None of it is coherent. And then he walks forward, walks around, and then all of a sudden he gets the ending credits. I was like, 'I was like okay, all right, I guess that happened if you're.'

16:09B:Ever looking for though, like I don't know, it's like a very nerdy kind of documentary thing, but it's just for me it's very easy to just like sit there and watch. I don't know what about like the way these particular ones are done is just it's so easy to just tune out and just like yeah almost like in the background, like it's not that I'm doing something else; I'm just literally watching it, but just like it's relaxing. It's just yeah, it's very like—yeah, I don't know, meditative. I encourage you if you've never done it before. I was gonna say that feels a little wrong, but uh yeah, if you've never watched one of these something, so there's a few other individuals who also do descriptions of these time-time runs which are also really good. There's a I guess a community of them, and they're all very good.

16:48A:Very cool. Um my first show topic is really wild. It's it's AutoGPT reaches a hundred thousand stars. So you know, it took PyTorch like multiple years to reach a hundred thousand stars. There's only two as far as I know; there's only—well now three GitHub projects have a hundred thousand stars: one is TensorFlow, one is PyTorch, and now we have AutoGPT. You know, to be honest, I've heard like the most insane stuff about AutoGPT. What I think it does is—I think you go in and kind of like connect this to a bunch of other projects, but what I've heard is are things like people have got AutoGPT to like reduce their monthly payments. So this one guy said, 'Uh, you know, AutoGPT, you know here's all my tax form or not tax forms, here's all my um banking statements,' you know, 'Reduce my monthly bill.' And apparently AutoGPT and this almost sounds fake, but I had to read it this multiple times to convince myself it's real. AutoGPT found this like Wi-Fi airplane charge and said, 'Hey, you know, you can actually send an email to the airline company to like cancel your Wi-Fi airplane charge,' and because they don't do it anymore, they'll retroactively like give your money back. And it even drafted the email, and so he sent—he copied the email from AutoGPT, sent it to Verizon, and they like credited them $30, and it like also like canceled some other things. I don't know. I've just heard absolutely insane stories about this AutoGPT. I have a friend who is in finance and was using it to like take care of some things like with taxes and all of that. I have to really, you know, we should maybe do a whole show on this. I have to really wrap my head around what this is.

18:46A:And like how it's different than just GPT-4 or, or you know, ChatGPT, but it's just blowing up in popularity. Yeah.

18:54B:There's a few of these self-prompting GPTs. So I watched a video on one because I was curious as well, and they are mashing together a bunch of tools. So there's a couple of like infrastructure plays. So there's like vector databases um and there's a few paid and unpaid, and you can sort of connect them up to one of those. You can also just tell it to like do its work locally. There's various trade-offs depending on what computer you're on, like how much, you know, memory you can use. And we talked about like locally running some of these as well, and it'll use the like tokenizer from OpenAI so you can kind of plug that in and just configure it on your computer, and and it's not hosted, so different than going to like OpenAI and going into ChatGPT and typing something. You're like connecting it or hosting your own sort of infrastructure. And the one that I watched is is more or less—and you see this like oh, it built me a website or whatever, and you sort of give it a task list, like something it's trying to accomplish. Uh so the one I watched was like 'Build me a healthy meal plan for this week.' And so uh it tries to figure out, okay, what do I need to ask you to do that? And then it'll ask you some questions, like, 'You know, hey, what kind of food allergies do you have?' or whatever. And you're kind of giving it answers. So rather than, you know, asking it questions, you're sort of saying, 'I want you to do these things.' And it's figuring out what order to tackle them in. Okay, I'm gonna build you Thai food on Monday. And you can tell it to either go ahead and just do whatever it wants or the mode I was watching it was like prompting, 'Hey, I want to issue a Google search for like best high dishes that are high in protein,' or something, and it'll sort of figure out that query. It'll issue it off to Google if you tell it to, and then it sort of scrapes and parses the feedback and decides what to do next. So rather than you saying, 'Hey, next I want you to do Tuesday,' it knows. Okay, I'm done with Monday. I'm gonna go to Tuesday like, and now I'm building Tuesday. Tuesday, I'm gonna give you uh, you know, burger and fries, and so like, you know, I'm gonna ask for something, and it could suggest for stuff it knows about, 'Hey, I want to,' you know, schedule an air—schedule a.

20:52B:DoorDash for you on Tuesday to like get I know McDonald's burger size. It's terrible. It's not very nutritional, but like you know, and this is where those sort of kind of—I want to call like magical moments come in, where it's like, oh, that's really cool. I wouldn't have thought about that, right? Whatever. And it's this sort of like more of an assistant thing rather than a chatty agent on the other end, and I think that reformulation is pretty interesting.

21:16A:Yeah, it's tapping into all the state, right? Like if you had a plug-in that kept—now this is itself an OpenAI problem—but if you had like the state of your refrigerator somehow, you know, it could tap into that. You could say, 'You know, make me Thai food,' and it would say, 'Oh, you're based on what you have in your pantry and your fridge. Like here's something you can make right now, or like here's the bare minimum you need to get from the store.' But it's like yeah, adding that state, I think is just like taking something that's already a breakthrough, and I think it's just making like breaking through another barrier. I think we'll

21:52B:see some really cool stuff. But I think we're also going to bump into some limitations where like it's when it's sort of initial, and even today if you try to ask the Google Assistant on Android or you try to ask Siri on iOS, like, or Alexa when you ask it something, it knows about it will give you very incredible results, and you'll think like it's really smart. But then occasionally ask something like just off the beaten path or a phrasing that's equivalent but it doesn't understand that phrasing, and it just—it kills over and just gives you something like horrible. And it's just like when you stay in the lane, it's super cool, but as soon as you get off the edge, the like it's a very steep cliff, and it just degrades. There's no like graceful fallback, and it often misunderstands you, right? Like it thinks you're asking one thing; you're asking actually something very different. And so I think we're going to run into the same one here. I think there's this initial excitement, and I think a lot of folks coming out and saying this is not something that AI researchers sort of predicted, but like the practical application sort of got there a lot faster, and sort of like the question is where does it tap out? Like do we hit a, you know, asymptotic slowdown and sort of like need the next unlocking? And there was some, you know, OpenAI stuff saying they're not going to just grow the network again; they need to go think about architecture and, you know, how to be more efficient. They can't just keep adding more.

23:09A:parameters. Yeah, I mean actually a lot different ways to take that, but one thing I wanted to say really quick because I've seen this come up a lot—I've actually had people ask me this. So people working in AI research for like, you know, FAANG companies or OpenAI, like we're not all part of a CIA operation. Let me just get this out there: I've had people like tell me, 'Oh, this is all run by the government,' or everyone's like a state actor or whatever. And no, it's not. I've not heard this; this is news to me. I didn't know you work for the CIA. I don't know why I keep getting this, but like no, like and so like, you know, I think you're right that people and AI are just as surprised as anybody else that it took off. I mean, you know, it's actually been incremental, you know, on the research side, but it just the way the commercial applicability, like the way that—that just took off, really surprised everyone. I know people at OpenAI, and they're surprised. And so there is no like conspiracy here. Everyone, you know, this is just like one of those when you're playing Civilization and you just get a breakthrough or something randomly. Like it's just one of those things; there's no—there's no secret there. I do feel like, you know, it's going to have a big impact even as is. I think copywriters actually got a spam call which was I'm pretty sure a GPT-based spam call because, you know, it had like my name, and it was like—it felt like a real person, but it was in the uncanny valley. So I knew it was a recording, but it was so contextual that I thought, 'Well, let me just see where this goes.' So I said, 'Yeah, this is me. This is Jason,' and then it went off and like said this whole monologue, and I couldn't interrupt it, and I hung up. But like you know, that first part I think was actually done by like—that's more text-to-speech, I guess, than GPT, but I guess we're just getting to the point.

25:06A:where like I think it's really gonna have all this research you've been doing is really gonna have some effect on the world, which is kind of wild. And like anything it's going to have, you know, a thousand positive effects, 999 negative effects, and hopefully it will be slightly better at the end.

25:21B:Okay, we could dwell on this, but I'm gonna keep us moving because I have a lot of thoughts here. So my next is—I guess I could have made a book, a show, but whatever. Here it's here in the news links, which is 'Build Your Own Database From Scratch.' And this is free online, but they also have a print version that you can purchase by James Smith, who actually has a couple other similar style books of build something from scratch. I will disclaim—I have it; this is why it's in the news section, that the book section. I haven't actually read this book yet, but I did skim through the topics. I have it as like my perpetually growing list of bookmarks, which is most of my Steam backlog. But you know, I'm trying—I want to kind of go through this about, you know, how do you start with key-value store? B-trees, that kind of stuff, move to relational queries, query planning, query execution? What I will say is, you know, I took a database course in college. They were—I feel like it missed a lot because it kind of went into the what is that called sort of relational algebra stuff whatever. That I just really wasn't going to be an academic researcher; it wasn't interesting to me.

26:22A:Yeah, this is a total blind spot for me. I mean, I use databases, but I have no idea how—oh, I'm about to talk about.

26:29B:You hold tight one second, but I was forced to learn SQL as well as part of that, like the practical portion of the course, and then I kind of forgot about it. I did a bunch of embedded work, and it was no nothing. Then when I moved to Silicon Valley, one of my early jobs required me to write a lot of SQL and really learn sort of joins, and I'm by no means an expert. But what I realized after learning it is even if you're not going to use SQL day-to-day—and people who don't—I've begun to realize there's people who are very scared of it or slash don't know it, and so they'll do anything they can to not have to write a SQL query. And then people who know it are just like, 'Oh yeah, I'll turn one of those out for you, no problem.' And there's this wrap—and we've had a series of interviews, really great interviews with people trying to kind of say, 'Stop worrying how this like 10-year ago mentality that if you have a relational database in your stack, you're doomed to not scale.' Like this is not a problem. That it used to be—it's still something to consider, but there's a lot of workarounds for. There's a lot better tech stack now around it, but we sort of a lot of people, not that you've sort of self-incriminated, but kind of like fell into this like, 'I'm not going to write SQL queries,' and I'm going to use this crutch excuse that.

27:40A:Like, well wait, wait. I'm not that—no, I use the SQL query; I just don't actually know a lot about how databases work under the hood. Oh.

27:48B:okay all right well then you're not the one i'm talking about i also don't know how it runs and i'm not a great sequel query writer but i know enough to like get it done and i think there's a lot of unlock there that happens when you sort of know how to do joins you know how to do where clauses and just yes you could do it in python that's why everyone said oh i could just do it in python i could just open this and open that but if you're going to do a bunch of ad hoc queries like many of them and each of them are sort of ephemeral you just want to do them once get your answer and be done and you're never going to run it again you don't really care about it there is a value to having a bunch of data crammed into a database i'm going to refer to this in my tool to show but just you know that style of data available so you can run your analysis and get out not everyone has to do that but it occasionally comes up and sort of just writing a python script takes a lot longer because you can do more it can it's more flexible it's more powerful yes but that's not what you're looking for and so a lot of people are really hesitant i found for whatever reason from traditional cs backgrounds it's like a stigma i think from database administrators from a while ago like no one wanted to get into that job they didn't appreciate that job i have a ton of respect for those people but i think it was a thankless job for a long time i think that's changed and uh anyways there's a well-off topic i think i'm really excited to check this book out you like you said jason i also i know what b trees are i know that they're used in databases yep i get sort of the vague thing but how is indexing done how are relational queries like optimize this is super fascinating to me i would love to learn more uh and it feeds into my like general diatribe about like hey embrace sql stop hating it

29:24A:yeah yeah i actually uh i've been exactly the same position where you know so many people do python or even c or c plus plus it's like uh to to you know analyze some data files and um there's actually an amazing tool called duck db oh almost had that's my tool of the show oh that's literally your tool of the show no no no no no no i almost did oh almost did duck db okay tldr duck db is a way where you can take files on your computer like csvs you know column separated files and all of that and you can write sql queries against them and uh yeah i mean the thing about sql is nothing beats it as well except maybe pearl but nothing beats sql in terms of being short and you know doing what you want yeah i think sql is much more readable than pearl so so i would say sql is really the best if performance isn't you know super critical and you want to you know synthesize some data sql is just absolutely amazing another

30:26B:rant we could get on we will have to come

30:28A:Back to this later. Yeah, totally. Man, I mean we're some of these things like we're recursively generating more episodes. We are RGPT. Yeah, we have PT GPT. So cool. My second one is asking generative art AI to render mathematical theorems. And so what this person did is they suggested they took a bunch of mathematical theorems and they put it into Midjourney, which if you don't know, Midjourney is similar to DALL-E or one of these other ones where you just type in a text prompt and they generate a picture. And so this person put in these math theorems as text prompts, and the pictures they generate are awesome. I mean, they're not—you know, you can't recover the theorem or anything like that, but it's just like these crazy kind of repeating structures. So, for example, he has the Bare Category Theorem, which states in a complete metric space the intersection of countably many dense sets remains dense. And so he puts that into Midjourney, and what you get out of it is this like fractal of new age architecture. It's like these buildings that are kind of embedded inside of a very sort of like 3D cubist kind of environment, and they're just fractally, you know, going off into this like fixed point in the distance. Now let's just say another one I thought was really cool was every vector space has a Hamel basis. That would be really hard because you know even a GPT system probably isn't seen Hamel basis so rarely that it's probably not in there, but what it generated was

32:19A:like a bunch of planets in this like space universe, and the planets are like some of them are just floating, but then some of the planets are trapped inside this cube that it looks like it's being assembled in real time. And you know there's this is all just a static image, but you kind of get this feeling of time. And underneath the universe there's an aurora, and standing on the aurora is like a family looking at the whole thing. It's just amazing. I mean, any of these would be just phenomenal portraits to have in your house or at your dorm room or something. So yeah, check these out. It's definitely pretty cool eye candy.

33:04B:Yeah, I feel like I'm not—I will admit, I'm not a sort of art appreciator historian. Like I go to museums and I enjoy looking at them, but I don't know get into the background explanation like the cause or the feeling of it, but just like going and appreciating like the craft or the novelness of doing these remind me of this procedural generative art stuff, which is pretty cool because they're not generating code to generate the thing; they're just generating it directly. Right? But it definitely picks up that style. There are some weird nuances, like this one about every prime number greater than one or every number greater than one can be represented as a product of prime numbers, and it has a bunch of numbers on the globe, but the numbers are really distorted. It's kind of interesting.

33:51B:interesting piece to look at. But yeah, it's kind of funny how it kind of knows what numbers look like, but it doesn't actually know what valid numbers are. So you get kind of like a rune set. It's kind of—yeah, this one is wild. This is a bit weird, but yeah, definitely cool to look at. I mean, if nothing else, I could totally see me as not an art background person doing something like this and then using it as inspiration to clean it up and do my own thing. It's sort of the—you know, yeah, growing up I had that or whatever. Like when I was a younger programmer, like someone just give me an interesting problem. Like I feel like I have tools but I don't know what problem to solve. Feel like you could go here, type one of these, get like inspiration, and then just go make the art you want to see. Like, oh, this is really cool. I'm gonna go do that. And it's sort of akin but like an on-demand search of a lot of folks start their art by copying existing art and sort of learning what makes it that. And so I feel like this could be the same thing except like you're not copying someone, so you don't run into the same like what's derivative versus like plagiarism.

34:55A:I have a mind-blowing idea. Okay, okay, all right, let's go. So if anyone does this—you know, I don't have to be like get shares of your company, but I at least want to get added on at it on Twitter. So here's my idea. Okay, it's a little bit creepy. It's one of these things. It's like you can't tell if this is creepy or cool. Um, you tell me Patrick: Creepy or Cool? So you get a digital picture frame, like a digital portrait. Right? So imagine like a large canvas portrait, but it's a big screen, right? And so you hang that up on your wall and it's got a microphone, and so it listens to your conversations, right? And then in some privacy-preserving way—hand wave sends a maybe it's running DALL-E locally. I don't know yet, but let's say it takes the conversations you're having in your house and it creates art based on the things you're saying in your own house and it displays the art. So if you're—if you start

35:54B:I know this is gonna go sideways really fast.

36:00A:So if you have an argument, you know even the pictures start getting angry. I think this would be wild. I mean, I think it would take a lot of tuning to know like what's appropriate, what's not appropriate. That's—I

36:14B:was gonna say also like you're having a conversation about someone who's not in the room and then they come into the room and like

36:20A:yeah, like you have a conversation, like, you know so-and-so is so rude. And then like well unless they're famous, you know it's not going to show them, but it's going to show just like a rude person on the wall. Or oh, so you have privacy?

36:33B:preserving microphones, but the cameras that's too creepy. Like you—it's never going to learn who people are. That crosses the line. Oh yeah, so you can listen to what you are

36:39A:but right, right. So it's okay. Oh, that's an interesting idea. I hadn't even thought about the camera angle. Like maybe oh, what about this? I got it. I got it. Okay, so this might be more practical. It has a camera and it draws like a caricature of whoever's sitting on the couch.

36:57B:That could be like a court portrait kind of thing. Like yeah.

36:59A:You know when you go to someone's getting offended really fast? You go to like a tourist destination, and they will draw like your caricature. You know, and it's basically the same three, the same theme. Everyone kind of looks more or less the same.

37:13B:Like giant, I don't know why when I tried this mine just keeps drawing me as a professional bodybuilder. So I—I—oh.

37:22A:Man, I think oh man, this would be really well. I wonder if I have a, you know, I actually have a spare monitor. I wonder if I could mount a spare monitor on the wall in the living room and actually make this—this, I mean.

37:37B:Guess you have to add us on Twitter if you do? Or add us on Twitter is like you add all?

37:44A:Of Twitter. Oh man, yeah, that's—that's awesome. All right. We'll go to Book of the Show. Book of the Show. My Book of the Show actually speaking about Retro is an app. I think it's on iPhone, but it's definitely on Android, called Fighting Fantasy Classics. So for people who don't know, there were a bunch of books called Fighting Fantasy books made by Steve Jackson and Ian Livingstone, and they're basically choose-your-own-adventure books, but along the way you need dice. So they have to bring your own dice, and along the way you occasionally have to roll dice, and certain things happen, and you can actually get to a point where you know you just roll badly, and—and you lose. But there's also an adventure sheet. So the idea is you would xerox the adventure sheet to make copies of it, and you would keep track in the adventure sheet of your equipment and your health and all of that. And so based on the choices you make, like you might make a choice that says, 'You know, lose one stamina,' or roll the dice, and if it's, you know, 10 or less, you lose a stamina. And if you lose—if any of your attributes go to zero, that's it. So if your skill goes to zero, you're kind of just like incapacitated. Your stamina goes to zero, you're dead. I think your luck could go to zero, and then you just lose every luck roll after that. And so, you know, the books were amazing, really great stories. There are about 400 sections. There'd be multiple sections in a page. So, you know, at the top right there's sort of the page count, but instead of going one, two, three, four, you know, like one page might be, you know, one through seven or something because it's seven short things telling you like go to the next section. I love these as a kid. I thought the stories were great, a lot of really interesting themes that I think, you know, they're really exploring. You know, nowadays everything is either, you know, zombies or World War II or, you know, Space Marines, but, you know, in a lot of these Fighting Fantasy books,

39:40A:there are some really exotic, you know, worlds that they were putting together here. You know, some became tropes like space assassin, but then others—I'm trying to remember one of the more esoteric ones. There was oh, there was um one I think it was called Battle with Evil, but Evil was an acronym: E-V-I-L, or it was like Date with Evil? And it was basically this like technocratic future where everyone has crazy augmentations and stuff. So I loved it. And anyways, with this app you can go and buy all these old books, and so I think it's like a dollar or two bucks a book. The app comes with one for free, and I've been really getting into it. It's been a blast. I've been getting my kids into it. You know, depending on the book, you know, some of them might be more kind of like—you might need to have older kids, like there's one that was something from Hell Creature from Hell or something like that, and it's like pretty demonic. And so like I think it would give one of my kids nightmares. So you have to kind of like judge how whether you want to be up all night with your kid the night after. But I've been—I've been a big fan. I've been really, you know, geeky out on that very.

40:48B:cool i've never gotten into these i always wanted to but i always feel like a unresistible urge to just cheat my stats or like to get a role that i wanted and and then just i don't know well

40:59A:This kind of fixes it by adding achievements so you can they actually have three ways of playing. First way, you could just cheat. So basically, you can go to any page you want. You can make every roll to be whatever you want it to be. There's a second mode where you basically have a rewind button and um you have X number of rewinds, and you can get achievements in that mode. And then there's the hardcore mode. The thing about the thing about these books is I do feel like you don't want to get, you know, three hours into the book and that's like, 'Do you go right or left?' I go right, you die,' you know, or something like that. Like you need a rewind button otherwise I just it's just hard to use your time efficiently, right? And you can play with rewind and get achievements, and that kind of keeps you motivated to play. Honestly, semi.

41:48B:Honestly or exploratory, I guess. Yeah, yeah, right. Very cool. My Book of the Show is not related. It is Evolutions and Bread Artisan Pan Bread than Dutch Oven Loves Loaves at Home Nice, which is a mouthful by i Ken Forkish. I've talked about his—I don't know if it's his first book, but another book he has which is Flour Water Salt Yeast. Um I—I don't know for whatever reason I got it in my head that like I wanted to do like cook some bread pastry something like once a week just to like develop improvement at it because I was always like kind of messing around with it and it just never kind of went anywhere because I didn't do it often enough. I always forgot what I knew before, and people like, 'I'll keep a journal.' I'm terrible at doing that, like I don't know. So my solution was I just keep doing it and you know making sure that you know I do it often enough that I sort of keep it in.

42:39A:My—I was gonna say how often do you do?

42:42B:This like now I'm trying to do. I've been trying to do some sort of like well yeast risen thing so either a bread or like a pastry or something once a week. So oh nice. Um just probably very terrible for my—uh, like you know, distribution of nutritional value eating a lot of carbohydrates. Um but but it is what it is. Um but I attempt various things. The original Flour Water Salt Yeast and like the sort of movement in sourdough had always been to do these like free form, I guess they call—I don't know how to say that—we're boules B-O-U-L-E, like these. Oh yeah, where it's like just like a dome circle, right? Or maybe um like slightly an oval. And this was like you know you kind of put them in the pan, but you don't give them any support. They just sit on like a hot stone or in a Dutch Dutch oven which is like a basically like a cast iron pot with a lid to—to kind of keep in the steam. Um but this is actually very difficult because it just if you don't do a good job, it just like spreads out and becomes more like a pancake than a beautiful like tall loaf of bread. And so anyways, this new book Evolution of Bread is sort of talking about Ken sort of points out he's sort of like evolving his how he's thinking about this easiness and who does he need to prove this to? Like what is—what is the point of doing that? It is like a pinnacle if you're trying to show your like artisanal abilities and that you are very competent and able to handle everything that's needed to go exactly right to generate one of these loaves. It is an achievement, but if you're ultimately just trying to like have a tasty loaf of bread, if you put it in a pan, you get support. So the window of like technique that allows you to get into a nice edible loaf of bread when you're sticking it in a loaf pan is actually like it's a much bigger target that you're aiming for than what's required to do one of these sort of like free form unbounded loaves. So long-winded way to.

44:38B:Say like if you're at all like thinking about making bread and you sort of the whole pandemic thing was like everyone make a sourdough starter because you couldn't buy yeast and like make these beautiful loaves, and people boasting, 'This is my first time I ever made bread,' and it's just—yeah, that wasn't me. And I actually even though I was doing it routinely, I still fail quite often. And it was somewhat.

44:57A:Frustrating wait. So when you fail, is it like you said, like a pancake? Is that the most common way that fails? So I mean, it can fail for...

45:05B:A lot of reasons. If you're trying to do, like, a self-cultured sourdough, there are a lot more ways for it to fail because you don't know. I wasn't getting familiar with my sourdough culture. You don't add commercial yeast. Commercial yeast is very standardized. So if you know the temperature of your house, you pretty much know when to expect the bread to be ready to bake and how much gas the yeast has made is a very important factor to success. It's a very narrow window from underproofed to overproof. If you miss the window entirely, you end up with something that doesn't really look very good. And if you end up on either range, it's just not perfect. Getting it exactly right is how you sort of get the perfect one. But with the sourdough culture, depending on when you most recently fed it, what the distribution of bacteria to yeast is in the culture, like exactly how you did it, there are many more variables. And so yeah, when it fails, you typically, in my experience, you either end up with a sloppy mess that just doesn't really rise and isn't really bread, or you end up—it's sort of right but it expands and so instead of staying tight and getting tall, or you get like a fluffy loaf, it sort of collapses down and you get a denser, like, you know, very flat loaf.

46:20A:What do you do when you mess up? Do you feed that to the ducks? Like, is it safe? Do you still eat it? It's totally...

46:26B:Safe to eat. There's nothing wrong in it. Okay, it's completely edible. In fact, actually, like right out of the oven, if you've never had it before, even like the mistakes are still very good. Oh, okay, they're warm and it has the beautiful bread smell. I mean, if you burn it, sure, like, right? It's kind of ruined. But if you just, like, I still cook them even when I know that they've gone wrong, and I still eat them, and my family will eat them as well. But after sort of the couple-hour window after making it, they typically—it's not something you would want to eat anymore. So, like, you said we could—we'll toast it up for croutons, or you know, you can kind of do, like, you said, you can feed it to the ducks. You don't need to throw it away. Yeah, that is awesome. The book advises potentially considering what I have tried as well. If I know it's a failure now, I'll sort of scrape it and put it in a pan and bake it. And the chances when it goes into a loaf pan, it's got support so it'll rise, and it tends to be closer or more useful than not—not as good as like a properly done one. Into a pan, but like a lot better than the alternative. Yeah, makes sense. If so, like if...

47:31A:You were to go to the grocery store and get a loaf of bread where they all kind of look the same? They definitely use pans, right? Because they look so...

47:39B:Uniform. They sometimes will sell like these sort of round ones. But yeah, if you get, like, to the bread aisle—not to a bakery—yeah, those are all in pans. Yeah, got it. Okay, that makes...

47:50A:Sense. Cool man. Um so if you are interested in making bread, check out our link on the show notes. If you use our link to buy any of these books on any of the episodes, you know that helps out the show. Allows us to reach more people. Um if you whether you do that or not, if you also want to support the show, you can follow us on Patreon. So that's patreon.com/programmingthrowdown. We really appreciate all the patrons out there who, you know, kind of continue to support the show, help out with all the hosting costs, help us reach new people and all that good stuff. And with that, it is time for tool of the show. My tool, the show is Ginger. Have you ever used Ginger? Patrick once—once. Okay, enough to...

48:37A:Know what it is? Cool. Yeah, Ginger is really interesting. So the way that a lot of people know about, you know, templating is through PHP. So, you know, people have done PHP where you're writing some HTML and then you maybe want a button that is red if the person's credit card is expired and it's gray if the credit card isn't expired. And so you're, you know, going through writing and then all of a sudden you do like—I forgot what it is in PHP. I think it's like open bracket question mark PHP, you know? And you can write some if statements and then you can emit, you know, different, you know, colors depending on some logic in your program. But then all of that is in your HTML code. And so, you know, you're now your HTML file or maybe your .php file, you know, has like a...

49:27A:Uh, you know, a bunch of things you want to output—a bunch of text or content you want to output—but then also has some logic. And that logic is sort of describing a whole bunch of different outcomes, like it's your own adventure book. And so then what happens is you pass it through, you know, some type of templating engine. And so it will, you know, replace all of that logic with, you know, whatever answer it comes to. And so you end up with just a pure HTML file at the end. So it turns out you could do this for anything. Like you can have—there's no reason why this only has to be a PHP thing or HTML thing. You can just anytime you're emitting any text, you can emit like you could create something that has, uh, you know, logic built into the actual text and then you just have to wrap it in in this sort of templating command which will, you know, you give that command the text which has the logic in it, and then you give it the context. So, you know, the variables that you know you expect it to be using. And then what it will return is, you know, the final text with all the templates resolved. And so there's actually a lot of places where, um, you know, either you're sending text to somebody, you're rendering text on a website, you know, even if you're running like—I'm building a command line app at the moment—and so the command line app is, you know, writing a lot of text to the console. You know, you could put all of this logic in Python or C++ or whatever language, but it starts to become really cumbersome because now you have sort of this disconnect where you have, you know, a lot of your content sitting in one file, but then you have all this, you know, rules to manipulate it in the other—in somewhere else. And you know, templating engines kind of solve that in a really elegant way. I was looking at, you know, I needed to do something like this pretty extensively. You know, I looked at a lot of different options and I...

51:22A:Found Jinja to be really quite nice. I used a combination of Jinja and afterwards I did a regex kind of a post-processing step, and between the two of them it actually produced like a pretty beautiful kind of environment to make content. So um so yeah, if you've never used a templating engine, if you don't know what that is, you know take some time to go through the Jinja docs. Understand like what it does. There's a really great documentation that has a lot of examples and it might be sort of like a whole your craft, like there's actually I think a place for for these kind of things in most apps. So...

52:00B:I think as you mentioned, it can be really useful to separate out. People have that—hey, I want to make a report, maybe it's as an HTML page or a LaTeX, and you're going to make, you know, compile it to something else, or or whatever. That you know, hey, I'm going to just emit the like Markdown from my, you know, let's just say C++. I'm going to standard C out. You know, all the output are standard to the file stream, you know, the stream out stream operator out. But then you end up with this like nasty coupling, like your code doesn't really want the data to live in it like that. It's just—it's sort of ugly. It doesn't work that well. So what I've been able to do, yeah, like, like you're mentioning is sort of say, hey, I want to make a weekly report and I keep just writing the same thing over and over again. So I want something that says, you know, put the date here and I'm going to make an image and I just want the file path where I make the image to go here. And then you run it through and it—it's a time saving and it separates it out. And depending on how much of that style task you do, it could be a huge...

52:59A:Game changer. Yeah, totally. I wonder if there's some type of plugin for Google Docs where you can write kind of like macros that you know have Jinja templates in them? There's got to be.

53:10B:Something like that. I'm sure I feel like I've never used Customer Relationship Management (CRM) tooling, but I feel like a lot of CRM tooling also supports these kinds of templating because that's how you get those like personalized spam emails, right? Like, 'Hey, it's been 73 days since we last contacted you and you bought this thing, and your name is [Name],' and like it's like... I—this is that's how those are all done.

53:32A:Assume? Yep, yep. That makes sense. Have you ever gotten—have you ever seen something where there was an error in the CRM tool and you got someone else's stuff?

53:43B:I've seen ones where they didn't like replace out whatever macro.

53:47A:Yeah, that's a good one. You bought on [Date], and it's like, 'Oh, you bought on [Date].' No, there was an issue where Uber was trying to solicit former Googlers, and they were sending kind of blanket emails to basically anybody they knew who worked at Google in the past, but there was an off-by-one error. So I got an email, and it was like, 'Hello, Jonah Grossberg.' You know... Like, well, you should come to Uber. Like we foreex last year or whatever. I was part of this like ex-Googlers kind of like LinkedIn group. I think I'm still in there somewhere. And yeah, someone posted and they're like, 'Hey, you know, I got an email from Uber for someone else's name.' And yeah, it turns out they sent like thousands of those. It's like... Yeah, I don't know what the

54:35B:line is between the like mass marketing emails, the like CRM individuals, and like these kind of middle ground, like blanket things. I—I don't that ecosystem. I understand what the tools are and where they live kind of thing, but I don't actually know. I'm not very familiar with the space.

54:51A:Think that there's a huge opportunity here for... I mean, think about like all the communication that you do that is very regular. I think that you almost wonder if we need like a CRM for, you know, normal people? Like something that doesn't require so much ramp.

55:06B:Up. They have one. It's called Julia CRM. Oh yeah, no, that's not it. There is one that's like a CRM for your home life, right? Right. I always liked—I have it on my list somewhere. Oh, no, this okay. We'll have

55:23A:to look at it. But yeah, I think that there is an opportunity there. But yeah, that's my tool. This show if you don't—if you've never used templating in your stuff, definitely check this out, learn about it. It's a good skill.

55:35B:To have. Oh, Monica. Ah, that's it.

55:37A:All right. Good.

55:38B:I found it now. I don't feel so bad. Monica Open Source Personal CRM helps organize your social interactions with your loved ones. Wow, interesting. So, I think yeah, you give it your contacts. You sort of say when you've talked to people, how often you'd want to talk to them—that kind of stuff. Wow, this is fascinating. Okay, all right. I don't have a review for it, so maybe next time we can make it our tool of the

56:01A:show. Yeah, we'll have to love to do a bit of research on this.

56:04B:All right, my tool of the show is RealR-I-L-L, which is a little difficult to Google, so it's realdata.com/rill.data.com. And its sort of tagline is 'Radically Simple Metrics Dashboards,' but it feeds right into what we were talking about. I believe actually a default like back end it wants to use is DuckDB, which is why I was like, laughing, oh, interesting. It's a local dashboard you can run. You can sort of start it up, you can shoot it on a notebook. You run the tool, it opens a website, and then you can sort of add in a CSV file, you can point it at a SQLite database, you can point it at a PostgreSQL database if you have one. And what it does is it kind of does those things you automatically want to do. Like if you had a CSV file that was like for me, it's always like, 'Oh, how many of this type of thing? How many of that type of thing?' Where I have some string that is an enum or whatever, and it'll give you the sort of like breakdown, like this percentage. That percentage lets you sort of explore your data. You if you have timestamps, it sort of understands timestamps and can show you things over time and bucket it. You can also give it models, which is like how to extract the kind of thing in a SQL query for what you want. You can have dashboards where you sort of write SQL queries against it, and it's just a really quick way to just like point it at a CSV and sort of have it pretend or a set of CSVs and how to sort of join across them and have it presented without actually needing to sort of insert it into a database. Although you can, and I think DuckDB is different than like SQLite. SQLite sort of focused on the transactional, right? You know, like single-user thing versus I think DuckDB's focus, which it's more on this sort of like streaming log of data just dumping out and you're sort of aggregating and doing analysis. And there's sort of this distinction arriving between the two approaches, and there's a lot of trade-offs here. Yeah.

57:54A:Go ahead. I think DuckDB is read-only, and so because of that they could do a whole bunch of optimizations.

58:01B:Ah, okay. Yeah, so it's sort of like you're not meant to be doing the like CRUD stuff, right? And right, you know, just sort of like going in and manipulating single. It's just sort of like you just keep appending to it, and it's keeping in large, but I think that's an entirely sort of like powerful thing to do. And like we're mentioning not being afraid of SQL. I think putting a bunch of stuff in a SQL dashboard and allowing you to write queries on it rather than I find myself often trying to do things in Excel with CSVs, and not very powerful Excel either, and it's not really the right tool for the job, right? It's like a set of tools in the tool belt that exists in this space that I know I'm

58:39A:Personally weak on. This is amazing. What is the dashboarding like? You know, you can create like time series and pie charts and all of that. Is it pretty good?

58:47B:Yeah. I mean if you sort of look, it's going to be hard to do over an audio podcast, right? If you sort of look at their in the documents and sort of look at the developer documents, they have some pictures of the kinds of dashboards they do. I don't think it's as big as you might get from something like a, you know, production dashboarding company, right, like Tableau or something, but I think it's in the same, but yeah, exactly. Yeah.

59:11A:This is really, really cool. Yeah, folks should definitely check this out. This is awesome. Oh yeah, I see that backed by DuckDB and Druid. Yeah, very cool. Yeah, this is a winner for sure. I'm actually going to try this today. I scored one.

59:27B:Yes, one. Jason hasn't tried yet, and it wasn't a video game.

59:30A:I wonder like how. Yeah, gosh, so many tools of the show have become just permanent parts of my workflow now. I think I feel like that's for us. You know, for us, I think that's the personally like the part of the content of the show that I get that I get the most out of. You have some amazing tools over the past like 10 years.

59:49B:Or so that's what you got to get AutoGPT to do is like get a task list which is like go through and extract just the 'Tools of the Show,' just the book of the show, just whatever into like mini podcast channels. Yeah, that's.

59:59A:So true. Yeah, we should have or or just I was I thought you were gonna say a Google Sheet, like it'd be awesome if GPT could scrape programmingthrowdown.com and put all the 'Tools of the Show.' If you do this, no, no, but people want to listen to our rendition about why it's such an amazing. Oh yeah, even better would be if it put the MP3 like timestamp or something like some deep link into the MP3. I don't know if that's even a thing. Okay, all right. Let's jump into position localization. Yeah, take it away. Yeah, so you know.

1:00:32B:I'll kick this off by saying that neither Jason nor Ives is a disclaimer we always give or mostly give. Neither of us are experts in this, but I think we've both been around it a little bit and familiar. So this one a bit like set up the problem and then introduce some of the kind of like terms and kinds of things you'll hear. I apologize in advance. You may want to tune out if this is like the thing you do all the time. You may find some of this events it's the same thing we give for some of our more impassioned Lisp programming languages that we talk about, but one of the things that isn't always obvious to people is how do you just like figure out where you are? Like we have this, you know, as humans, I guess kind of understanding of where you are. When we start talking about a phone—let's just talk about a phone—like how does the phone know where it is? And I think people probably mostly but we'll go over it. Say, oh, it's just using GPS, right? So we have these satellites up in orbit. Um roughly the satellites know where they are very precisely because it turns out like orbits are something that math can predict really well when there's not, you know, air friction, and we sort of know how they change over time and where they are. So you can kind of download basically a description of all the GPS satellites and like what their orbital parameters are and sort of understand kind of what you would expect these things are somewhat knowable. And without getting into the details of how a GPS receiver works, it basically listens to the radio signals coming down from all the GPS, and by knowing the figuring out the exact time it is with atomic clocks and knowing this orbital data correcting for some things and listening to a bunch, you you have an understanding of how far your phone is from each of the satellites. And knowing how far your phone is though the GPS receiver is the antenna is from each of the satellites, you can run the solution to figure out where you think you are. And the more satellites

1:02:28B:You have the more refined you can get, and this normally gets you—you know—and if you're in the middle of a, you know, open prairie, yes, is going to get you down to a pretty good like couple meters, you know, a few yards kind of thing without a lot of things. And there are a lot of improvements you do on getting that refined and fancier and fancier GPS. There's noise in the upper atmosphere, and you know, you can use base stations, known positions to offset this. It is a whole thing we're not going to get into how like GPS position—position works. Those positions only really come in sort of like, you know, normally kind of get only once a second or maybe 10 times a second, which you say, 'Oh, that's pretty fast.' And you're right for like a human walking around. Like, you know, that's probably, you know, generally knowing where you are if you have—you know, if you had lost your phone and you knew where your phone was within a couple meters, and you know it only updated every second, you'd be able to find it. Yeah. And so this is the most common common one. The first problem you run into with that is okay, but what if I'm inside? Turns out like the things that make our shelters are, you know, buildings, our houses, our malls, or even like downtown in the city, you start to run into a lot of complications. Either the GPS signal is bouncing around, and it's—it's called multi-path. It's very confusing. Or if you're inside, you know, a bomb shelter, let's say, you just can't your phone can't hear the radio signals. They can't receive; there's not enough radio energy making it through. And so, you know, then what do you do?

1:03:59A:I have kind of a maybe a dumb question on the part you already talked about. So like, GPS—so there's satellites that are going in orbit around the Earth, or maybe they're not? They're in some kind of stationary?

1:04:10B:No, they're actually very low orbit, so they're actually going pretty fast across the sky.

1:04:14A:Okay. So there's GPS satellites going fast across the sky. What does your phone do to talk to? Like are these satellites just constantly just sending out like spheres of energy and your phone's picking up on them? Is that how it works?

1:04:27B:Yes. Yeah. Okay. So they're all broadcasting a set message basically at a specific rate, and as you pointed out from radiating out from the antenna—an antenna has like a specific kind of pattern for strength, but basically travels at roughly the speed of light in a bubble. Got it? So if you're and there's no two-way communication, I guess that isn't obvious, but your phone doesn't talk to the GPS satellites. GPS's are broadcast only, or at least for the intensive purposes of this conversation, right? Only broadcasting out. And they're just broadcasting out there basically it's like their metadata with the time code and this kind of stuff, and they're just broad broadcasting out this kind of thing. It's a very, very specific pattern with some specific nuance in it that allows you to do a precise timing of understanding the two things: when the satellite believes it sent the message and when you heard the message. Got it? So if you know that, then you know the distance to the satellite, and if you know the satellite's orbit, then you know sort of where in the Earth frame like the satellite was at that time, and you're trying to work backwards to where you are based on it. But giving you one—if you know, let's just say you assume you're on the ground, which isn't a good assumption, and that the Earth is the complex thing to describe, but basically the Earth is roughly a ball. I'll just say that. The Earth is a ball, and you assume you're on the surface. You kind of project out that sphere, so then you would kind of know where you are in a roughly circular shape, right? Because a circle is the shape that you get when you intersect—oh, right—radiating ball of a GPS satellite and the ball of the Earth. Um I'm hand waving a bunch there. If you had another one and you knew another distance and you intersected all three: the ball of the Earth, the ball of satellite one, the ball of satellite two, you would sort of know where you are instead of on a circle—the sort of two points on the circle where is the intersection of all?

1:06:24B:three? So if you imagine taking a circle and intersecting it with a sphere, right? Because you know the distance, you know sort of like what those two points could

1:06:33A:be. Yeah. So I think it's like every time you intersect you eliminate a dimension, right? So if you have two three-dimensional things you intersect, you get a two-dimensional object in this case—a circle. And then when you intersect the circle with something else, you get a one-dimensional object which is these two points on a line. And then if you could do a third one, I guess at that point you would get a single point or a fourth one whatever it is. Yeah.

1:06:57B:Yeah, no, no, you're right, except that we have to work backwards a little because that first assumption that you're on the ground and that the Earth like that and that the Earth's sort of surface is flat—is actually horribly untrue, right? And so like it could be in a plane. You could be on a mountain, right? Like any of these things. And so you actually want one additional one and so to eliminate so that you don't have to start with the assumption of that you're on the center on the surface of a, you know, ball. And so oh, I

1:07:25A:see. I see. Yeah, you're right. So if you don't—if you didn't make a ball out of the Earth, like if you left that one out, then then you could you make up for it just by having more satellites. You could

1:07:35B:do the same thing. So you start with one satellite, you know you're on a sphere, you know. Then you say the second one. Now you know you're on. Yeah? So it's just you start one back. Got it? Right. Cool. And so this works incredibly well. There's a ton of nuance about refining the—so you're eliminating the dimensions as sort of first step, and as you get more and more satellites, you sort of reduce your ambiguity. But only to a point because then there's sort of issues with listening to these signals and understanding like how precise can you measure that distance? So you're not actually getting a point; you're getting sort of an uncertainty. And the uncertainty is depending on like for instance if the satellite is down on the horizon, it's—it's traveling through more atmosphere, and it's more ambiguous versus if it's straight overhead, that's really good for you. And just like a variety of other factors that give you a sort of blob of uncertainty that's shaped in a certain way, and you can do various techniques to kind of like squish it tighter and tighter so that you know better where you are. And none of that works as soon as you can't see the satellites. So before we go into size, I guess while we're on that topic, we should go to one other thing people kind of know about which works roughly in the same way but it'll also work for indoors, which is what I described works great except if you've ever used an old satellite receiver or today if you know you use like something that doesn't have a cell phone antenna or Wi-Fi antenna in it for this purpose. Um so like I have a little drone, like a DJI drone. You take it outside and it takes a while to acquire satellites. It sort of says 'Acquiring satellites,' 'Acquiring satellites.' Right? Right. I had this old Garmin that did that. Yeah, exactly. And the reason why is it has to kind of receive these messages, do a bunch of disambiguation, and it takes a while to sort of update itself to learn the new corrections to all the orbital mechanics to kind of all this stuff has to take place, and it takes a

1:09:23A:while. And as you said earlier, if you're inside, there's—Yep, it will.

1:09:26B:just never finish because it doesn't ever get a radio signal that it's just sitting there listening for. And so there are all these techniques for helping figure out where you are a lot quicker and then you know what to listen for. You're resolving ambiguity. So one is, if you have a cell phone antenna, you can do the same thing with cell phone towers. So we cell phone towers are very precisely location known; they don't move around a lot, right? Because installed on a giant tower, right? And so you can use cell phone antenna location and relative strengths to do effectively the same thing, right? It is very similar, but you're using instead of just the message and the timing, you're using signal strength to estimate how far away you are. Um because this, the signal attenuates much quicker. It doesn't—it's not broadcasting from space, right? And then when you move inside, it turns out you can also do the same thing, but cell phone towers a little harder; they're not really inside a lot more noise. But it turns out we installed a lot of mini cell phone tower equivalents, which are Wi-Fi routers. And so a long time ago people learned, oh okay, we can go around and most people don't move their Wi-Fi antennas around. So if you sort of understand where all the Wi-Fi—so you've there was I'm not going to sort of name companies, but several some court cases, some other stuff where it turned out people were recording what network name and router MAC address and stuff were coming because they were trying to understand. They were taking this survey; they knew where they were. They could sort of estimate where the Wi-Fi router was by looking at it over time. And then if someone else saw it later, you would basically have some database which shows you this special number, router with this name is at this location. And sometimes it would be wrong because a person could have moved it in their house or turned it off. But if you have enough of them all over the place, this becomes helpful. So this along with some other stuff allows you to more or less do the same kind of thing but inside as well outside and sort of an additional input. Oh, that makes sense.

1:11:20A:So it's kind of like almost like a Google Maps type thing where somebody walks through a mall and while they're walking through a mall, they're recording the signal strengths. And because they're walking like a certain pattern, they know exactly where they are. So so they're like they're giving ground truth. They're saying like, okay, these signal strengths mean I'm in Macy's. And so then when you go to Macy's, you know you're not telling Google you're at Macy's or whoever. You're not telling that company you're at Macy's, but because you have the same signal strength as this other person, you know more or less that's how it's working.

1:11:57B:Yep. And I mean just in general too naive way would be like if you know where a Wi-Fi router vaguely is and you can see the Wi-Fi router, you kind of know where you are in sort of absolute terms, you know approximately, right? Not close within a few hundred meters, you know sort of where you are. Because if you weren't there, you couldn't have heard that—that you know Wi-Fi router. If you assume Wi-Fi routers don't move hand wave that a bit. So these work generally pretty good for what I would say is sort of like sort of these slow update rates. But now if we start talking about hey, you're going for a run or you're you know in a vehicle in an airplane or in a car or on your scooter or your personal transit of choice—rollerblades—and you're trying to do more than just understand where you are, but you want to know how you're positioned. So like if you're on a watch and you're trying to measure like someone's steps, you're trying to say like how is the arm position, you know, changing over time? And then this is where you start to add other sensors—sensors that don't give you your absolute position. They give you a relative position. And the relative position is just at each time stamp how you've moved, basically how you're oriented or how you're accelerating. And the things that do that we call those Inertial Measurement Units (IMUs). And you have little gyroscopes and magnetometers and accelerometers, and these tiny tiny little chips that fascinating video actually. You can go look it up. I think it's by if I recall breaking taps. I've not talked about this before. He does this whole thing about MEMS—these micro-electronic machines. Oh yeah, I've seen. I don't know if I've seen that one.

1:13:32A:But it's amazing.

1:13:33B:Yeah. It's like how they move relative to each other. They're actually these tiny vibrating pieces of metal inside of these that are etched out or silicon, I guess. They're etched out and doped in a certain way, and how they move relative to each other allows you to take these measurements with it's not no moving parts, but not moving parts as we traditionally think about them, like things sliding over each other. There's just these little tuning forks basically that are vibrating around in your in your watch. Yeah, it's amazing. Those those can run much faster, you know, hundreds of Hertz, hundred Hertz. Like this is really easy. So you get a faster update rate. Now you can kind of see where we might be going. If you want a very precise positioning, you're mixing the two, right? So you know where you are kind of globally, you know how you're oriented also now know how you're changing over time. So you can say between two positions from GPS, how did I move? How did my—you know these recordings how did they move? And if you do integration, but integration here just means adding up the sort of how much acceleration you had over how much time, tells you, you kind of double interpolate it. You add it twice. You accumulate these and what you're getting is you sort of take one position and then you add it up and then as you get the next one, you sort of know it again. And what you get is in between how you were moving. So instead of just interpolating between the two and saying I went on a straight line, you can actually say no, I went on a bit of an arc or I went on a bit of a swerve or a dip, or I went really fast and then I'm starting to slow down. And these other things begin to combine, and this is where you start to get into needing a little bit of mathematical help. So I've hand weighed over the math of getting all of those, but now we can sort of set up the problem where you end up working a lot of these kind of positioning problems that one—these all have noise. So it's not that GPS gives you a position with some error, this uncertainty. It also moves around over time. So if you sat in one position, you'll sort of notice your dot kind of wiggles and a lot of stuff tries to combat it. Now, but like your actual position being reported is not stationary even though you might be stationary.

1:15:33B:So let's say you have an IMU and you're standing still waiting, you know, for whatever. You're outside. You're just standing still as a test. Um your IMU says hey, I'm he's not moving; she's not moving like we're just still. But the GPS is moving us like one meter right and one meter left. You can go—well, that's not really true. Like yeah, I have counter evidence to this. And so you need a mathematical framework that's going to let you handle the fact that some of this stuff is a little contradictory. As well as we talked about doing this integration, but if I integrate my accelerometer, my IMU readings over time to understand my position and then I get a new GPS reading that's different from that, how do I handle? I clearly didn't instantaneously move from the result of my estimating using my accelerometer to the new GPS position which we know has noise. So how do I smooth all of these together and benefit from the sort of pros and cons of each of these? And that's where you start to enter a lot of the mathematical framework of this part of the problem. Yeah, that.

1:16:33A:Makes sense. I mean, you're getting samples over time and so the error is going to be different each time, right? Because there's it's drawn from some distribution. And then you're right, you have all this sort of contradictory things going on where nothing nothing's going to be exactly true. Hopefully that they line up, but but yeah, you have to have some way of resolving there isn't like a—it's not like you just overwrite one with the other. They're both sort of contributing like some amount of evidence, that's right.

1:17:04B:And so the one and I'll just there's several here in variations, but the one that is like the topic everyone has probably heard is Kalman Filter. So when you hear Kalman Filter, most people have heard of this. It's useful for a variety of other things as well. I won't get into because mainly because I'll say it wrong, like what exactly is trying to do. But this is where that tool comes into play. And the tool comes into play in understanding—I'm going to not use the mathematical words because I am groaning already at people, I know who will be upset. So if you sort of say hey, I have a certain kind of distribution that my GPS error gives me the variance of that signal. If you have an understanding of that and you have an understanding of the kind of noise in your IMU readings, the Kalman Filter is a structure, a framework to allow you to take those known things in advance, add them to the construction of the filter, and then you have a sort of I got a new measurement. I have an update. I'm making a prediction about where I'll be next time based on that. I get a new update. My prediction was wrong. How do I update my internal state so that I'm combining between the two? So I have an error number coming in, which is real—a number coming in which is noisy. I trust it a little, okay? How do I update where I think my position is using that, you know, that number and how much is that different from where I thought I was going to be? Refine my internal state a bit so that over time I'm sort of adapting to this. And you mentioned earlier sort of ground truth—the reality is you don't actually ever know the right answer unless you sort of after the fact take a survey or do some very complicated something. You're not really going to know, and it's also not really that important exactly where you were. So your goal isn't to get better. Your goal is to say hey, I'm going to try to do the best I can with the information I have. And Kalman Filter and its various forms allow you to sort of do that, allow you to say how do I do this?

1:19:03B:Predict, update, measure, understand the various performance of my sensors, put them all in together, and sort of not just do a hacky, which is what I would probably have done, which is like take my GPS number, do something, take my IMU number, do something. It's just one sort of call into a class that handles all of it, right? You know, you're telling it in advance what it is, and it's able to kind of take care.

1:19:26A:Of this. Yeah, totally. I mean, I think the thing that's nice about Kalman Filters as opposed to us trying to like hand code something or using a neural network or something like that—although there are now like Deep EKFs and there is that's a whole separate topic—but yeah, um but you know Kalman Filters are, you know, they work off of the covariance. So you know if two approaches are always wrong in the same way, then you won't end up sort of overcompensating. Like let's say your let's say your GPS and your accelerometer they're always—I'm making this up, but they're always sort of biased to the east. Like when one has an error that's too far to the east, the other one also has an error too far to the east. So like you have to think about all the pairwise or co-variances. So you have to think about, you know, if you have 10 different sensors, you have um what like 10 times nine different, you know, kind of relations there that you have to like keep keep track of. You know, the Kalman Filter, you know, works off of those, you know, co-variances that it accumulates over getting a lot of samples and so it's going to take care of that for you. And so it's extremely powerful tool. Do we ever do a show on KF? I don't think we did. I don't know. I don't feel like it.

1:20:47B:Always one of those things that's always like just past my like feeling confident to talk about or is.

1:20:52A:You know we should do. I know we both know who I'm talking about. There is a person who we should try and get on the show who we know who is amazing at this stuff. I'll give the really TLDR, but we'll we should do a show on this. But there's there's something called a Gaussian Process, and the idea is you have a bunch of inputs and you assume that they're normally distributed or Gaussian distributed. And so that just means if there's going to be an error, it could just as easily be smaller or larger. That's the real hand wavy way of saying normally distributed. And so you know if all of your inputs are normally distributed and you have a process and the outputs of those process are like more linear distributed numbers, normally distributed numbers, then a Gaussian Process will go through and like layer a bunch of functions to figure out like what composition of functions will get you from the inputs to the outputs. Um and so a Kalman Filter is like a specific type of Gaussian Process where I believe the function is quadratic. I think you're making some assumptions there. It's basically it makes some assumptions that take a really really computationally difficult thing and make it so that you can run it, you know, on a watch. Um That's basically they figured out like what are the assumptions that I need to make to run this thing, you know, in reasonable time. So like if you have a ton of compute and not a lot of data, you know, a generic Gaussian Process is really the best thing you can do because it will like it will explore all the possible functions and compositions of functions, but that's just not practical to to do on, you know, on a reasonable device. And also like it it you can get really odd things that you're not expecting. And so a Kalman Filter allows you to set really level expectations and on.

1:22:50A:Good compute. Well, we should do a whole show on it, but it's an amazing achievement.

1:22:55B:So I basically have one more sort of like category of things to talk about, just in this introduction, sort of like drop a bunch of terms and then run out the door. And that is like when we talk about these things as well—we've been talking about right now—I'll describe as, and this means a lot of different things to a lot of different people, but real-time, right? Like I'm trying to decide right now where I am. And as Jason mentioned, using computationally efficient mechanisms because you want to do this potentially on something that doesn't have infinite power. And so you're trying to figure out where you are right now. You don't know where you're going to be a minute from now or a second from now. You don't know any of that information. And that's pretty different from if you say, hey, I have a recording of all this stuff. Like I went for a run and I get back. This is not how they do it. I'm just saying they could, but if you go out for a run, you collect all your sense of information. And for instance, you know you started and ended at your house. Well, then I know, for instance, I can put a constraint that at the beginning and the end the two positions must be equal to each other because I went through my front door. And so if you snap those two together, then you can backwards like make offsets along the whole thing so that you sort of get a reasonable answer out. And you kind of call this like a batch process. You batch all the data and you process it at once. And at any point in time, you can look ahead and you can look back for certain milestones or things you're looking for. But what if you need to do something similar? Like what if you wanted to impose constraints but while you're going in real-time? Like hey, I'm running past a fountain and I ran past the fountain again. Like I knew that at the time. Like I don't need to wait till later to know that. And there are some techniques that use that because one thing we've not talked about is if you sort of watch any of the companies trying to do sort of robotic vehicles. If you look at little robots that move around interior spaces, they have need to kind of do something in addition to just figuring out

1:24:53B:where they are. They also need to figure out how to move around the space and how to understand that there's—you know, hey, I'm a little tank-treaded robot that's going to serve Jason a very refreshing milkshake when he starts to get hungry because his AutoGPT asked it to. And so like the tank is in the other room, but it needs to know he left his sweater, you know, in a pile on the ground to avoid it, right? So it needs to understand where it is, understand the world around it, how to maneuver. And it does this via a process called Simultaneous Localization and Mapping (SLAM). You'll hear this referred to, and it uses some other things, but they all kind of fall in line with what we've already been talking about. So it can shoot beams of light out and measure distance using beam pulses of light to a wall or to us—you know, to the ground. It could use the same thing with ultrasonic sensors like a, you know, kind of bat chirp. It chirps, and then here's the response back, and it's getting all of these distances to objects, to the wall. It can use its camera to understand things, and all of these become what I said—these sort of constraints. So for instance, if it goes in a circle and comes back and sees the same spot it's been before, it knows those two things must be like equal to each other, right? In space, even if there had been some drift and it didn't initially think they were. And so you get these events that occur. You input all these different measurements, and you're at the same time building an understanding of the space where you are and where you are in that space. There's some Udacity courses that go over this, and like some of their autonomous driving—they have a class about it. I'm pretty sure they are free or at least were free at some time because I watched them. I think they're free. Yeah, okay, cool. They have a whole discussion about a lot of these topics. So if you want to go like next level deep, they have stuff where they talk about Kalman filters, they talk about SLAM. They have actually—I've seen it some data where they give you input and tell you you're a little robot, and they talk about things we didn't talk about here, like particle filters, how to sort of understand

1:26:48B:multiple hypotheses about where you are. And as you're getting new input, how do you build a framework where you're incorporating it? Definitely would recommend checking that out. And it's pretty cool because they have some data you actually can try to do this for yourself in a simplistic thing without having to integrate a radar onto your milkshake-delivering AutoGPT. Yeah.

1:27:07A:No, SLAM is amazing again. It could be a whole show, but you know it's just fascinating. I think the way to think about it is you send out a little ray out of your camera or it's passive so it doesn't—not literally this way, but you've collected a ray of data and it's red. And so you know that like if I was to shoot out a little ray that there would be a red thing there, but you don't know the depth, right? And so you know you say to yourself, well, somewhere on this ray is some red pigment. And then you turn a little bit and you look, and you say, okay, now I have another ray. And if you have a lot of rays of red that all intersect, they say, oh, like this area in 3D is red. Maybe it's a part of a wall, a red wall or something. And so yeah, all that stuff is just really fascinating and it runs at that scale. It's just like a really—yeah, I would highly recommend checking out that Udacity course. I think you can basically take the course, but then if you want the certificate, then that costs a little extra, but you can audit the entire course. Really amazing stuff, I think. So now you know, if you pull out your phone, if you go to Google Maps, Apple Maps—here any of these map apps, you know, and you see the blue dot there, now you have an idea of all the work that's being done on your phone to put that dot in the right spot. It's really remarkable. And you also know, I just one last recap because a lot of people have asked me this, you know, if you're downtown and the dots bouncing all over the place, you also know why that is? Because the satellite energy is reflecting off those high-rise buildings, and now it says it thinks you're further than you are, and so that it accidentally draws that sphere larger. And so now it's trying to reconcile that with all this other data, and so you kind of find yourself either bouncing around or it says you're going east, you're going north. Like that's all because of that. This is a fun topic. This has been a good show. Yeah, this is great. Definitely. You know send us some feedback if you do want to.

1:29:06A:Folks out there in the audience if you want us to cover SLAM or KFS or stuff, you could definitely do that. Just let us know what y'all think. It's great to get the feedback. There was somebody who posted on Patreon—I don't know if Patreon feeds are public or how this works—but somebody posted saying, you know, they watched our show. It gave them the tools they needed to get into like some coding bootcamp and now they're a C++ developer, which is amazing. Again, I don't know if the Patreon stuff is public, so I don't want to say your name, but that is amazing. If you're hearing this episode, major props to you. Definitely makes us feel good that we're able to help you out there, which is really cool. And yeah, definitely send us emails, post on Patreon, and it's always great hearing what folks have to say. Thank you everyone. Yep, see you later.

1:30:03B:Music by Eric Barndoller Programming

1:30:07A:Throwdown is distributed under a Creative Commons Attribution-ShareAlike 2.0 license. You're free to share, copy, distribute, transmit the work to remix, adapt the work, but you must provide attribution to Patrick and I, and ShareAlike. And thank you.