165: Differential Equations

0:00A:Programming Throwdown: Episode 165 - Differential Equations. Take it away, Jason! Hey everybody.

0:23B:This is going to be a cool show. We're going to talk about something that Patrick and I are definitely not experts in, but I've had to do a lot of it recently, and I think it's very important. So we're going to talk about diff eq. Oh man, you already shortened it. I'm already out. You know, I don't know if did we have to take Diff Eq in college? I did; it was a requirement.

0:47A:Requirement? I did very, very bad.

0:49B:At it. You know, I honestly don't even remember taking it, so that goes to say something right there. I don't know if I took it or not, but we are going to dive deep into it. It turns out it's really important, and so we'll get to that. But before we do, I want to talk about spreadsheets. I actually fell back in love with Excel—in this case, Google Sheets—but you know, we really take it for granted. I mean, sometimes we even make fun of people who do tons of things in Excel. You know, we say like, 'Oh, like, you know, we could have done so much of this in Python or R or what?' But you know, one of the really powerful things in Excel is the sort of recursive nature of it and how you can kind of incrementally build on top of some data, and you can see all the intermediate values as long as you don't make the functions too complicated. So I had this thing where it's...

1:53B:Actually, it's kind of pretty nerdy, but there's this video game called X4, and in X4, you—it's one of these grand strategy games. So you know, you mine ore, use ore to like make solar arrays, and solar arrays give you energy, and you can use the energy in the ore to make spaceships, and anyway, it goes on forever. And I thought someone posted a Google Sheet of all the different resources and some stats on them. For the resources that require you to build things like solar arrays—you know what are the ingredients? Okay. So I had this Google Sheet as, you know, material, um the cost to acquire it if it's some raw material or if it's not a raw material, the time to build it, and then all the requirements, right? And so using some Excel functions, I was able to like recursively figure out the time it takes, the total time it takes to get any ingredient. Oh, so it's like the time it takes to build that thing plus the time it takes to build all of its components—you know, times however many of those you need. So a solar array, it takes 60 seconds at your base time plus, you know, it needs 10 of this other product, so whatever the total cumulative time is of the other product times 10 plus this plus this, right? And so that other product depends on other products, and so it's this recursive thing. But you know, because I just specify—and we talked about recursion a couple of shows ago, what's a pretty good tie-in—so you know, because I just put this recursive step in, and then I just control dragged that equation onto every row, and Excel like instantly figured it all out and said, 'Oh, for...'

3:48B:You to build a medium-sized battle cruiser, it actually takes like 18,000 seconds or something including all the other things you have to build.

3:59A:Then you cried because you had already done it.

4:01B:Twice. Yeah, I mean, it's—it was so, you know what made me think about is like that that's really cool. You know, like I just specified this simple recursive step, and then Excel handles like not calling the function too many times and caching the values. It kind of did all of it for me. And I wonder if like a Python library that does something like that would be really useful, like some extension to NumPy or something where in certain values in the array you could put equations, and then Python would just do what Excel does—like it would just figure it all out for you.

4:44A:Yeah, I ran into something not as complex as what you were doing, but we were talking about index investing a bit two episodes ago again. Oh man, look at us tying it in good us. Yeah, look at this. But I had this thing where, you know, some time goes by and then you want to put some more money in, but you want to rebalance what's there, right? So the idea of rebalancing is you have some target—I want the most common, you know, one that gets talked about is 60% stock and 40% bonds. You know, go read up on that, as I'm not actually advising you to do that. But yeah, let's say you want to do that. So you have some index that is your stock allocation and some that is your bond, and let's just say you have two for simplicity. Then over time one goes up and one goes down or both go up, and you want to get back to your 60/40 target, but you also want to bring in some new money. So could you run this on a calculator for two? Yeah, yeah, it's really not that bad, but just dropping them in—in this case, you know, I think Excel has extensions to do it, but I was doing it in the Google version, Google Sheets, and they have a 'Get the latest stock price,' so as long as if I updated it each time I do it, then it knows how many shares I have. So it'll tell me the new price. I say how much money I want to put in, and then it basically calculates my trades for me. And if I want to, for instance, like disallow selling—like I don't want to sell something; I just want to buy new—then you know, you can do that with the conditional logic and sort of like you're saying you iterate back on it and sort of step on it, do not even iterations, but just one column to the next where you're sort of rolling forward your computations. And it was just like, 'Oh,' because it turns out it wasn't doing it with two; I was doing with like five different indexes, and so you know, it's just so much nicer to like set it up once and then be able to just rerun it. If I had sat down in Python, I know there's an API for pulling down stock prices. I know I could have put my stuff in a CSV or JSON and read it in and done it, but I—I'm not clear that I wanted additional flexibility. I'm, you know, I'm not...

6:40A:Trying to solve this for every person or every case. This isn't my job. I just wanted a quick solution. But I will agree with you that my hobby horses on this are people don't want to learn SQL, and we talked about that before on the show, but I think people don't like to learn spreadsheets. There's a group of people who just don't know how to run Excel or Sheets, and they don't know how to get graphs really easily, and they say, 'Oh, it's okay, I do it in Python.' It's like, but I sort of guarantee if you only made me make this chart once or only for one thing—you know, doing it first in the spreadsheet, and then if you go to do it again, fine, I'll give it to you that the second time you have to do it, you can do it in Python. But that first time, you are almost guaranteed, if you know what you're doing, you're going to get it faster in the spreadsheet—just dumping out some output, you know, newline-separated dump from your program, and then you can copy/paste that into Excel and get your plot much, much, much quicker. Yep.

7:28B:Yeah, totally. And in your case, you're talking about balancing, so you know if you had a certain allocation, you know a certain state, then you know like how much of something to buy or sell to get you to equilibrium, right? So you have actually a differential equation. You have an equation that says here's how I get from where I am to where I want to be. But then if you take a really large step there, you might end up buying like way too many bonds and then you're unstable or something. Okay, well.

8:06A:Before we get into that, we will—two episodes ago in our duo, we were talking about the buzz and excitement of LK-99. Yeah, and it feels like what is the meme? It's back, and then it's over, and it's back. And if you followed it on X—I mean, just gonna go with Twitter—if you followed on Twitter at all, you saw these memes go by. But you know, it seems that a lot of people who sort of bided their time and really thought through it and did some analysis are pretty convinced that at least at this point there's no evidence yet that LK-99 is a superconductor. So there's a Nature article now out sort of saying this, and some sort of well more well-reviewed stuff is out. People are that the aftermath is somewhat interesting; there's still some studying to be done of the material, which you know happened. But then there's it's very divisive. I'm not a science academic scientist by trade or even a scientist period, I guess, but there seems to be a big divide between was this a okay thing that it happened this way? The original article was a pre-print, right? So it wasn't actually in a peer-reviewed journal; it was just a pre-print that everyone got excited about. Was this a waste of resources? Or is someone lose reputation? Or is this—yeah, this is just a more open involved, you know, broader scientific community and hobbyist sort of thing. I'll say from the sidelines that's the most excitement I had had about material science in a pre-print article ever, and I learned a lot. Yeah, clearly there was a lot of speculation going on on that. We talked about a little last time—people saying things that just weren't going to happen or weren't true, even if it had been everything it promised. But yeah, so what like you know, I don't personally see what was harmed if people decided to try to pursue replicating this using resources that they otherwise shouldn't have. I feel like that's a game theoretical decision that they made, and it's sort of case by case. So I don't—I don't sort

10:06A:of hold the broader community, but again not my domain, so it's possible that I'm missing some part of the equation there.

10:13B:Yeah, I mean, I think that—it's a good question. There's I have a couple of things here. One is, I've seen some people say that okay, it's not a superconductor, but it's still important because it you know, it's like a better conductor, and there are certain things that we learned because of it, and so it seems like it is some kind of incremental step forward. Um the other part of it is, you know, there's definitely like an outrage economy. And so you know, I think on things like Twitter, you know, like a lot of people probably got a lot of followers for being really passionate about it, and so that's a really big problem, I think that—you know, it's hard to overcome, do you know? If they if they're still like is that material useful even if it's not a superconductor?

11:10A:Yeah, I think some people think it might be interesting, but I thought once the hubbub has sort of died, a lot of people became a little more soft-spoken about things. Yeah, right? Like anything once it becomes more subtle, right? The you know, it's a little harder for people outside to maybe understand the implications. I expect people will still continue some amount of studying down this avenue. It seems like there's some novel properties, but yeah, I think the unlock of actual room temperature—and to be clear, it's room temperature and room pressure, like you know, sort of normal atmospheric pressure—the two things missing either one of those sort of makes it just not the step function of a new discovery. So even if it is still a superconductor at low temperature, which isn't clear that, like it could be a massive improvement, but not getting to where we are, you know, where the speculation was going, it's sort of a letdown. And maybe that's where people are pointing out is because it's a letdown of the public, then the public loses faith in the process.

12:21B:Yeah, that makes sense. My article is on normalizing flows. Have you ever heard of this?

12:28A:I know these words individually in many contexts, but I do not know what this is about.

12:35B:All right. So I'll explain. So um we have you know in random math, right? We have probability density functions, right? And so what this means is, you know, if you look at like a coin—a coin has heads or tails, right? And so as 50/50 chance. If it's a fair coin of being heads or tails. If you look at a die, you know, a die has one through six printed on it, and when you roll it, you know, if it's a fair die, you have an equal chance of getting one through six, right? Well, how do you do this with random numbers? And so the random numbers, you do it through probability density functions, which are like actually really non-intuitive. Um maybe like a good example is like voltage coming into your house. So you know, you have this voltmeter and it measures the voltage. You get a multimeter, and it measures the voltage coming into your house, and it says 120, but it's not like exactly 120 or like exactly this number of electrons past the multimeter. It's some kind of rolling average of some you know estimate. And so if you were to like dive into the multimeter, it's probably getting 119, 121, you know, and these are going to be floating point numbers like 119 dot x, 120 dot y, right? Um and then it's averaging them out and saying 120.

14:00B:Um, and so what you really have in your house, the voltage in your house is going to be some density function where the mean is 120, and then it has some tails, like sometimes when you read it, you get numbers that are greater or less than 120, right? Does that make sense? Yep. Okay. So it turns out, you know, if you have an expected value and there's just some error on either side, you know, a normal distribution does a pretty good job, and those are pretty easy to calculate, right? What if you have a really weird thing, like for example, what if I have a data set that has American voltage data and European voltage data in the same data set? So like, you know, half the numbers seem to be around 120, but then half the numbers seem to be around 230, and they're all just mixed together, right? So if you just fit a regular normal distribution to that, you're gonna—it's gonna tell you oh like almost all your voltages are around like 180 or something plus or minus some error, which is not really representing what you have. What you actually have is what's called a bimodal distribution where you have one mode, you have one cluster around 120, and then you have another cluster around 230. Um so you

15:20B:know what you want to do is you want to know the distribution of these kinds of data sets, right? These complicated data sets. And once you have the distribution, there's all sorts of interesting things you could do. Like for example, you know, if you know the distribution of the voltage in your house and let's say it's 120 plus or minus one volt, right? Then if the voltage is like 130 or 150, you know, like oh my house is really messed up. Like I—I'm gonna like my electronics are gonna start blowing up. I need to do something about it, right? If you measure the voltage in your house and it's like 120 plus or minus 20 volts and everything in your house is still working, then you know, like okay, I have a pretty good tolerance for voltage in my house, right? Um And so when you get to these really weird distributions, you still want to know the same thing, which is like have I seen something like this before? And so to answer, you know, have I seen something like this before questions, you need to have a density function, you know, like how dense is this part of the system? How dense is this function? And so normalizing flows are a way to get like almost arbitrarily complex density functions. So if you have really weird shapes that you know are—are um you know that that you can't really categorize as with any normal density function, you can always run normalizing flows and it'll it'll learn that function for you. Um and so it's a bunch of really cool things you can do. It's something like this. Like I'll give you.

17:04B:An example. You know there's these like breast cancer data sets, right? And they're pretty famous. They came out of UC Irvine, and it's basically a whole bunch of data of like stats on people. And then I think there's maybe pictures—mammogram pictures—and then there's like whether there's a tumor or not. And you know, you could train a classifier and say like yes breast cancer or no breast cancer. And there's a bunch of tutorials on how to do this on the internet, but you know you could use something like Normalizing Flows to say like have I seen a candidate like this before? If the answer is no, then maybe you don't trust, you know, the screening as much. So if the screening says cancer, but you've never really seen anyone like this who had all these qualities and maybe you ran an additional test, but if it says cancer and you've seen like tons of people just like this and they were all cancer, then you're much more like certain, right?

18:06B:And yeah, so there's a whole rabbit hole there. But check out the link; it has a really good explanation of how to do this. And this is a relatively recent thing. I mean, it used to be that you would take that normal distribution—the one I explained before—and you would do what's called mixtures. So mixtures are good if like the example I gave or something's either 120 or 230. So if you had something that was either like one of several values just with some error, then that's fine. But if something's just like arbitrarily distributed, like it's just like some weird shape, there's not really any way to recover that until some of these recent methods. So I've been keeping an eye on it, and they've been making tons of progress. This isn't even the latest one; it's just the one that I found the best explanation for. That's relatively modern, but I think that there's going to be a good future to these methods.

19:09A:Yeah, this is interesting. I naively would have thought—yeah, I have come across this problem before. I didn't know this was done this way. And what I did is sort of you think about histogramming and then just taking the data you do have and sort of making—I guess you almost call it a discrete density function—and just using the data you do have to estimate the percentages, which of course is difficult for things that once you've done the histogramming, like things that come in between, right? But yeah, there's some balance there. But that's actually really interesting. Yeah.

19:41B:I mean, if you've ever built a heat map, I mean that's basically a density function, and so this kind of takes it to the continuous space very.

19:52A:Cool. So my next one is probably going to require a bit of explanation, but it's how is Llama.cpp possible? So Llama is—and Jason, you're gonna have to help me if I get the background here wrong—but Llama is one of the sort of ChatGPT equivalents that came out of Facebook. And importantly, Facebook released the parameter weights as well as the architecture. And they just recently announced Llama 2, and there's sort of a debate around is it open source or is it not open source? They have some wording—go read your go read the contract yourself. The answer is for most people that aren't like already established big company, you can pretty much do whatever you want with it. Yeah, and there has been a person, a group of people who had come out with this tool Llama.cpp, which is basically implementing without—I don't actually know—I guess Llama isn't probably PyTorch or something similar, but without any of those frameworks from just sort of bare metal C++. Implementing all of the required tensor operations and you know things that you need to do to take the weights and to take inputs and to get the expected outputs. And the reason this is really interesting is if you follow the space at all, the expectation of running Llama is that you do it on either some specific tensor hardware like a TPU or whatever, or more commonly on a very now expensive from a series of sort of circumstances very expensive GPUs. And the biggest thing is you want to think about transferring the model weights to the GPU's memory and then it allows all these operations to take part very, very, very quickly. And this article sort of goes into to this explanation that this is mostly about memory bandwidth, right? So the GPU has very expensive, very high bandwidth memory as well as discrete parallel processing units for doing all of the operations needed.

21:49A:Llama.cpp doesn't benefit from you know sort of all of that GPU work. What they are trying to do is using SIMD operations—so it'd be like SSE, Neon, AVX, these kinds of things—which allow you to use your CPU's parallelization but also using the quantized models. So instead of floating point or double parameter weights, they do some retraining for a lot of these to get them down to four bits or eight bits per parameter. And these have billions of parameters. I think most of them are like sort of like 12 billion parameters, right? So you're talking about a huge reduction in the amount of memory and then running them on just basically your CPU now, albeit a sort of accelerated thing. But the nice thing is you normally have a lot more memory in your system—RAM, just normal sort of like DDR4 RAM or whatever—you know tied to your main CPU than you do on your GPU. So most people would not be surprised to hear 32 gigabytes or 64 gigabytes or even more of system RAM, but GPU RAM, you're probably, you know, 24 gigabytes would be really big. Even 12 gigabytes is sort of large. And so a lot of people don't have a GPU big enough to do this, but they do have CPUs. And so if you're willing to lose some performance due to the quantization—although not as much as I was expecting when you say I'm going to use four bits parameter per parameter—I assumed all this thing's going to be just degenerate, but that's not true. It actually retains a lot of its functionality, which is fascinating. Yeah, definitely.

23:18B:Then you can do.

23:19A:It on your CPU and the token speed sort of the amount of tokenization that occurs is still, you know, kind of low, only like maybe one per second or that kind of thing. So not enough for sort of like that interactive chat feeling exactly, but also not bad for not having to go out and buy a thousand dollar GPU and power supplies and everything to go.

23:41B:With it? Yeah, this is amazing, super exciting. Yeah, I think it's really cool. There's there's a ton of posts about this, and yeah, the four-bit quantization pretty is pretty impressive. I thought that you know with just what 16 numbers to represent each weight that you would have lost something really important, but sure, but yeah, it works really well. Yeah, I actually have with the four-bit quantization, I have a Llama with seven billion parameters that runs real time on my phone. Oh, yeah, that's cool. Yeah, there's an app called MLC Chat. This is for Android. I'm not sure if they have it for iOS, but you can run MLC. You can install MLC Chat and then you get a seven billion parameter model right there on your phone. It doesn't go to the internet or anything. It's pretty remarkable, and it will definitely hallucinate. You know, I asked it like what was the capital of Sri Lanka, and it made up a town. It might be another town, but it's not the capital. So it'll definitely hallucinate. You have to be careful. But I asked it how much Pittsburgh Pirates baseball tickets cost, and it nailed it. So I mean these are like pretty esoteric questions. You know, you basically have the power of a Google on mushrooms on your phone, and so it's pretty amazing. The.

25:18A:Thing that's interesting, I guess, is if someone was having this debate like, oh, if you're going to be stranded on a desert island, is it better to would you take a USB stick with this—is a terrible question—but USB stick for survival of like ChatGPT and what it knows and doesn't know, or a compressed Wikipedia?

25:39B:Oh, interesting. Probably probably ChatGPT, right? What do you think? I.

25:45A:Don't, I mean, right now probably Wikipedia, but given like if we sort of assume things continue to improve, I think that might start to change. Well.

25:55B:Actually, the Wikipedia one. Would you get a search engine too, or do you have to like? I think so. I think oh, okay, that's kind of a game changer. Okay, yes, I'd probably go Wikipedia.

26:05A:Yeah, but I think the interesting thing is ChatGPT would potentially know how to surface in a more interactive, queryable manner or even distill something. So you know, we were talking about—you're talking about normalizing flows, right? And some stuff. If I had questions about this going to Wikipedia might quickly over my head, and it's going to take me a very, very, very long time to get to where I need to be to understand versus, you know, ChatGPT could synthesize that down for me and potentially explain it. The problem, like you mentioned, is right now it's sort of not self-aware of when it's hallucinating, and so you could potentially end up getting an answer that it's very confident but very wrong. Yeah.

26:46A:I also don't know if you described something right. Like if I described a coconut to ChatGPT, maybe it would be able to tell me it's a coconut. It would be very difficult to query Wikipedia for finding out, you know, is it what is this thing I found on my desert island and can I?

27:02B:Eat it? Yeah, I mean, I was thinking about building a device like this. My idea was basically a flashlight that had an open-source Large Language Model and speech-to-text and text-to-speech. So it's like you could hold the record button and say, 'You know, is this tree with four leaves poison ivy?' And then it would say something back to you that—that.

27:30A:Is really cool. Although you could also use it to start a fire with the heat it generates, so you know, like perform two roles.

27:36B:Oh man, all right. So actually, my news is also related to LLMs. It's this cool website called chat.lmsys.org, and you can go there. You can ask questions to a variety of different open-source Large Language Models, and you can even ask questions to multiple at a time and see what the different answers are like in the spectrum there. So I typed into the 13 billion Llama 2 model: 'What are normalizing flows?' And it nailed it. It said, 'Normalizing flows are a type of general model that uses a series of invertible transformations to model the distribution of the data,' and it went into the different types of flows. Um it doesn't.

28:28B:talk about radial flows, which are the most popular. So it missed out. It listed four flows but in my opinion missed out on the most important flow. Um so you know, in general, I think that you know that's kind of in line with what you'd expect where like some of the details are kind of hazy, but it gets the overall idea right. Um yeah, I mean this thing nailed it. It explains why it needs to be invertible. Really impressive. Um so yeah, check this out. If if um if you if you haven't really been able to access open-source models—it seemed like it's kind of too far from reach technically—um this is a super easy website. You literally just click, and it's basically the ChatGPT interface but for open-source models. And so you know, you can play around with this. It's using someone else's compute, so it's totally free. Maybe at some point you'll get rate limited. I haven't tried that. I haven't gotten there yet, but um but you know, you can just play around with this, and if if it solves something for you, then you could—that could justify, you know, taking the time and energy to, you know, get one of these running on your on your own computer. Also, this LMSys is all open-source, so you can actually clone this repository and make your own version of this where on your computer you can ask, you know, yourself questions or your hardware questions. Um so you could go from, you know, this ChatGPT interface to something running on your own hardware, which is pretty exciting. I've talked to folks who are generating real value with ChatGPT. ChatGPT is so cheap it's really hard to justify, you know, the open-source models. You know, a lot of the folks I talked to.

30:24B:are just using ChatGPT. Um, you know, my interactive fiction game—I have a bunch of folks playing it, which is really cool. And even with all the folks playing it last month, my bill was 57 cents, so it's hard to compete with that. But but yeah, I mean, yeah, I think for sensitive things or as a curiosity or or for whatever reason if you want to do the open-source ones, this is a great opportunity to see how they compare and how they how they've been maturing.

31:01A:Time for Book of the

31:02B:Show. What is your Book of the Show?

31:05A:Patrick, my Book of the Show is as adequate for my level of thinking math with bad drawings by Ben. So I have not—I have not dug all the way into this book. I have this book, and I am familiar with some of the writing here, and you know, I've read a little bit of it. But for me having, you know, I'm not a mathematician by background or by training. In fact, I did very poorly at it. Spoiler alert for the upcoming topic, but I will say that like I do end up having to do a fair bit of math as I think many people in computer science end up at one point or another, or you know, I guess maybe depending on your sort of role, but I do do a fair bit of math. But for me it has to make sense to me and it's useful to find people who at minimum find alternative ways of explaining, but I will say find non-academic ways of explaining intuition behind things. It is sometimes hard to to sort of explain arbitrary topics via intuition from start to finish, um but there's a number of people I'm very happy to have found. Um and we talk about one later, but there's like three blue one brown is like a YouTube channel we've talked about before where he does these math explanations. Even watching, you know, just math videos on YouTube in general that aren't—aren't classes are good, but Ben Orland has written this book and sort of covers a variety of topics there. There's several follow-ons in the same thing: one about playing games, one about sort of calculus, and just sort of some comic just to kind of like keep you engaged, but also just sort of like cheesy illustrations in order to to sort of help you help you think about it, and it just vibrates sometimes with how I think about the world. And so uh I recommend this book if you if you're at all curious, if you're a math tourist, I would say who's just interested in kind of like a light treatment, a popular treatment, I guess, of a lot of these these topics, definitely check it out.

32:59B:Very cool. I actually, you know, you sold me on it. I literally just bought it while you were talking. Oh no, it's gonna be a Book of the Show next time for Jason. It's gonna be my Book of the Show. You know, we were talking about this before The Show how um you know sometimes a lot of like the book I'm reading right now is kind of invalidated because it was Patrick's Book of the Show like seven years ago or something. But doesn't mean you shouldn't read it. Definitely go back and read all of our Books of the Show. A lot of them are.

33:30B:Real winners. All right, my book of the show is Beyond Reading Brayden Sanderson, which we talked about a couple episodes ago. I was playing this text adventure called Overboard. It's from the people who made 80 Days, which was my book of the show in 2017. This one is interesting. You know, the challenge with interactive fiction is replayability, right? I mean, clearly like a book has very little replayability compared to video games—a regular book. And so interactive fiction, you know, you have to kind of figure out if you have to create all this content for all these things a person could do, but then they only do one path and then leave it. Then, you know, a lot of that content is might not be explored by most people. So they want you to be able to play the game multiple times. And this one takes an interesting approach where they have a system of achievements. So, you know, you play the same game every time there are multiple endings, but a lot of the same thing happens; it's deterministic. So if you do the same thing at the same point in time, you'll end up with exactly the same ending. But as a way to keep you finding new content, they have this system of achievements. Achievements are based on what you have and haven't seen. So if you play the game and just coincidentally you get the best possible ending, then you'll get an achievement for like 'kill as many people as possible,' you know, be like the most evil person possible because you haven't tried that yet. I thought that was really clever. I feel like it wasn't executed perfectly because it's not clear how to get some of these achievements. So you see the achievement.

35:30B:And maybe you play, you don't get the achievement. You have to keep trying and trying, and while you're trying, you're just trying to do random things. It didn't do the best job of keeping track of what you've ever done in the past. So, you know, it has some work to do there in execution, but I love the idea. I think this idea of achievements kind of takes interactive fiction and makes it more like a rogue-like where you start the game with like advantages the more time you play. So, I think there's something really powerful there on the game design front, and I had a lot of fun. It's a good story. Let me just check really quickly what it costs. I can't see what it costs because I already own it, but yeah, it's probably like five bucks, totally worth it. Highly recommend 649. Definitely, it's worth it. You'll easily get a few hours of entertainment minimum out of it. And if you don't want to buy that game, you can actually buying that game doesn't support us because it's not an Amazon book, but if you don't want to buy the Math with Bad Drawings book like I just did, you can also support us on Patreon. So, if you go to patreon.com/programmingthrowdown, big shout out to our supporters. You know, we have had an influx of supporters, which is really cool. I wonder if that's companies forcing people back in the office has something to do with that. I don't know if you are forced back into the office and that caused you to support us on Patreon. We lament the fact that you know you're forced to go back in, but we really do appreciate the support. You know, all of that money goes in an account which we use for the show. We don't really pocket—we don't actually literally don't pocket any of it. We use it to try to bring more people.

37:28B:Into the show and spread the word. So we really appreciate every single dollar of support that we get there. And with that, we'll go. Oh, go ahead, Patrick. Oh.

37:39A:No, just saying thanks. You go ahead.

37:41B:Oh yeah, thanks, thanks everyone out there. We'll go to Tool of the Show. Why don't you go?

37:48A:First? Okay, mine is a website: FFImproviser. Good luck trying to Google that. You could just, but this is—it that's not even the full website. This is a GitHub link to a sort of interactive website with, I guess what you call it, almost FFImpeg one-liners. So, I've had FFImpeg as my Tool of the Show before Jason did one of us did super powerful tool, horrible command line interface. I'm sure it has to be definitely; it probably has to be. I'm not saying I could do better, to be clear. Please don't—don't at me. Well, good luck. You can at me. I don't see them. Yeah, you.

38:22B:Don't have an ad where you.

38:23A:Would at me at, but anyways, the benefits. Okay, sorry. You can send me a letter if you could. No, no, no, don't—don't. Yeah. All right. So, FFImproviser has a bunch of things you may want to do with FFImpeg, and it gives you sort of one-liners that you can use to combine together, and so definitely makes a very powerful tool but hard to use, a little easier to use, you know, helpful. So, definitely check it out. I also am bumping into this. I did see recommendations for a group of similar tools including some like Visual, if you ever played with like LabView, like you drop little modules in on like a data flow graph and it'll sort of generate FFmpeg commands for you. So, shout out to a bunch of people like out there trying to make ways of FFImpeg easier to use because truly a powerful tool. I mean, it's used all over the place. This is really awesome video, and video codecs are insane. I still don't even profess to understand what any of them do or are or the difference between container versus, you know, I'm not even going to say it anyways. So, definitely check it out: FFImpeg Improviser, FFImproviser—very useful tool for helping you because if you're like me, I yes never remember. I have like little README sprinkles all over the place for places I normally run FFImpeg to try to remind me of the command I.

39:56B:Used last time. Nice, very cool. My Tool of the Show is Pandas read_ods and read_excel function. So, I know that's kind of really specific. We have talked about Pandas, but it's been probably—I think it was 20—is 2016 when we talked about Pandas. Pandas is basically a library for getting data frames in Python. If you've ever used R, you know what a data frame is. If you haven't, it's basically very similar to an Excel table where you have columns. Each column has a name and a set of values, and then you can do arithmetic; you can add columns together and stuff like that. And Pandas actually can read ODS, which is the Excel file format for the open—open Excel file format. So, you can save a Google Sheet to ODS. Obviously, LibreOffice and OpenOffice read and write ODS. Excel probably will save ODS as well. Pandas can also read Excel files like the XLS and XLSX and all those files. And what Pandas will do is it will actually evaluate all of the formulas, you know, resolve them to their number. What it won't do is it won't on the fly, you know, evaluate the formulas or anything. That's something we talked about at the beginning of the show, but if you have an Excel spreadsheet with a bunch of complicated equations and you want to bring that data, that output into Python, you can just read it in one line using Pandas, which is pretty remarkable. So, yeah, I think there's a good synergy there. And yeah, I was really impressed when I found that function that.

41:49A:Is very cool. Yeah, the fact that you said it even does like evaluation, not just like reading it as columnar data, is pretty cool. Yeah, I'm not sure. Like, oh, go ahead. I was gonna say I want to make a joke about like VBScript and whether it evaluates all the random macros, but.

42:05B:Oh god no. You know, I actually don't know if the file format saves the function and the value, you know what I mean, in the file or if Pandas is doing all the calculation.

42:20A:I really don't know. I will say I've never written a complex enough equation in my spreadsheets that like I would have been able to tell the difference of whether it was like cached or computed, you know? Sort of like them—I would like when you open the file does it take, I guess, if you could construct processing complicated enough that it takes literal minutes to sort of finish computing your spreadsheet, in which case it would make sense that they write the value out as well, right, as a cache. Oh yeah, but so it'd be easy enough to test. I've just never written such spreadsheets myself, so.

42:55B:You know, my guess is it probably has the cached value because otherwise the Pandas people would have to implement whatever library they use would have to implement like every Excel function and test it to make sure you got exactly the same answer.

43:10A:It's probably one of those things. You know, it's almost non-existent the number of Excel files with a lot of very random esoteric functions, so it's how like do they support 99% of the files? Five nine six nines? Like most Excel sheets probably don't even have a formula in them. Yeah.

43:29B:It's a good point. It's a really good point. But either way, give it a shot. You know, take it. I mean, Pandas is extremely well supported, but still, I would always have like a little degree of caution with something like that because you are going from one language to another. Yeah.

43:46A:There's probably like a decent way to force it into a good state even if you have a stale spreadsheet, right? Copy the values out or something and you lose the formulas, but Pandas would still be able to read it.

43:59B:Yep. Yeah.

44:02A:Totally. All right, differential equations. So you know we

44:08B:talked about recursive functions scaring new programmers, and now we have something that scares us? Yes. I'm—I'm

44:15A:actually pretty nervous, dude. I'm.

44:18B:Shaking right now. So let's start as we always do with the motivation—why people should learn about diff eq. So you know if you usually when you hear about differential equations, you think about physics or, you know, maybe economics? Actually, we'll put economics aside for a minute. You think of like physics or or you know these kind of signal processing, these kind of things that are really kind of low level, you know, things that interact with the real world through sensors and all of that. But there is a reason why a lot of people even if you're building, you know, a website or something like that, or or you're driving engagement to a website or you're doing something, you'll need differential equations. And that is because of this thing called the law of large numbers. Have you heard of the law of large numbers? Not?

45:11A:In this context? So okay, think of it in statistics. So I'm excited to see if it's the same.

45:16B:or not. It's exactly the same. So the law of large numbers says if you have a lot of things and you need to reduce those things—like let's look at dice, you know? If you say I want to throw, you know, 17 dice and I want to get the average, right? So you know, you'll get you can just keep track of all these numbers and take the average, and you're going to get something, right? What if you get to like millions of dice or billions of dice? You know what can you do? What—what the law of large numbers says is that if I average a sequence of numbers from a distribution, that average is going to be centered around some point, right? And so with dice, for example, you know if you average a lot of die rolls, you'll get something around 3.5, right? If you do it enough times, you're going to get around 3.5 even if like the first 20 times you rolled you just coincidentally got a one. You know, eventually with enough samples, the odds of that kind of anomaly go infinitely close to zero, and so you will end up with a number around 3.5. And so you know if you want to do things like how much time are people spending on my website or how many people a month are visiting my website or maybe a minute by example, how many people a minute are visiting my website? So anything where you're going to end up with many, many, many, many samples, you're going to end up probably doing an average but taking some kind of estimate of these samples so you're not working with billions of numbers. And as soon as you do that, now you have a continuous number. And so if you were thinking about like traffic coming into Google, right? They're not they're not looking at it in terms of

47:16B:Like okay, I have this exactly this many people and then exactly this many people. And I'm going to draw some insights. Like no, they're taking averages and they're looking at sort of trends over those averages, and they're constructing some type of differential equation that's saying, 'Look, people, you know, this metric is going up, this metric is going down. Um here's the covariance,' which means as one metric changes how it varies the other metrics. And so based on the velocities of these metrics and the covariance of all of these metrics, you can construct a differential equation that shows you like how you think these metrics are going to evolve in the future.

48:03A:Yeah, so looking at the—oh, I'm going to start saying words we're not supposed to. But anyways, looking at the rate at which the people are coming out and then sort of understanding how those change over

48:17B:time. Yep, yep, yeah, exactly. And so you know there's many situations where you have sort of what you want to achieve in the next time step, and you know that is easy to compute. I think the balancing stocks was a great example where it's like, 'You know, if I have 60 percent stocks and 40 percent bonds, and I want it to be 50-50,' well then I know what I need to do. I need to sell stocks and buy bonds until I hit 50-50. Um and so you know, you have your difference—like what from you have your difference from your goal to your current state. That's relatively easy. What's hard is sort of keeping that stable.

49:04A:So yeah, and this is why they pop up in if you ever read stuff about controls and control systems. So,

49:14B:Yeah.

49:14A:Trying to figure out how to turn the steering wheel on your car to get to where you want to go quickly but not overshoot. And how do you sort of think about computing these things?

49:26B:Yep, yeah, exactly. So like let's say you had—so let's say you know, in this case it's clear. It's like okay, there's a percent and so if I get one percent of one, it's kind of one less percent of the other. Well, let's say you didn't really know that these stocks and the bonds had to add up to a hundred percent, right? So you might say, 'Well, the stock number is 40 percent; I need it to be 50 percent.' So you know, I'll buy 10 percent more stocks. But then but you—oh, you don't have the exact 10 percent, so I'll just buy more stocks. But then you buy more stocks and then you overshoot or even a better example is, you say to yourself, 'Well, my house is 80 degrees; I need it to be 79 degrees.' And so you know, I'm just going to turn the AC on, you know, full blast. But you do that, and then it causes you to overshoot, and you have to kind of keep going back and forth. Um so it turns out that. For a lot of these, what you really want is something that is pretty stable. And oh, another part of this is it gets really complicated when things affect other things. So for example, let's say you have, you know, air conditioning and humidity. But when you run the air conditioner, it causes your house to become less humid. And so you might have a humidifier, but then the humidifier drops the temperature. And so these things are affecting each other in this weird kind of feedback loop that can spiral out of control. Like you could end up in a situation where your humidifier is full blast and your air conditioner is also full blast, and they're just going to war with each other because the differential equation is not stable. And if you've ever played a video game with a physics engine, you might have seen this where you know if you do something that is unstable, like you try to attach a car to another car and the chain is like not long enough to support both of the cars, and everything starts like jittering and then spiraling out of control. That's a case where you know the game because you know it wants to run in real time, it's going to take kind of larger steps, and it's going to spiral out of control. Um so yeah, we should actually talk a little bit about step sizes. So you know, in Patrick's example of the stocks, right? You might say like buy 10 percent more stocks. So let's say you instantly do that, and you know Patrick is a billionaire, right? So that's not a small transaction. So

52:29B:that's not a small transaction. So 10 percent of all of that—let's say that influences the stock market or even just it's happening at a really volatile time where stocks and bonds are going up and down. It's crazy. You know, you buy that 10 percent, but you buying that kind of changes the market dynamics. Maybe it makes the price go up, and then also like other crazy things are happening. And so you find that oh, after I did that buy, I'm actually even further from the goal. It's like being bad at mini golf. You know, you're bad at mini golf, and you putt, and it goes past the hole, and then you're even further from the hole, and you putt again, and it's just getting worse.

53:12B:Right. So you know one way to do that is to say, 'Well, here's what I'm going to do. I'm going to buy one share of a stock or a bond.' Let's say buy one share. It's going to get me a little closer to 50-50, and then I'll measure again. It's like okay, now I'm at 40.1 stocks bonds, so I'll buy one more bond. I'm at 40.2. I'll buy one more bond, and so you could do that, and you'll eventually get equilibrium, and it'll be stable. But it will take forever, right? And so that's the paradox: if you go really slowly, then it takes a ton of either compute or time or what have you. Um if you go quickly, then you can cause like second-order or multi-order effects. And there's an example I have that folks should check out in the fun example section at the bottom. It's predator prey relationships. So the idea there is basically, you know, every fox eats one deer, and deer multiply at a certain rate. And the question is, you know, how do you end up with something stable? So if you have 20 foxes and 20 deer, and you take one full step, well what happens? All 20 foxes eat all 20 deer, and then all the foxes die because all the deer are gone, right? So you can't just take one full step or it just destroys itself, right? So you could take baby steps and say, okay, like again I'm going to do this. I'm gonna assume that because these law of large numbers of these 20 foxes are actually, you know, 20 million foxes, and so I can kind of divide them into smaller and smaller numbers, and so you know the 20 foxes start eating more deer, but they don't all eat a whole deer. There's just tiny micro changes.

55:05B:and the deer population starts tanking, and then that means the fox population starts dropping because they start starving, and that allows the deer population to recover, and then you get this sort of like harmonic situation. And you know basically you can see from this site like based on the step size either you know all foxes eat all deer and then they all die, or you get this like oscillation thing, or with a small enough step size it kind of converges to some equilibrium where enough foxes are eating just enough deer to match the growth rate of the deer. And it's all kind of exact. And you know if you had started with that end state, it would be an equilibrium, and you can set the step size to anything, and it would just stick there. But because you started from a state that wasn't in equilibrium, you know the step size really starts to matter. Does that make

56:06A:sense? Yeah, totally. And I think this is part of the sort of modeling setup and the iterative nature of a lot of these things and running them forward. And you already kind of mentioning but simulations where you hear about this a lot, and thinking about the consequences of what you're doing and how to simulate these things. I think with like what you're saying, I tend to think of a more like discrete way, right? Like oh, you could have like the foxes be in cells and the deers—I think there were foxes and deers—deers be in other cells, and then like having them move around. And that is one way of doing it, but I guess you call it more like an agent simulation. But that's the only way to do it. You don't actually need to model any individuals where you're kind of saying by the law of large numbers, you don't need to actually model any individual deer or any individual predator. Like you could just model it at scale and talk about the full thing, where doing an agent-based simulation actually would run into a lot of problems. So they're sort of treating the problem with different approaches.

57:08B:Yep, yeah, exactly. And so, you know, a lot of biologists—I mean, definitely biologists in like the 1900s or 1800s and stuff, you know—they couldn't do the simulation, and so they had to rely on these really coarse things where they'd have people go out and, like, you know, count the number of foxes in like a whole county and then do that a few times until they could get some kind of statistical average. You know, they could use the law of large numbers even if they don't have a lot of numbers, right? And so just tolerate some kind of error there and then and then move into differential equations. Um, so yeah.

57:47B:So I think, um, okay. Yeah, so we'll dive into, um, you—you'll hear the word ordinary differential equations and partial differential equations. So, if you look at like the fox example, you know, the fox, the number of foxes affected by the number of deer and and vice versa, and all of that. Um, but you don't really care about modeling it in those dimensions. The only dimension you really care about is time. So it's like, you know, what is going to happen in the future? So, you know, if F is the number of foxes, D is number of deer, and time is T, you know, T is the only one you're exploring now. F and D are changing as you, you know, move through time, but you're not moving through the number of foxes, you know. You set an initial value and then you move through time. And so what that means is you're only differentiating through time. So it—that's where partial differential equation comes from. You know, the equation has many different variables, but you're only—you're only—you're only interested in a part of them for the purpose of differentiating. And so if you're only differentiating through one variable and there's maybe some other conditions too, then you have an ordinary differential equation. And if you're if you have an ODE, then then a whole bunch of techniques open up to you. Um another thing about I think ordinary differential equations is there's no discontinuities. So this is—this is again a really point that folks can get stuck on, right? They might say, 'Well, you know, I might have some if statement that says well, like if I'm in a certain case then the foxes eat the deer; otherwise, the foxes don't eat the deer.' And so there's this break, right? Um there's some variable that's like binary, right? Um and if you do

59:52B:that, then then, you know, all of these techniques aren't going to work. And so that might sound really limiting, but again, when you start looking at really large numbers, you should kind of expect everything to have some inertia, right? You shouldn't expect there to be like these really sharp discontinuities. Just like, you know, you wouldn't expect like everyone to leave Google tomorrow. Like it's just not—not really reasonable. Like even if Google like somehow the whole page is full of ads tomorrow or something they made some kind of really bad product decision, there'd still be like this decay, right? So um so, you know, at this scale, you know, a lot of those constraints are pretty manageable. And um um and so once you have an ordinary differential equation, then there's a whole bunch of interesting things that pop up. Um, you know, in the way that we were talking about solving, you know, Patrick's balancing stock balancing issue, we were using Euler's method. We were basically saying, you know, we know how to get exactly to the goal, but we know that if we took that big step that um, you know, it would cause some disruption and it would move the goal post, and we would be in trouble. So we'll take baby steps, and at each step, we'll reevaluate the distance to the goal. You know, this is um another good example of this is like um like missile tracking. So if there's a target and you have a missile—a missile might, like, you're playing some kind of video game or something. The missile might say, 'Oh, my target's here! I need to draw like this arrow. Like I need to go in this direction.' But then, you know, the target's moving. So as the missile is going to the target, the target's also moving. And uh and so, you know, the missile would end up kind of curving towards it if it didn't have any predictive capability, right? Um and so that's

1:01:52B:that's another example of a differential equation. Um oh yeah. So so Euler is a first-order method. Um there's also like second, third, fourth-order methods. And what these do is they look at a series of predictions and they use a series of these to get an even more accurate step. Um so for example, um this is using the mini golf example. Let's say you overshot the hole, and uh um you kind of learn to yourself like, 'Okay, you know, I need to hit it a little less hard next time,' right? So even though like maybe you overshot the hole and you're twice as far as you were last time, you kind of learned, 'Okay, I need to hit it twice as hard, but also a little bit less because I clearly hit it too hard this time.' Let's say you hit it and you hit it, you know, 1.5 times as hard, and you still overshot the hole. So you kind of learned like, 'Okay, I need to hit it even less than that.' And so with these other multi-order methods, you can take bigger steps towards the goal and still have less of this um overshooting problem because it's taking more information into account.

1:03:14A:We were been talking about I guess like controls and like controlling things or taking action. Um but I think we also see them pop up in sort of running physics engines and games. And to me, there it's a little different. You're attempting to model the setup of the differential equations, like the force of gravity on a ball falling down in your video game. Um and there you're trying to say, 'I'm actually wanting to compute position by giving an arbitrary time.' So I'm going to give you some time in the future, which is, you know, my game last displayed here, and I've moved X time step forward, and setting up your equation so that instead of defining that step in advance, you can compute the position at an arbitrary time into the future. And so there, I think to your point, you're sort of changing the situation a bit rather than what action am I going to take? You're sort of giving the system, 'Hey, here's a new time. Can you tell me what all the positions are now?'

1:04:20B:Yeah, that makes sense. Yeah, actually it's a really good example. Like imagine if you have a ball that's moving towards a wall. You might say to yourself like this is a pretty simple situation and I could take a pretty big step. In fact, if there's nothing around, maybe I take such a big step that the ball hits the wall and I don't have to think about anything in between. So, you know, just maybe the ball is so far from any other object that you just say, 'Look for the next three seconds, this ball is going to arc and it's going to bounce off the wall,' and I can come back three seconds later. I don't have to do any math at all. Um conversely, like maybe the ball is chained to another ball, and both of those balls are now like you know moving through the air but in weird ways where they're kind of pulling on each other, right? Well, that is a really unstable, and uh and so you might need a really low time step. And all these physics engines do kind of dynamic time stepping and stuff like that, but you know using a multi-order method is even better um if you can afford it. So the equations are more complicated. I thought you

1:05:30A:were going to—you said two things. I thought you're going to do three, and we're going to talk about the three-body problem. Um but no, you avoided that. But I mean just to reference that, I mean, I think one of the things you'll see pop up when you're talking about differential equations is if you think about as Jason was mentioning, things interact with each other. So it's not just, you know, one object moving through the world without consequences. If you think about in space, if you think about two planets or two massive objects, or low mass—anyways doesn't matter—think about two things in space near each other. They are pulling on each other gravitationally. There's a force, right? So this is like you are trying to understand why does the Moon orbit the Earth, but the Earth-Moon together also orbit the Sun? And so if you sort of think about the interactions of these as they play out and fast forwarding through and understanding where they're going to be at some time point, it—it depending on the level of precision you care about. Yeah, the Moon is way smaller than the Earth, but it does actually cause the Earth's orbit to wobble around as it spins around. It's pulling on Earth just as Earth is pulling on it, which is um, you know, you'll see things like, 'Oh, we are predicting that there's a planet that we can't see because there's these slight disturbances in another orbit of a planet,' you know, as something is passing by and they pull on each other. And you know that it's sort of a continuous thing. Um and it doesn't matter how small or big it is now. At some point, you probably just don't care about it anymore, right? You know, a baseball space debris floating around the Earth has an influence on its orbit, but it's such a degree you're probably going to ignore. But as you sort of try to account for more and more of these at, you know, increasing levels of precision—a precision uh, you know, imagine trying to launch a space vehicle to meet up with Mars and make a precision landing, right? You need to be able to solve these differential equations in a very precise manner, as Jason's pointing out, like allowing

1:07:25A:to layer on more and more uh, you know, ways of increasing the accuracy and precision because you really care—you really want to know the exact moment that the two things are going to meet up because it has a range of consequences, including how much fuel you put on your rocket. And how much fuel you put on your rocket determines how much velocity it's going to have, and it's a it's a cascade of things, and you need a setup that allows you to iteratively converge on a solution.

1:07:49B:Yeah, totally. Totally. Totally. Um yeah, I mean there's—there's so many things where when you start modeling at a high level, you run headfirst into differential equations. Like uh um like imagine you're making some kind of strategy game, right? So it might be that if you want to know if your game is fair, if it's balanced, um it might be hard to do that in simulation, you know, because you have all the different units and they're moving around. And and it could be really difficult to think about all of that and compute all of that. What you could do instead is, you know, come up with some mathematical models that say, 'Okay, you know, if I have uh so maybe I have a measure of army strength, which is just a number that I've learned, and I've learned that, you know, for this configuration of units, I have a strength of this. For this configuration of units, I have a strength of that.' And I learned some function that says given two army strengths, which one wins and maybe what's left of their army after they win? Uh or so, given given two army strengths that fight, you know, what are the army strengths at the end? One of them will be zero, and the other one will be something. Um, you know, now you can construct sort of differential equations that talk about um, you know, should I, you know, if I like like what happens to my army strength as a function of the current base? Yeah, am I like how is the army strength growing and subtracting and all of that? And so, you know, once you start abstracting out um to to kind of a higher level, then you start running into differential equations. So if you ever play like SimCity or Rise of Industry or these these type of simulation games, you know, they're going to be full of partial differential equations, and I bet you even beyond what's in the game, uh you know they were

1:09:49B:tuned, and that game was tuned using uh differential equations solvers. You know another like more kind of industrial example or commercial example is PageRank. So PageRank was the um algorithm that Google used initially to rank all the content on the internet. And the way it works is they manually give some amount of rank—think of rank as like a resource, right? They give some amount of rank to a bunch of websites that they uh code up by hand, and then they say when those websites link to a website um some of that rank is diffused onto that website. So um so if if uh you know um uh like msn.com links to some some uh kotaku.com and some energy passes from one to the other. And so they used initially they used Euler's method. So they said, 'Well, we're not going to give just all of MSN's energy to like the first link we see; that would be crazy.' So we're going to look at like roughly how many pages, how many domains does this domain link to and will diffuse uh you know some portion of their energy equally to all of those pages. And they're they're linking to other pages, and those pages are linking to other pages, and so you have this multi-order effect. And so um you have a you have a partial differential equation. So as time progresses, you know, this rank diffuses through all of these websites until eventually it stabilizes, and using um you know differential equations and you know

1:11:48B:the theory behind that, you can compute the PageRank, you know, more or less efficiently while still getting a pretty stable answer. And even, you know, when you're trying to figure out how much rank should I diffuse among the different pages and what's a sort of what's the most fair way to do that, you know differential equations will allow you to do that very quickly. So for example, if you were to just use Euler's method and compute PageRank on the whole internet, maybe that would take days, maybe months. But if you were to use faster methods, you could get the same answer in like two minutes. And so getting it that quickly allows you to do a lot more interesting analysis.

1:12:31A:We haven't yet told people how to actually solve differential.

1:12:36B:Equations. Jason, oh man. Yeah, that's a really good

1:12:41A:point. I'm teasing. I think this has been—this has been a great sampler, like an overview of the various aspects. And I feel like it's one of those things where you know hearing it over audio, I guess maybe some people are out there furiously scribbling notes, but I feel like that's the right way to cover it, to give sort of the list of things, the sort of what's the next hop in the journey to learn more about these things, to understand if you've encountered these things and didn't know the right words for it. We were talking about—about that, right? About Wikipedia versus ChatGPT, like how do you—how do you sort of find the next thing? So I—you know, hopefully this has been useful to people. I mean, without going into a very lengthy discussion about how to actually run Euler's method or do some of these things, I feel like this has been a great introduction to the topic. Yeah, I

1:13:31B:think this is one of those categories where you should use software off the internet, so SciPy. Yeah, like SciPy for Python is amazing. They have like seven different solvers; they're all great. But I would say the first thing you need to do is come up with a function, and you can pick whatever language you want, but come up with a function that says given my variables—so like using Patrick's example, you know, given my percentage of stocks and bonds, you know what is the step? Like what is the difference I need to make to solve that? So in Patrick's case, you know, if he has 40 percent one, 60 percent the other, then the difference is to get it to 50/50. And I guess you know you need the denominator for that. But but you know let's say you pass that in just like here's what I could do—that would like instantly solve my problem based on what I know right now. Come up with that function, make sure it doesn't have any side effects, doesn't need to read from a file or anything. It's some pretty small function; it doesn't use globals or anything like that. And then you can plug it into a solver, and you can actually explore different solvers and see what happens. Cool. Very cool. Yeah, definitely check it out. It's a good—it's an important thing. You will often be in a situation where you know what needs to be done, but then you do it, and as you do it, you change the nature of the system, and it seems like you never really get to the goal. It's that's what differential equations are really good at fixing. So yeah, check it out. Check out the links. We have a ton of really good content in the show notes. You can go to programmingthrowdown.com or you can look at the show notes tab of your podcast app if you have that available. You can also go on our

1:15:29B:Patreon, and we have all the show notes there, and a super fast RSS connection over there.

1:15:39A:Thank you everyone.

1:15:41B:Yeah. All right, thanks everybody. My head hurts. How do you feel? Is your brain on fire right now?

1:15:46A:I guess it's time for me to compute the rate of change of hormones in my brain from—yeah, neural connections. I need to compute.

1:15:54B:How hard my head hits the pillow.

1:15:57A:Oh no. All

1:15:59B:right? Everybody, hopefully we didn't melt brains. It's a super interesting topic, and we will catch you all next time. See you later.

1:16:18A:Music by Eric Barndoller Programming.

1:16:22B:throwdown is distributed under a creative commons attribution share alike 2.0 license you're free to share copy distribute transmit the work to remix adapt the work but you must provide attribution uh to uh patrick and i and uh share alike and kind you