Rethinking AI Harnesses

0:20S1:Hey what's up? So I think the way we're thinking about harnesses is not correct. I think it was the right way to think about them before basically as like coding systems, like coding assistance and like a set of tools and stuff to do coding better and to release apps faster. I think we have to start thinking about this completely differently. The way I imagine a harness to be, the way I think about harnesses is kind of like a content management system. Okay. The problem that we're trying to solve, actually, is that it's not exactly clear what is in our brains as a kind of a feeling of what we want. And it's not easy even within our own brains to like, translate that feeling into actual words, right?

1:03S1:Then we have to actually capture it properly, which like has been known as like prompt engineering, right? So we have a prompt engineering problem, but before that, we have a feeling in our brain problem and how that's going to translate into actual text, which we can then capture into a prompt and then which we can then give to an AI and hopefully it understands. Hopefully it's smart enough to understand the prompt, which is the whole purpose of prompt engineering. Well, I think we've confused this whole harness idea into thinking that it has to do with coding, thinking that it has to do with basically, what are the best tools we can give? What are the best prompts that we can write? What are the best like specs and prds and all of that plans, all of that stuff. Which is all true, but I think it makes way more sense. And this is the way I've been building our system called Pi, which is now being renamed to life OS. So if you see life OS, that's actually Pi. And the model that we use with life OS is essentially to have your harness be essentially your life. Your AI harness is everything that you're trying

1:57S1:to accomplish. It's all of your intents. It's your current state, it's your desired state. It's what you're actually trying to achieve in life, right? So it's your personal goals, it's your work goals. It's your, you know, coding projects and your apps and your desires and your preferences and your challenges. Now you might think, well, okay, but that's life stuff. And over here we have coding stuff. But I think if you think about this in the way that I'm describing, you'll notice that this all blurs, this all disappears, because what we're trying to do is get to a point where we can say just the smallest thing, okay? We could say, imagine you had a twin sister or a twin brother, and they were like synced with you. You could say to them, and this is like a really important point. You could say to that person, you could say, hey, do you think this works for us? And they could just like glance at, say, an event schedule, or they could glance at like a new vehicle or they could glance at a job description. When I say, hey, do you think this will work for us? I have communicated volumes. Okay.

2:55S1:This this is the whole magic of a harness. Okay. It's it's context engineering. It's prompt engineering. It's a whole bunch of other things that we've been talking about for years. But it's different, okay? It's different. It is compression. It is efficiency of going from a brain thought or a feeling all the way through the translation layers into the prompt engineering, all the way to the AI. The goal is to be able to say as little as possible to achieve the best possible context transfer and intent transfer and intent articulation into your AI. That is the purpose of the harness because it doesn't matter what you're talking about.

3:30S1:It doesn't matter if you say, hey, draw me a butterfly wing. That is beautiful. Okay, that is very abstract. It's very general, it's very non-technical. But there are questions there. What do you mean, a butterfly wing? Like, how big should it be? Like 1 or 2, you know, should I use watercolors? Like, what is a pretty butterfly wing? According to you write this. These questions need to be answered. And if you have that in your harness, because guess what?

3:55S1:You're a famous artist known for drawing butterfly wings. Well, hopefully you wouldn't be using an AI to draw your wing for you if that were the case. Let's just say you're a famous AI artist and you put out different animals or whatever. Whatever it is, you should have so much context inside of your harness of what great art looks like, what great AI art in this case looks like. I don't like this example, but we'll just go with it and essentially like, what are all the pieces?

4:19S1:What does it mean to have a good one? What does it mean to have a pretty one? What do those even mean? How do you make that testable? How do you make that something that the AI that your AI harness can actually hill climb on to achieve a positive result, right? And this is where this whole concept of like bitter pill engineering comes into play. There's multiple things that are built into life OS that are oriented around rethinking what a harness is. So one of them is the concept of ideal state. We're trying to transition from current state to ideal state. In this case, the current state is I don't have a pretty butterfly wing.

4:50S1:The ideal state is I do have a pretty butterfly wing. So you're trying to, you know, you the AI or trying to transition me from current to ideal state, right? That's the same thing. If you're building some complex application, you're trying to start a new business and you need all the paperwork. It's all about current to ideal state. The second piece of that is the document itself. That has to do with pursuing the ideal state. When we build something inside of life OS, we're actually building an ideal state artifact. That ideal state artifact is a generalized version of all the stuff you've heard about before with, you know, specs and prds and plans and, you know, prompts and all of that. This is a single unified document that captures the kind of the journal say like the what the person wants, what they say they want that's captured changes to that are captured in like a changelog.

5:35S1:It's broken out into features and like approaches and stuff like that, which the AI comes up with for you. And then if you make adjustments to that, it's modified. There's an HTML version, there's a markdown version. And basically this is the format that we use to hill climb. So we are giving a goal to our system to hill climb using this ideal state artifact, which is in my opinion, like the future of like a PRD. Think of it as a universal PRD for any task. And that is fundamentally what life OS is based on, right?

6:05S1:Is this concept of pursuing the ideal. So really what we're talking about here is a system that can universally pursue ideal for you, right? We are not code generators. We are not app builders, right? Not fundamentally. It's a task that we are doing. Why are we doing that? Why are we building apps? How much money do we need to make? Are we trying to transition out of our corporate job to have, you know, work for ourselves? Well, then that's the goal, right? So how much monthly recurring revenue are we making? Are we doing a good job?

6:33S1:Are people enjoying this? These are questions that we want to ask, and I shouldn't have to say and set up every single question that I ask to the AI, or set up every single app that I want to build with the AI to describe the stack and describe the goal that I have and describe all of that. No. That is the purpose of the harness. Have all of that already built in? It's all in your context. It's in your various context files, it's in your various tools, it's in your hooks, it's in your prompts, it's in everything.

6:59S1:It's already pre-built. It's already pre-baked into your harness itself. So you could say something like, hey, look, is this a good system? I heard there's a new memory system out for cloud code. Is this as good as ours? I literally just do this. In fact, I'm going to show you. I'm just going to go over here to my system. Hey, what do we think about this thing? This is Gary Tan, a friend of mine, cool guy. And basically we talk about a bunch of harness stuff, but he has like this Google brain thing that 24,000 stars.

7:25S1:A lot of people are using this. It's like a got recipes. It's basically a whole harness, right? With memory and stuff like that. So my question is I want to look at this thing and say, hey, how good is this? Like, can we use it or whatever? Now, if I have a generic harness with not much in it, right. Something that's not like life OS. Well, I'm going to have to explain. Hey there. You know, there are memory systems. We have a memory system. This is a memory system.

7:49S1:I'm trying to figure out how good this is compared to ours. I want you to look across all these different criteria, blah, blah, blah. Right. That's a whole bunch of stuff that is intent engineering, right? That is prompt engineering. It is context engineering. It's like all these names that we've been using. It's all the same thing. It's us trying to get our ideal result right. We have a feeling in our brain of what we mean when we give this to our AI. What we really mean is how good is this? Should we use it? But in order to give a proper prompt for that, you could actually spend half an hour or an hour or two hours writing a great prompt and you would get better results.

8:24S1:The goal of a harness, in my opinion, is to be able to do that automatically with as few words as possible. Right? All right. So what I'm going to do is I'm just going to grab this URL here. Okay, so I got that copied, and I'm going to come over here and I'm going to open a new session right here ready to go. Kai says hello and we're going to say, hey, take a look at this and see if we should use any part of it.

8:44S2:Evaluating G brain repo.

8:46S1:All right. So Kai is going to head out and take a look at this thing. And what's extraordinary about what Kai is about to do is what did I say? I said, hey, take a look at this and see if we should use any part of it. I didn't explain myself. I didn't write a prompt that took me two hours. Kai already knows when it sees the project, it knows okay about itself. Look at these context files. I've included my skills, my life OS system prompt projects, my identity, Kai's identity architecture summary. That's our architecture, life OS all that stuff all unified together so that Kai understands what I mean when I say very simple sentences.

9:25S1:What we just basically did was we cheated. We basically cheated and injected.

9:32S2:Skip the DB, steal gap analysis and hybrid retrieval, both testable. Want me to prototype the gap layer.

9:37S1:Look at that. Look at that. Look at how much it looked at that quickly. Okay. And this is all because it has tons of context built in. I'm already at 13%. And people will say, well, if you have a harness, you're doing all this context injection, you're wasting tokens. No, you're spending tokens. You're not wasting tokens. You're doing quite the opposite. And tons of people who use Pi OS, the new name of it basically will tell you, yes, sometimes it takes a little bit longer. That didn't take long at all. That took one minute, right? 20s. But it can take a little bit longer. But you're saving spending five times more time in tokens, doing rework. I don't have to do rework very often here, and when I do, I just rebuild that into the harness so I don't ever have to do it in the future.

10:18S1:So I really encourage you to think about a harness in this way as a much larger, zoomed out version of life management, which is why we just renamed the Pi basically to life OS, because this is about managing what you are trying to do in your life and what you're trying to do in your work overall, what you're trying to accomplish, what is your ideal state so you can get your AI harness to transition from current state to ideal state. That is the whole purpose. And inside of that, contained inside of that is this concept of intent engineering. Okay. It's intent engineering, sitting on top of context engineering, sitting on top of prompt engineering. And ultimately it comes down to the clearest possible thinking, right?

10:58S1:The clearest possible articulation of what you're trying to accomplish, which is why we have the algorithm, which is why we have the Issa document, ideal state artifact document. We are hill climbing towards what we're trying to accomplish, whether it's drawing a butterfly wing or building a brand new application or like improving our relationships, whatever it is, think at that larger scale. Think of your AI harness as life management, essentially, right? And it doesn't matter what platform you use, doesn't matter if you use our open source thing or you're going to use some new thing that's going to come out from anthropic or OpenAI, it doesn't matter.

11:29S1:So I just want to prompt you to think about harnesses in this much larger way, not AI harnesses, but life OS slash life management slash intent engineering. And we'll see you in the next one.