#301: It Turns Out Analysts Are Natural AI Crafters

0:00[Tim Wilson]: Welcome to the Analytics Power Hour. [Announcer]: Analytics topics covered conversationally and sometimes with explicit language. [Tim Wilson]: Hi everyone, welcome to the Analytics Power Hour. [Tim Wilson]: This is episode 301. [Tim Wilson]: And hey, here's a fun little fact. [Tim Wilson]: Did you know we had the founder of X.AI on this very show? [Tim Wilson]: But it's not what you think. [Tim Wilson]: The date was August 30, 2016.

0:33[Tim Wilson]: The show was episode number 044. [Tim Wilson]: And the topic was artificial intelligence with Dennis Mortensen. [Tim Wilson]: So at the time, X.AI was a company that was just trying to solve [Tim Wilson]: multi-party meeting scheduling via email. [Tim Wilson]: And we talked a lot about like highly specialized agents. [Tim Wilson]: And it was this wild conversation about how these simple things [Tim Wilson]: that we think are simple can actually be pretty complicated to hand over to the machine.

1:02[Tim Wilson]: But that was 2016. [Tim Wilson]: It was a totally different world. [Tim Wilson]: And it feels like that was kind of an ancient ancestor of the AI world we're living in today. [Tim Wilson]: And we are going to talk a lot about AI on this episode. [Tim Wilson]: And I'm going to do that with my co-host. [Tim Wilson]: Julie, Hoyer, you're from further. [Tim Wilson]: Is AI something that comes up at all in your day-to-day? [Julie Hoyer]: If I had a nickel forever, I would have heard AI, right?

1:30[Tim Wilson]: Yep. [Tim Wilson]: That's what I thought. [Tim Wilson]: And Val Kroll, you're from Fin, the company formerly known as Intercom. [Tim Wilson]: Is AI something that your company, I mean, are they doing anything at all with AI? [Val Kroll]: Yeah, we're just starting to dabble, really. [Val Kroll]: Just starting to just get our toes, get our toes up. [Val Kroll]: No, it's top to bottom. [Val Kroll]: Top to bottom here at Fin. [Tim Wilson]: The fundamental grounding of the primary offering.

1:53[Val Kroll]: Yeah. [Tim Wilson]: And I'm Tim Wilson, or maybe I'm not. [Tim Wilson]: And this is just an AI agent that Tim built to record this episode on his behalf. [Tim Wilson]: Who knows? [Tim Wilson]: So sure, all three of us have been exploring and learning and using AI, [Tim Wilson]: but none of us have dived in deeply enough to write a really useful book about it. [Tim Wilson]: And as far as I know, none of us were product leaders [Tim Wilson]: at Microsoft focused on the creation and build out of Power BI.

2:16[Tim Wilson]: But our guest has done both. [Tim Wilson]: Rob Collie is the CEO of P3 Adaptive, [Tim Wilson]: which does custom AI and data work for mid-market and Fortune 1000 companies. [Tim Wilson]: He's a former Microsoft engineer on the Excel team [Tim Wilson]: and a founding engineer on what became Power BI. [Tim Wilson]: He's the host of the raw data with Rob Collie podcast. [Tim Wilson]: And he's the author of an upcoming book, [Tim Wilson]: Fair Game, Customizing AI to Your Business is Easier Than You Think.

2:42[Tim Wilson]: It's not publishing until August 15th, but we got a preview. [Tim Wilson]: It's amazing. [Tim Wilson]: And it's available for pre-order now wherever books are sold, [Tim Wilson]: including fairgamebook.ai, [Tim Wilson]: which gives instant access to the first few chapters and a few other perks. [Tim Wilson]: And today, Rob is our guest. [Tim Wilson]: So welcome to the show, Rob. [Tim Wilson]: Thank you so much. [Tim Wilson]: I've enjoyed this already.

3:06[Tim Wilson]: All right, so we don't usually kick off the show with a, [Tim Wilson]: hey, give us your professional history. [Tim Wilson]: But one of the reasons we really wanted to have you on specifically [Tim Wilson]: is that you're not just kind of any old run-of-the-mill AI thought leader, [Tim Wilson]: you know, half the people who have LinkedIn accounts right now. [Tim Wilson]: You actually kind of came from our people. [Tim Wilson]: And I mentioned in the intro, [Tim Wilson]: but you had a formative part of your career working at Microsoft, [Tim Wilson]: including being part of like the birth of Power BI, [Tim Wilson]: which is inarguably like one of the premier, [Tim Wilson]: most successful BI platforms out there.

3:40[Tim Wilson]: So I think that actually may be a good place to start. [Tim Wilson]: So can you sort of talk about your role, you know, from Microsoft through kind of connect [Tim Wilson]: the dots for us as to how that set you on the trajectory to where you are today [Tim Wilson]: and like like helping organizations figure out like useful paths forward with AI? [Tim Wilson]: I would love to. [Rob Collie]: So I didn't, like most people who love data, I didn't know that I love data.

4:04[Rob Collie]: I didn't grow up saying I want to be a data professional, whatever. [Rob Collie]: I've talked for years and years about how some of us just sort of carry this thing [Rob Collie]: I call the data gene and we eventually discovered that we have it. [Rob Collie]: Like when we collide with Excel or whatever for the first time, [Rob Collie]: most people bounce off like in terror. [Rob Collie]: They don't like Excel, right? Yuck. [Rob Collie]: But like it actually is about one out of 16 people [Rob Collie]: who collides with something like Excel and sticks and goes, [Rob Collie]: ooh, that was kind of cool, you know, or their first collision with SQL.

4:36[Rob Collie]: It's usually Excel. [Rob Collie]: That's the first sticking point, but not always. [Rob Collie]: And so I discovered that I had the data gene in my course of working at Microsoft. [Rob Collie]: In fact, at one point earlier in my Microsoft career, [Rob Collie]: I said to someone that I would never go to work on something like a product like Excel. [Rob Collie]: Never. It was like beneath me. [Rob Collie]: But then I went and I ended up kind of getting re-organized into Excel [Rob Collie]: and discovering, oh my God, this is totally where I needed to be.

5:03[Rob Collie]: And so I got to know a few different really important audiences. [Rob Collie]: I got to know the people who really kind of make the world go around, [Rob Collie]: like the Excel power users, that one in 16 crowd. [Rob Collie]: Back then, even really to this day, they kind of make the world go around [Rob Collie]: in a very kind of almost like thankless way. [Rob Collie]: I got to know those people and really, really, really develop an affinity for them [Rob Collie]: in a way because I'm kind of one of them and I was building products for them.

5:34[Rob Collie]: And then at one point, I also got grabbed to be in charge of the BI investments in Excel. [Rob Collie]: And so that was being dragged into like the corporate IT zone, right? [Rob Collie]: And I learned the old style of BI through that experience. [Rob Collie]: And then later, I was recruited to join the Power BI team [Rob Collie]: when it was still called Project Gemini Super Secret Project. [Rob Collie]: And the contrast between how the old style of BI, most of us who are working in data [Rob Collie]: or working in business intelligence, were never part of the old traditional style BI [Rob Collie]: because it was so insular, it was so arcane.

6:25[Rob Collie]: It was a very tiny priesthood that was involved in this stuff. [Rob Collie]: And I couldn't even learn how to do it. [Rob Collie]: I was working on it and I couldn't learn how to write the formulas in these old systems. [Rob Collie]: So helping to build the Power BI, which was aimed at that original Excel crowd, [Rob Collie]: those people that I like so much, the data gene crowd and giving them industrial strength tools, [Rob Collie]: I never thought that was going to work. I love the project. It was exciting.

6:53[Rob Collie]: It was definitely the place that I needed to work at Microsoft when they came to pitch me. [Rob Collie]: But like everything else, I've just realized, this is probably going to suck. [Rob Collie]: Or maybe it'll take three versions to get it right. [Rob Collie]: And then I was shocked at how good version one was. I mean, it was really fucking good. [Rob Collie]: And I knew the world was going to change, but I also knew that consulting companies of the time [Tim Wilson]: were not going to want to work this way. They were not going to want to move at that pace.

7:24[Rob Collie]: They were not going to want to work with clients of different sizes and pick up smaller, [Rob Collie]: faster projects, which is what the world really needed. And that's what led me to start the [Rob Collie]: company that I did. I wanted to go fill that niche. And guess what? I got to hire those types [Rob Collie]: of people that I liked so much into these new consulting roles. They became consultants. [Rob Collie]: And that's what our company is largely staffed by, not 100%, but largely staffed by that data [Rob Collie]: gene crowd that grew up in the business more so than in IT. And so it's sort of like I got to [Rob Collie]: create a home for some of my favorite people at the same time. It was really kind of hard to pass [Tim Wilson]: up. So that's the first part of that journey. And then, I mean, the link you make in the book [Tim Wilson]: is that those are the people that you say they're kind of crafters and why crafters [Tim Wilson]: specifically are uniquely suited to do stuff with AI. So maybe that's my follow on, [Tim Wilson]: is can you talk sort of about like the leaders, crafters, developers,

8:35[Tim Wilson]: knowledge workers, and why that's an AI thing? Sure. Yeah, absolutely. And first, [Tim Wilson]: let's just bridge one tiny little gap, which is like when I, I didn't set out to write a book [Rob Collie]: about AI. So, you know, I'm a CEO of a data company, you know, 50 plus person data company, [Rob Collie]: and I see AI coming. And it was leading to this sense of dread, which I think we can all kind of [Rob Collie]: relate to, right? Like AI is really good at writing code, you know, it's actually really good at [Tim Wilson]: doing a lot of the things that data professionals do for a living. I went on a research project [Rob Collie]: with my own sort of like journey down the rabbit hole to figure out what the future of our company [Rob Collie]: should be. This was about survival. Like what, you know, I had to do this. And what I found [Rob Collie]: down that rabbit hole was far different than what I expected. And far more encouraging and far more [Rob Collie]: like, oh, we can do this sort of thing. And far more, oh, this is actually a data problem.

9:41[Rob Collie]: Like it's like right in our wheelhouse. And the difference of sort of like [Rob Collie]: feeling like AI was like opaque and inaccessible, going from that to going, oh, that's it. [Rob Collie]: That feeling of like, oh, that's it. Like that feeling is what compelled me to write a book. [Rob Collie]: Because when I found myself in possession of that feeling, which by the way is exactly what [Rob Collie]: led me to write the first Power BI books that I wrote, you know, 10 plus years ago, [Rob Collie]: like me feeling like I understood something and knowing that there wasn't anything all that [Rob Collie]: necessarily special about me, like the rest of the world can do this too. Like I felt like I owed [Rob Collie]: that book. And I felt the same sort of the same sort of vibes again, but like at a greater magnitude [Tim Wilson]: this time, because like everyone's worried about AI. But yeah, so back to your question.

10:32[Rob Collie]: You know, I've been calling these people the data gene crowd forever. But you know, what they really [Rob Collie]: are, and I think present company probably is, you know, it's we, right? It's not they. [Rob Collie]: You know, what we really are are people who are willing to think systematically about business [Rob Collie]: problems, while also remaining close to the business and understanding that, you know, [Rob Collie]: there's the technologists that kind of like want to be like locked in a closet and have like notes [Rob Collie]: passed under the door of what to build, right? And that's that's a lot of tech, right? And it's [Rob Collie]: but it's a very and you need those people. But that's a very inefficient way to work for most [Rob Collie]: business solutions. Like being immersed in the business, while at the same time having some [Rob Collie]: tech skills, enables a completely different style of work, a completely different tempo and a [Rob Collie]: completely different quality of output. But with the with with AI coming along and the ability for [Rob Collie]: people like us, people who picked up Excel and smiled, right? Like, when you sit down with people

11:43[Rob Collie]: like us sit down with something like Claude Code for the first time, we sort of like get over our [Rob Collie]: fear of trying something new. It's that same feeling like captivating feeling like, Oh my God. [Rob Collie]: And so I needed to come up with a new name for these people. It's no longer the data gene crowd. [Rob Collie]: I needed a noun for these people in the book. So I had to come up with something and crafter is [Rob Collie]: where I landed. So that leaders, crafters and developers and ratio to 580. And so for every, [Rob Collie]: if you have a if it just works out that way, if you had a population of 87 people who work in an [Rob Collie]: office, you know, like two of them are developers, five of them are crafters, and 80 of them are [Rob Collie]: knowledge workers of various, various flavors. So like only so five relative to 80, like that's [Rob Collie]: how many people again, one to 16 are willing to engage with something like Excel. And what I'm [Rob Collie]: seeing is that that same data gene crowd, plus the ability to use coding agents, AI coding agents,

12:51[Rob Collie]: to build custom software. I mean, this is like going to be the new spreadsheets. You know, we [Tim Wilson]: application development is now within range for this crafter crowd. And we're just in the earliest [Rob Collie]: stages of the world waking up to this, like most crafters haven't yet discovered that they can do [Rob Collie]: this. But we're at steady state, we're going to have about the same number of people building [Tim Wilson]: software as we had building spreadsheets. Michael, how does your team share AI generated analyses [Michael Helbling]: today professionally? We say, hold on, I think I asked Claude this last week and then sort of [Michael Helbling]: disappear into a chat thread like a raccoon in an air duct. Oh, nice. That's that's elegant. That's [Tim Wilson]: clearly scalable. I mean, actually, that's horrifying. Why thank you. It's called innovation.

13:43[Tim Wilson]: That's the type of innovation that ask why is trying to solve with Prism. Your AI work shouldn't [Michael Helbling]: be trapped in one person's called innovation. I have started to realize my private chat history is [Michael Helbling]: not a data strategy. It's kind of a joke drawer with confidence. Well, Prism helps preserve context [Tim Wilson]: and memory across users, you know, like metric definitions and source of truth tables, business [Michael Helbling]: rules, prior analyses. So when I teach it, a qualified lead excludes internal traffic, spam [Michael Helbling]: forums and that one campaign we don't talk about, the team can build from that shared context instead [Michael Helbling]: of starting over entirely. Beautiful. Let's ask Michael what he meant, more the system actually [Tim Wilson]: remembers. And with Prism, analyses are organized, traceable, auditable and reusable with your other [Michael Helbling]: data sets. I like it. So my best AI work becomes company knowledge, not a screenshot named. Is [Michael Helbling]: this important? Maybe.png. Exactly. I guess it's time to stop being a data trash panda when it comes

14:45[Tim Wilson]: to AI. Definitely. No pandas in this context. Go to ask dash y.ai and sign up for the wait list. [Michael Helbling]: Use code APH to jump to the top of that list. That's ask dash the letter y.ai code APH. [Michael Helbling]: Because check my cloud history is not a collaboration plan. Tim, one of the trickiest [Michael Helbling]: parts of measurement is user data gaps. Right. Someone gives you an email in one session, [Tim Wilson]: comes back later from another device or converts offline and suddenly the customer journey gets [Michael Helbling]: hard to connect. Exactly. Staped and richer is a state power up that stores selected data from [Michael Helbling]: incoming requests and uses it later to enrich future events when information is missing.

15:32[Tim Wilson]: So once a user has been identified, and richer can help recognize them in later events and add [Tim Wilson]: previously collected details, all while aligning with consent settings and handling personal data [Michael Helbling]: securely. Yeah, it's especially useful for cross device journeys, offline conversions, longer [Michael Helbling]: buying cycles and connecting CRM outcomes back to earlier website activity. You can set up [Michael Helbling]: and richer to work with cookies, state store or both. And it supports uploads of historical data [Michael Helbling]: from systems like Shopify, Clavio or your CRM. So if you're tracking has gaps and who's doesn't, [Michael Helbling]: Staped and richer can help create more complete consistent event data. Yeah, go to state.io [Michael Helbling]: to learn more about Staped and richer, improve event match quality, attribution and visibility [Michael Helbling]: into the full customer journey. That's Staped.io, more complete data with consent and privacy in mind.

16:28[Val Kroll]: Besides like the just the usual FUD, what do you think are some of the barriers to crafters? [Val Kroll]: Because like I remember where I was the first time I saw a pivot table and it was like, [Val Kroll]: blew me back in my chair. I was high school, but of like, what is holding, what do you think are [Val Kroll]: some of the major barriers that are holding up the crafter crew back? Well, here's one quick funny [Rob Collie]: story before I answer your question. I worked on the Excel team for a full year [Rob Collie]: before I knew what pivot tables were used for. And I love that. I would fake it be like, yeah, [Rob Collie]: you could throw this into a chart, you could throw it into a pivot, you know, I would just say things [Rob Collie]: like that without ever you really even knowing what it was. Just like, fake it. No, that's what [Rob Collie]: all the cool kids were saying. So I would say that, right? And then one day, [Rob Collie]: so for you to see a pivot table in high school, that's awesome. That is really, really cool.

17:24[Rob Collie]: So the thing about Excel is that, you know, is that it was so, I mean, I think the most people [Rob Collie]: are even even people who end up liking Excel are scared of it at first. You know, thinking back, [Rob Collie]: I think I was, you know, and, and even when I worked at Microsoft, like every couple of years, [Rob Collie]: people would move around between teams and office, you know, it's really an open season [Rob Collie]: recruiting. I tried to get people to come to work on Excel and they, they wouldn't want to.

17:53[Rob Collie]: They would want to go work on outlook and word and PowerPoint because they already [Rob Collie]: understood those things. But they could sense that Excel was deeper, you know, [Rob Collie]: Excel is an application development platform, you know, it's a spreadsheet is an application, [Rob Collie]: you know. And so, like, there's something intimidating about it. I personally, and I say [Rob Collie]: this in the book, I don't, I don't pick up new tech readily. I don't, I'm not an enthusiastic, [Rob Collie]: like, ooh, shiny, go grab it off the shelf kind of person, like I kind of have to force myself, [Rob Collie]: you know. And it's really just sort of like not even knowing where to start.

18:32[Rob Collie]: And like, how do you even get it installed? Like, even just going and installing something [Rob Collie]: like like cloud code, right? Like, at least it used to be harder than it is now, it's a lot easier. [Tim Wilson]: But and not understanding the nature of the system you're working with, like, [Rob Collie]: it just feels like magic. Like, so you're putting so much trust in something else [Rob Collie]: that's that's that feels opaque to you. I think there's a sort of like a like a sense of loss [Rob Collie]: of control that that is that is terrifying. And I'm seeing this a little bit with our own team, [Rob Collie]: like, you know, not as much anymore, but in the early going, they're like, we derive such confidence [Rob Collie]: from having touched every last single piece of logic, you know, and like artistinally crafted [Rob Collie]: it, right? Like, like, how can we delegate parts of that to another system, you know?

19:26[Rob Collie]: So there's a number of things like that. But for me, honestly, it was just kind of getting off [Rob Collie]: the starting line. I just had to sit down and build something. I have a little bit of a theory, [Tim Wilson]: and it bridges those worlds. And it's interesting when you say, you know, PowerPoint and Word, [Tim Wilson]: and Outlook and Excel. And I don't think I'd really thought about it. If I'm doing something in [Tim Wilson]: PowerPoint, I'm making slides, I know I am constrained to this thing of slides and Word, [Tim Wilson]: same thing with Excel, I feel like the people with the data gene, the crafters, [Tim Wilson]: somewhere along the way, they go from it's a table where I'm doing formulas to Oh, wait a minute, [Tim Wilson]: I can actually build an interactive app, you know, I can put slicers or, you know, I can put [Tim Wilson]: data validation and put drop downs in and I can, I can, I can kind of build a data product, which I [Tim Wilson]: think is one of the reasons Power BI is successful is because there's the progression through [Tim Wilson]: up to Power BI. And I think the same thing happens with any BI platform is you have

20:30[Tim Wilson]: that somebody with the data gene saying I'm building, I'm crafting an experience. [Tim Wilson]: And to me, what I think might be happening, and this was purely from actually reading [Tim Wilson]: the book that this little light bulb went on, we are all introduced to this new world of AI [Tim Wilson]: through the chat interface. And there are plenty of people out there saying it's not, [Tim Wilson]: it's more than just the chat, you got to build agents, you got to do all this other stuff. But [Tim Wilson]: there are people saying that it feels like it's, it's this other thing far away.

21:03[Tim Wilson]: But really where it's close is saying, okay, if you just step out of the chat and say, let me, [Tim Wilson]: let me use the chat to help me build a data product, let me use the chat to help me, [Tim Wilson]: like, create something like that's where that, that parallel. [Tim Wilson]: Okay, yeah. [Tim Wilson]: You know, I started to, to see because it just that light bulb hadn't gone on that, oh, I can [Tim Wilson]: create a deliverable for my audience, for my user. And I've now got a broader set of things [Tim Wilson]: in my fingertips, just like going from, I got to hack around and excel to build an interactive [Tim Wilson]: dashboard, I go to Power BI, I've now got a deeper, a lot of these things, I don't have to [Tim Wilson]: come up with the indirect formula in a data validation dropdown to make this filter work.

21:50[Tim Wilson]: It's just natively there. [Rob Collie]: I think you just nailed it. I think you nailed it. And I didn't, and I'm not saying that like, [Rob Collie]: that's exactly what I was thinking. I'm saying that like, I think you just gave an answer that I [Rob Collie]: didn't have. But I think it's the, I think it's a really good point. So there's a, there's a mixture [Rob Collie]: of kind of like gaps you have to, you have to hop, right? Like, you know, smooth climbing curves are [Rob Collie]: one thing, smooth climbing learning curves are one thing, but having to like jump from point to [Rob Collie]: point, like over like a, like a void, you know, those are the places that get us. And there's [Tim Wilson]: two big voids here. One of them is that, you know, we, we sit down and we use the off the shelf AI [Rob Collie]: systems. This is a recurring theme in the book is that the, the off the shelf AI systems are [Rob Collie]: amazing for like personal use. They're amazing for individual use. They know everything about [Rob Collie]: like all of human history. It's insane. I was using Claude the other day and it was telling me

22:48[Rob Collie]: about the geologic construction of the ground underneath my house here in Seattle. It, without [Rob Collie]: even having to search the web, it didn't have to search the web. It knew what the geographic [Rob Collie]: makeup and so we were calculating the speed of sound in the ground near my house and it didn't [Val Kroll]: have to search the web. You've got the data, Gene. It was family night. Okay. Look, there's a [Rob Collie]: power driver driving, driving a post into the lake near our, near our house and the sound arrives [Rob Collie]: in a two syllable thump. You get a thump through the ground and then you get the clank through the [Rob Collie]: air. It's one thing, but because the speed of sound in the ground is faster, the ground wave [Rob Collie]: gets here and I was measuring the delta between the, the time between the two arrive and using [Tim Wilson]: that to calculate the distance to how far away the pile driver is. And yes, there's something [Rob Collie]: that the whole family is aware is going on. Like dad's calculating this is like, it's two, [Rob Collie]: it's, it's 200 meters plus or minus 15. But anyway, so like, like, but to think about it,

23:53[Rob Collie]: like that is publicly available knowledge that's going into this LLM. [Rob Collie]: You know, but like if it's about my business, and in the book, I show an example of this, [Rob Collie]: like I just have it like write a proposal, write me a proposal for this project. [Rob Collie]: It goes and tries to do it, but it knows nothing about us. It, it just gives the lowest common [Rob Collie]: denominator like a proposal and it is completely worthless. It is awful. And so there's that, [Rob Collie]: there's that gap between business use and personal use, which is like, again, it's, it's, [Rob Collie]: it's this void, right? Like it's, it's not a, you have to jump it all at once.

24:32[Tim Wilson]: Well, it's the same, it's the same void of saying, I know I'm not supposed to put [Tim Wilson]: private data into this if I'm using a public or I, I'm like, I've been terrified that I'm [Tim Wilson]: going to put something in this going to be a problem. But if I just ask her to do something [Tim Wilson]: without it, so how do I put enough stuff in? And it's, that feels like the other connecting [Tim Wilson]: the dots, which again, I mean, yes, yes. So there's that sycophantic Tim, so maybe I am [Tim Wilson]: just an AI agent. I'm like, your book breaks out the different pieces and how to think of them.

25:00[Tim Wilson]: But that feels like it's again, it's like, you're using chat GPT and somebody's like, [Rob Collie]: but don't, don't give it any of your information. And so then this is the second void is that [Rob Collie]: you have to get comfortable building real, regular software, like stuff that's written in [Rob Collie]: Python or whatever, which is not what data people have traditionally done. Like we've [Rob Collie]: worked in ETL, we've worked in formulas, we've worked in data pipelines and things of that sort.

25:29[Rob Collie]: But until you can write your own custom code, you can't build the rest of the system around [Rob Collie]: the LLM to customize it. If you're in charge of that software, then you're also in charge of [Rob Collie]: the storage and you're in charge of like, which version of the API you call that is you can call [Rob Collie]: ones that are that are actually more protected and that aren't that they don't train on your [Rob Collie]: on your data. So you can you can candle that sort of sensitivity. But you have to go from [Rob Collie]: being a non software developer to being a software developer in order to get this customized [Rob Collie]: experience where you can control the flow of information and you can give context about [Rob Collie]: your business at the right time to the LLM. So you have to sort of jump both of these voids [Rob Collie]: at the same time to get started. And this is why like in the appendices of the book, the first [Tim Wilson]: thing you just did one of these the other night, I think, Tim, right? So the first thing I have [Rob Collie]: people do if they've never used cloud code or codex or whatever, like it's just build a website

26:35[Rob Collie]: on their computer. That's just like a survey or a quiz that gives a score or whatever, [Rob Collie]: just get used to the idea of building anything. No, no AI agent stuff, just get just start to [Rob Collie]: get more comfortable with building free form software. And then we start to layer in and [Rob Collie]: the next exercise, like, okay, now we're going to add some LLM to it and see, you know, where it [Tim Wilson]: goes from there. So I'll throw in now. This is I may regret this, but for shits and giggles, [Tim Wilson]: what I wound up doing was a little 15 question multiple choice quiz about my wife's and my [Tim Wilson]: relationship and about us that I sent to my three 20 plus kids, 20 adult children.

27:16[Tim Wilson]: I was like three 20 plus. No, three 23 kids who are all adults about like, it's like just family [Tim Wilson]: lower stuff, which was like a prompt. But it was fun standalone. I did wind up having to throw it [Tim Wilson]: on Netlify. So it was like, I would like one part of my like, Oh, I already have a Netlify account, [Tim Wilson]: I'll throw it there. But it's now, of course, gone to where now my parents have taken it. I mean, [Tim Wilson]: my kids do the best, but they've also had a few like, what you did? I mean, so, you know what, [Tim Wilson]: I'll throw it into it in the show notes and anybody terribly. Yeah. Why not? How well do [Tim Wilson]: you know, Tim Wilson's marriage? Yeah, I mean, it's going to be referred, it's going to refer to [Tim Wilson]: mom and dad. And that's going to, you're going to have to recognize that. Okay, I'm just going to [Tim Wilson]: have to be okay with it. Anyway, it was a great episode. But just, I mean, throw in that, that [Tim Wilson]: was like the little unlock was even though I've done stuff with co-work, even though I've done

28:09[Tim Wilson]: stuff with gyms, even though I've, I feel like I've done a lot, there was a piece by the time I [Tim Wilson]: got to the end of the book of saying, Oh, no, no, no, there's a software component where what I'm [Tim Wilson]: developing with limitations, right? Because you also talk about where you still need developers. [Tim Wilson]: This is not now you're turning the people with the data gene, the crafters into full blown, [Tim Wilson]: push stuff into production to hit the masses. The developers now are still needed for their [Rob Collie]: skills. Yeah. So, you know, we P three, we've hired this year, our first two full stack developers [Rob Collie]: ever. And it's kind of funny, like, in fact, I was even in Fortune magazine at one point, [Rob Collie]: talking about this, because, like, big tech is laying off developers, because, because AI is [Rob Collie]: so good at writing code, in fact, it's better writing code than anything else. At the same time, [Rob Collie]: though, while it's expanding the capacity of an individual developer to do so much more, [Rob Collie]: that makes their ROI pencil out much differently. And so, like, you know, in order to achieve what

29:21[Rob Collie]: we're achieving today with two developers who are all in on using AI to write code, [Rob Collie]: we're getting the results of, like, honestly, like, what are probably required a team of 10 [Tim Wilson]: developers to do before. So, I call this sort of like the migration to the suburbs [Rob Collie]: for developers. Like, so, like, like, there's been a concentration of developer talent in [Rob Collie]: big tech firms, because they build huge amounts of technology and they require engineering teams [Rob Collie]: of, like, 30 developers at a time. And I don't think they're going to require 30 developers to do [Tim Wilson]: the same sorts of things anymore. At the same time, though, the ROI on hiring a developer, [Rob Collie]: like, in a smaller company, or a department level of an enterprise or whatever, is going to be higher [Rob Collie]: than it used to be. So these people are going to have other jobs they can plug into. They're just [Rob Collie]: going to be in different places. And, you know, so, like, you know, the dev team of two that we [Rob Collie]: have here, we don't need to go too deep. And just to sort of underline it, like,

30:28[Rob Collie]: where we've built things that put us at probably about a year ahead of Microsoft [Tim Wilson]: in terms of offerings that Microsoft will have. It's like, we're able to do things for our clients [Rob Collie]: today with AI around Power BI and things like that, that, you know, Microsoft is absolutely [Tim Wilson]: building. But they're not there yet. And so, you know, it gives us an opportunity to fill some [Rob Collie]: gaps there in the meantime. And, like, that is just absolutely bananas. A year ago, me, [Rob Collie]: would never have guessed that that was a possibility.

31:02[Tim Wilson]: You would have said the tool that's not doable, even with our DAX [Tim Wilson]: amazingness, we can't do that. We'll have to wait until Microsoft rolls that out. I mean, [Tim Wilson]: or you'd find an alternative way to come at it. [Rob Collie]: No, I'm thinking it's more like I just hadn't realized how valuable developers were going to be. [Rob Collie]: Like, like, there's a difference between what a crafter, like, so I think of myself, [Rob Collie]: I'm 100% a crafter. I'm not a developer. And I talk about there's a form of discipline [Rob Collie]: that's present in a developer. These are mostly personality traits as opposed to, like, [Rob Collie]: intellectual IQ measurements. There's a disciplined patience in a developer, [Rob Collie]: combined with, like, an aggressive modularization instinct that isn't really present in most [Rob Collie]: crafters. It's not present in me. And so, the things that our dev team of two have built [Rob Collie]: are things that our crafter team would not have been able to build.

32:08[Rob Collie]: They're just levels of complexity. And even just knowing what to build and figuring out, [Rob Collie]: like, what the roadmap should look like would have just taken us too long. [Rob Collie]: The developer brain is still really useful. And so, one of the things we're working on [Rob Collie]: on our team is figuring out how to hybridize developers and crafters [Rob Collie]: to give the best possible results to our clients. Because most developers, and I'm painting with [Rob Collie]: the broad brush, are most developers aren't going to want to be the customer-facing interface [Rob Collie]: that does all that conversation. Like, they want to be writing code. Even if they're capable of [Rob Collie]: having those conversations, they love writing code, you know? And meetings take them away from that.

32:53[Tim Wilson]: But crafters thrive on interacting with the stakeholders, right? Like, that's the world [Rob Collie]: that they came from. They want to be right there, you know, like, almost like pressed up against [Rob Collie]: the audience that they're helping. And so, but there are certain kinds of code, [Rob Collie]: certain kinds of platforms, et cetera, that, like, you really, you're still going to want [Rob Collie]: developers to do those things, even though we're building solutions at the crafter level. And I [Tim Wilson]: do talk a lot in the book about trying to paint pictures of where those boundaries are, [Rob Collie]: but they're incredibly fluid, you know? Like, it's like, you can't paint a precise picture.

33:32[Julie Hoyer]: I was going to ask you to clarify, like, when we're talking about crafters, [Julie Hoyer]: are we, in my brain, and I feel like maybe some of this wording comes from, like, working at [Julie Hoyer]: further and search discovery, like, we always talk to him and about, like, our titles, [Julie Hoyer]: or maybe a little different than somehow, like, the industry talked about them. But in my head, [Julie Hoyer]: right, crafters, the way you mentioned it, Rob, to me, I would put in, like, this engineering group [Julie Hoyer]: that's more like building the back end of, like, a BI tool, and, like, surfacing the data type [Julie Hoyer]: engineering role, whereas I came up through more of, like, the classic, like, analytics role. We [Julie Hoyer]: kind of talk about them as, like, functional analysts, very tied to, like, the business. I [Julie Hoyer]: could, like, build a dashboard, of course, but I wasn't doing, like, the ETLs in the background.

34:18[Julie Hoyer]: I was doing more of the, like, business, what are you asking? What is your problem? What's your [Julie Hoyer]: hypothesis? Let me actually go manipulate the data, build something in R, right? Like, a model, [Julie Hoyer]: do an analysis, and then bring it back to you. In your framing is a crafter, both of those things, [Julie Hoyer]: because when you say, crafters need to become software developers, and, like, the way of using [Julie Hoyer]: AI, I'm just curious, are you talking more what I'm saying are engineers, or are you including [Julie Hoyer]: this, like, functional analyst group as well? So I think I should very much clarify that I'm using [Rob Collie]: a very Power BI-centric lens for this, because, you know, our company's all been all about Power BI.

35:00[Rob Collie]: You know, we've long said that we will use another tool if something came along that we like better. [Rob Collie]: But so, but we've been, we've been a Microsoft stack company, and it's not because of Microsoft [Rob Collie]: patriotism. Like, I literally formed a company around this thing that I saw. [Tim Wilson]: The Table of Army is going to be coming for this. The comments are going to be... [Rob Collie]: Well, I am, I am ready. I am ready for that. When we talk briefly about semantic models, [Rob Collie]: I'm really ready. So...

35:29[Rob Collie]: And the Domo people too, but that's only going to be like three people. So... [Rob Collie]: I know. Yeah. I mean, you just defarm them. [Rob Collie]: The people that I'm talking about have kind of always sat on the boundary between the business [Rob Collie]: and IT, but they've lived mostly in business. So they, you know, like they reported up through, [Rob Collie]: you know, some non-IT function. You know, they're the shadow IT folks.

35:56[Rob Collie]: And so, you know, the role you're describing is sort of like the business analyst that does all [Rob Collie]: the, interacts with the business and sort of figures out what the needs are. The people who do that, [Tim Wilson]: but then also go and like build the spreadsheet, you know, that addresses it, or in our world, [Rob Collie]: then went and discovered Power BI and started doing the ETL and started, [Rob Collie]: in sort of doing the data modeling and writing the DAX. That's the, you know, that's the, that's [Rob Collie]: my personal like, like center of gravity for all of this. So at our company, we talk about it as [Rob Collie]: like the decathlete model. So like most consulting firms, the way they've been constructed over the [Rob Collie]: years is they have specialists. They've got someone who throws the javelin. They've got someone who [Rob Collie]: runs the hundred meters. They've got, right. And so this team of 10 rolls in there on the engagement [Rob Collie]: and you pay a tremendous amount of communication costs between those 10 people. Moest of the elapsed

36:57[Rob Collie]: time of the project is actually the communication between the members of the team. It's like 99% [Rob Collie]: of the weight of the project is communication costs and not implementation. And so our whole [Rob Collie]: business model from the beginning was built around this idea that, no, no, like you, you need to be [Rob Collie]: the decathlete. You're the one that talks to the customer. You're the one that understands their [Rob Collie]: needs. You're the one that builds the ETL. You're the one that builds the data model. You're the one [Rob Collie]: that builds the dashboards and all that community, your bandwidth within your own head is just so [Rob Collie]: much faster than the bandwidth between people. And by the way, also less error prone. And so [Rob Collie]: at least again, speaking through the lens of my company, the company that we built, [Rob Collie]: this is the persona that's kind of like, like at ground zero for it. And so I think more on the [Rob Collie]: business analyst side than on the quote unquote engineering side. But, but in the Power BI world, [Rob Collie]: a lot of those business analyst types acquired these skills that had traditionally been the

37:59[Rob Collie]: purview of the back office IT types, right? Like we were invading their ETL world and invading [Rob Collie]: their data modeling world, right? Like, you know, tearing down that ivory tower. [Tim Wilson]: I feel like it's the media agency that would say, oh, we manually update this spreadsheet and we put [Tim Wilson]: our dates across in the columns and it just makes our skin crawl with the structure of it. And they [Tim Wilson]: manually date it and they put some hacky formulas and since some sort of garbage over the wall to [Tim Wilson]: the marketer. And then an analyst gets in and or somebody with the data gene gets in and says, [Tim Wilson]: this is error prone. It's not really giving the marketer what they're trying to ask. So I'm going [Tim Wilson]: to figure out a way to maybe it's VBA, you know, maybe it's Google, maybe it's Apps Script, because [Tim Wilson]: we're doing in Google Sheets. I'm going to figure out a way that we streamline the process and [Tim Wilson]: deliver something that is more aggregated, more useful. Oh, and I'm going to put a layer of [Tim Wilson]: interactivity on it. And I'm not going to be able to stop. And then suddenly I'm the person who

39:03[Tim Wilson]: the digital marketers come to and they say, can you make one of those spreadsheets? And like, [Tim Wilson]: that's where we've crossed over and we're like, wait a minute, are we are we developers? Like, [Tim Wilson]: no, you're still not a developer. You're a, but I also feel like it's where analytics engineers [Tim Wilson]: arose. That is a whole field that came out of analysts who said, damn it, I can't keep waiting [Tim Wilson]: for the engineering team. Every time I want to change the ETL, I got, I'm just going to build [Tim Wilson]: myself a little sandbox. And all of a sudden that became an entire position or a total role of [Tim Wilson]: where the analyst just kind of gets better. And the point I think that, that like in the [Tim Wilson]: light bulb that went on for me was like, oh, wow, I would never consider building a front end. I would [Tim Wilson]: never consider something that limits the rules that is actually software. But guess what? Those [Tim Wilson]: same people with a little bit more knowledge and a little bit of practice can actually extend it [Tim Wilson]: with a boundary of where does it cross over to a developer, but they can actually,

40:09[Tim Wilson]: they have this other thing that we've never even thought of as being in our arsenal. [Val Kroll]: It seems like one of the, the first steps to someone kind of seeing for these crafters have [Val Kroll]: their eyes open is to get out of chat, um, to start building. And I think some of those examples [Val Kroll]: you guys talked about was great, but something you touched upon earlier Robin obviously got [Val Kroll]: into it deeply in the book is thinking about how do you give it the context about your business [Val Kroll]: that is required in order for it to actually be the partner for you that it needs versus just [Val Kroll]: like the generic off the shelf, like how would an individual contributor think about [Val Kroll]: going about that in a way that's productive for them in role? Like do you have any advice for [Val Kroll]: taking that leap, which feels like another step? Yeah, I mean, the, what's really amazing is just [Rob Collie]: how much smarter even the LLM seem to get when you give them something to go on.

41:02[Rob Collie]: So the first example that I talk about in the book is this concept of what I call a handbook. [Rob Collie]: Um, so if you, if you had a new hire, uh, and one of the metaphors I use is, um, [Rob Collie]: an LLM is basically something that has a, especially the frontier new LLMs, they basically [Rob Collie]: have PhDs in every human subject, every, everything that you could get a PhD in the LLM has that level [Rob Collie]: of knowledge and even ability to reason to a certain extent in all of those subjects, [Rob Collie]: but it shows up every day knowing nothing about your business. And if you thought of it as a person [Rob Collie]: for a moment that showed up with all of that skill, but knowing nothing about your business, [Rob Collie]: they would be relatively useless to you. You know, like you don't care about the history of the Roman [Rob Collie]: empire, uh, most of the time in your business, right? You don't care about the ground speed [Rob Collie]: of sound underneath Moett Lake Seattle most of the time in your business, right? Um, [Rob Collie]: but you do have policies and a brand framework and workflows and things like that. And so like,

42:06[Rob Collie]: if you were going to hand, um, um, a new hire, some sort of handbook and instruction manual for [Rob Collie]: their job, uh, it would be written in English, like if, if you're in the United States anyway, [Tim Wilson]: right? Like, you know, it'd be written in natural human language. That same sort of document can [Tim Wilson]: be handed to an LLM every time it wakes up. And if you, if you, again, now you're writing custom [Rob Collie]: code, right? So you have, you've had to take control and you're not, you're not using the chat [Rob Collie]: GPT built in interface or the off the shelf interface anymore. Like you're sort of like [Rob Collie]: taking control of it. You're building your own software. Um, but it's not that hard. It really [Rob Collie]: isn't. And your, your software, the first thing it does when it, when your software wakes up and [Rob Collie]: someone asks it a question or something like that, let's say you're building your own chat bot, [Rob Collie]: the first time the chat bot wakes up every time it goes and grabs that handbook and feeds the [Rob Collie]: handbook to the LLM saying, here's what you're doing for us today. Here's your personality,

43:06[Rob Collie]: here's the rules that you follow. And that lands in the LLM's brain before any question from the [Rob Collie]: user does. And suddenly it goes from being this off the shelf useless thing that can't help you [Rob Collie]: with your business because it doesn't know your business to being a world beater. Um, and if you [Rob Collie]: just think about that for a moment, just pushing some instructions and some context, [Rob Collie]: just like some information about your business. Who are we? What do we do? Like what's, what's [Rob Collie]: your corner of the world here? And each of these agents you build like this, they become specialists [Rob Collie]: in that one role, right? You don't, you don't, um, I talk about like in the book, you don't, [Tim Wilson]: you're never going to build a super being that knows everything about your business because [Rob Collie]: everything about your business can't fit, believe it or not, it can't fit into the LLM's memory.

43:59[Rob Collie]: It will over, it will overload it. So you have to give it narrow lanes. [Rob Collie]: Um, and you know, so like if you have like, you know, but you can imagine agents with five [Rob Collie]: different roles, like I've got one that helps me write marketing copy. I created an editor [Rob Collie]: to help me with my book. Um, and they have different instructions, right? You know, [Rob Collie]: one of them is, is instructed how to write like our company. And the other one is instructed [Rob Collie]: not to write anything for me. The other one's instructed to help hold me to my standard of [Rob Collie]: writing. And when it makes suggestions on fixes, don't make suggestions that make it sound like [Rob Collie]: boring business voice. Remember that I'm still trying to sound like me when you're criticizing [Rob Collie]: my work. So like, don't, for example, don't, I don't want my editor to ever say to me, [Rob Collie]: Hey, Rob, you start too many sentences with the word and or but, you know, that's just how I write, [Rob Collie]: you know, so shut up about that. We're not going to do it that way, you know.

45:06[Rob Collie]: So this idea of handbooks is really like the, the first place to start. But then over time, [Rob Collie]: you start to give these agents, and again, you're just giving the LLM sort of a menu, [Tim Wilson]: you give it the ability to pull information that it wants. Because you can't sometimes, [Rob Collie]: you don't, you can't know in advance what information it needs. So you give the LLM [Rob Collie]: some ability to go and pull information it needs. And by the way, we've all seen this in action [Rob Collie]: already. Every time you go to an off the shelf LLM product and you ask it a question, [Rob Collie]: when it switches off and does a web search, it's doing exactly that. It's recognizing that [Rob Collie]: it needs more information from elsewhere and triggering something that's called a tool.

45:52[Rob Collie]: And the tool is just, it's something that literally just sends a chat request to the tool, [Rob Collie]: the LLM does, and then the out that the tool goes and performs the web search and returns [Rob Collie]: the results back to the LLM. And you can do that with your own internal databases. [Rob Collie]: You can do that with your own internal knowledge bases, which is crazy. [Julie Hoyer]: So for a very specific example, it immediately when I think of like AI and use of analytics, [Julie Hoyer]: you hear about every tool putting in AI chatbot or AI agent on top of their BI tool, right? And [Julie Hoyer]: it's supposed to like be so great. And everyone's got to have had experience with it. Yeah. And [Julie Hoyer]: as most people that actually go in and use them, they don't know anything about your business in [Julie Hoyer]: there. And I have not found a helpful one so far. And so then of course, that gets us into the idea [Julie Hoyer]: of like, Oh, it needs a semantic layer, it needs context. So I'm kind of like wrapping us into two [Julie Hoyer]: possible paths here, and you can choose whichever one, but I'd love to talk about them both.

46:45[Julie Hoyer]: One being the whole semantic layer conversation. But two, and what you were just talking about, [Julie Hoyer]: what advice would you give an analytics practitioner, a crafter of your company has this BI tool, [Julie Hoyer]: and it has this AI agent? Like, are you saying you can control that AI agent [Julie Hoyer]: with like these handbooks and things, which it sounds like maybe it leans into semantic layer, [Julie Hoyer]: or are you saying, don't use the off the shelf thing, go make your own custom one? Or is this [Julie Hoyer]: scenario not applicable at all? Option C. I think, I think they're both kind of the same [Rob Collie]: conversation, honestly. Like, the key thing to recognize about any AI solution, and this is [Rob Collie]: the thing that kind of blew my mind, is that the LLM is sort of like, almost like, it's just [Rob Collie]: something you pick off the shelf, and it's almost like, like the, an element on the periodic table, [Rob Collie]: it is exactly the same for you as it is for me, as it is for the, as it is for Satya Nadella at [Rob Collie]: Microsoft, it's the exact same LLM. And it's really just like a component. And most of making,

48:07[Rob Collie]: not most of, making these things do anything useful for us is all about just regular software, [Rob Collie]: not AI, just regular software. And the regular software's job is to handle like the user experience, [Rob Collie]: just like it always does, as well as the flow of information to and from the LLM, [Rob Collie]: like giving the LLM access to what it needs to know. So let's keep in mind for a moment that most [Rob Collie]: AI agents, we fast forward like two years, and we see what the future looks like. Moest AI agents [Rob Collie]: that are deployed at companies aren't going to be about analytics, you know, like, you know, like a, [Rob Collie]: a chat bot that handles the first level of interaction with customer service, for instance, [Rob Collie]: it doesn't need to be performing strategic queries against your, you know, your data warehouse, [Rob Collie]: right? Like it really just needs to have a, like a flow chart in it, and, and also some [Rob Collie]: instructions on how to talk to people and what not to say and all that kind of stuff. And that, [Rob Collie]: that sort of falls into the handbook category. So there's a lot of tactical AI agents that aren't

49:16[Rob Collie]: going to have anything to do with what we think of as analytics. I mean, I just want to level set [Rob Collie]: that. What's kind of neat, though, is that even those tactical things that have nothing to do with [Rob Collie]: analytics, they still, because they, they operate on information that needs to be very carefully [Rob Collie]: controlled and fed at the right time, that the types of discipline that we've developed as analytics [Rob Collie]: people, as data people, are that are the right mindset for making those things work properly, [Rob Collie]: you know? And now that you've given us, the world has given us crafter types, [Rob Collie]: the ability to write real software very quickly with the use of an agentic, you know, like something [Rob Collie]: like cloud code, we've got everything at our fingertips, that we're going to be really good [Rob Collie]: at building those sorts of solutions, even though they're not related to traditional analytics at [Rob Collie]: all. And so I want to make sure that I'm not pigeonholing us data crowd into just the traditional [Rob Collie]: data crowd type AI stuff. So that's, and the handbook stuff is not, is not part of what you're

50:24[Rob Collie]: hearing when people talk about semantic models or semantic layers. Handbooks are not, not really [Rob Collie]: part of that. Semantic layers are a decoder ring for your structured data. And that's, now, [Rob Collie]: now we're back in the analytics and data crowd, you know, we're back in our, in our, in our home [Rob Collie]: neighborhood now. Yes. And so the, the, I think this isn't even really controversial, honestly, [Rob Collie]: because so we were talking about, you know, the Tableau crowd coming for me. Okay, so [Rob Collie]: it doesn't matter what platform you're associated with, like if you're a Tableau professional, [Rob Collie]: you're a Snowflake professional, you're a Power BI professional. I think it's really just like [Rob Collie]: a foregone conclusion that you are going to be working with semantic models and semantic layers.

51:21[Rob Collie]: And, you know, I can give you the technical reason why, but I can also just give you the [Rob Collie]: market reason why. So let's take Tableau as an example, and I love this, this just tickles me [Tim Wilson]: to death. So for, for men, the best thing about Power BI, from a technical standpoint, the best [Rob Collie]: thing about Power BI was not the reason why it got adopted. The reason Power BI won the BI wars, [Rob Collie]: and has the largest market share is because it was cheapest and it was integrated with the rest [Rob Collie]: of Microsoft. That's it. No one was evaluating like, ooh, it's so cool that you have a semantic [Rob Collie]: model in this star schema stuff and all, like, like, it makes a difference in practice, but that's [Tim Wilson]: not why it won. And Tableau long neglected the idea of semantic models. It's, they were all about [Rob Collie]: each time you need a new dashboard, you go write a bunch of new SQL queries to make a big flat, [Rob Collie]: to make a big flat wide rectangle of data that you hide behind the dashboard. That's the philosophy

52:22[Rob Collie]: over and over and over again. And the thing is, it worked, right? I mean, like the IPO, [Rob Collie]: they sold the Salesforce, I mean, those people made gobs and gobs and gobs of money. So it's like, [Rob Collie]: you know, you know, I'd start arguing with them about like, well, y'all don't have a semantic [Rob Collie]: layer. And they go, well, I have more money than you, you know, but you're right. So they eventually [Rob Collie]: invented a semantic layer that sort of was an attempt to sort of add what Power BI had been [Rob Collie]: doing from the beginning, but then they neglected it. A friend of mine at Tableau was in charge of [Rob Collie]: that for a long time, and eventually he just retired in frustration. He just gave up.

53:00[Rob Collie]: Not even, not even making this up. Okay. Well, after so many of these companies have neglected [Rob Collie]: semantic layers forever, they all banded together to form this OSI thing, which I think you have [Rob Collie]: has been talked about on the show before. Open semantic interchange. And, and on the Tableau [Rob Collie]: website, a year ago, less than a year ago, there was an article says, the agentic future demands [Tim Wilson]: an open semantic layer. Okay. So everyone now, if they, every data company that wasn't in on [Tim Wilson]: semantic layers is now in on semantic layers. Every single one of them has something that either [Rob Collie]: are sticking with the one they already had, like in Microsoft's case, or they're banding together [Rob Collie]: in this open standard and rapidly integrating into that product and trying to get their customers [Rob Collie]: to adopt it. Now, why would they be doing that? As a company builds a semantic layer, I think, [Tim Wilson]: and this was when we had Cindy house and on to talk about it, there was something she said that [Tim Wilson]: I really liked. So she was kind of in the OSI heavy camp our last episode, we had Jacob Madsen,

54:05[Tim Wilson]: we talked about maybe we don't need one at all. So we've kind of covered, I think, [Tim Wilson]: my one question is like, how does the semantic layer not get held up as the monolithic [Tim Wilson]: boogeyman and the monster dev project of saying, yes, yes, yes, yes, we will do glorious things [Tim Wilson]: with AI as soon as we build the semantic layer, because we did that with data dictionaries, [Tim Wilson]: we've done that, I mean, at least in the days of power play cubes and SSS, like they were [Tim Wilson]: contained, but like where that's where I get nervous about semantic layers as it becomes [Tim Wilson]: an excuse and people like we have to build the entirety of the story of our data.

54:50[Rob Collie]: You could take your concern there that you just shared and you could plug and play replace that [Rob Collie]: with data warehouse, like seven or eight years ago. And that would be something that was a talk [Rob Collie]: and a spiel that I would give everywhere, like the data warehouse. So we got to go build the [Rob Collie]: data warehouse first. And the other thing is data warehouse is never done. And by the way, [Rob Collie]: the other thing about data warehouses is that you would find that you've been trying to anticipate [Rob Collie]: all of these future needs, and you anticipated them poorly. So you built plumbing to nowhere, [Rob Collie]: places that faucets were never going to be needed. But then the first time you need a faucet, [Rob Collie]: you go and you look and there's no pipe, the place you needed the faucet. So you over engineered [Rob Collie]: and under engineered at the same time. And yeah, I agree, like those sorts of monolithic projects [Tim Wilson]: that kind of become, they become their own thing. They become like a goal in and of themselves.

55:42[Rob Collie]: Those are danger. You do not want that. So one of the things we've been, one of our philosophies [Tim Wilson]: that are our company, we call it faucets first, we start with the faucet and we build backwards [Rob Collie]: from there. So yes, we do need responsible data plumbing. But if you're building plumbing forward, [Rob Collie]: you're going to lose. Now, if you're a consulting company and you're building plumbing forward, [Rob Collie]: you're going to win because you're going to build a lot of hours. And then when things don't work, [Rob Collie]: you're going to build a lot more hours to fix it. But if you go faucets first, [Rob Collie]: if you go faucets backwards, you can get everything that you need. So I think semantic models, [Rob Collie]: and the way we approach Power BI is to sit down and go faucets first. We don't go build a big [Tim Wilson]: data model before we build our first dashboard. The whole point is to get to a [Rob Collie]: first dashboard as quickly as possible so that the customer can go, okay, that's not right [Rob Collie]: in these three or four different ways. And then we go modify the data model to fit it.

56:50[Rob Collie]: The same thing here. If your goal is to build, and okay, so let's bridge a really important gap [Rob Collie]: here. The reason why a semantic model is suddenly so important today, it was important to BI, [Rob Collie]: but no one cared other than Power BI really. But everyone now cares is because you don't have [Rob Collie]: what I call the semantic shock absorber of the people that you lock in a room and make them [Rob Collie]: write the queries. So each new dashboard that's created is really representing new questions [Rob Collie]: that have emerged. The business has new questions, so you can build a new dashboard.

57:26[Rob Collie]: This has been inefficient really, even in the Power BI world, this is really not optimal. [Rob Collie]: Okay, but it's even less optimal than the tab low world where it requires like a week's worth [Rob Collie]: of SQL work before you get the first dot on the chart, right? If an AI agent forms a new question [Rob Collie]: or a user comes to an AI chatbot with a new question, there's no time for someone to go [Rob Collie]: write a week's worth of SQL. And you also don't want to trust the AI to go write that SQL [Rob Collie]: because it will write it slightly differently each time and it won't know which column means [Rob Collie]: what in the source data. This is why the decoder ring analogy I think is so apt.

58:02[Rob Collie]: Symantec glare is just a very nerdy way to say decoder ring. We wrote down what things mean. [Rob Collie]: We wrote down how to translate our company's structured data into meaning that matters to [Tim Wilson]: the business. What is gross profit defined as? There's a lot of nuance to that that's [Rob Collie]: different for every single business. And so if you look at it through an AI lens and you go, [Rob Collie]: okay, what does this AI agent need to do? And then you build the Symantec model that is required [Rob Collie]: to power that. You're in a much more focused means of engagement than if you're trying to like, [Rob Collie]: let's go model the whole business, just the whole thing. And let's just party on this thing forever [Rob Collie]: and never test it and never write. No, that's not the way, right? And I was going to say too, [Tim Wilson]: like I love what you're saying and I just feel like people are finally waking up to this idea [Julie Hoyer]: and that like surfacing the data alone was not the valuable part. And so like why people could [Julie Hoyer]: get the budget approved for data warehouse or like projects or the huge Symantec layer projects.

59:19[Julie Hoyer]: I feel like it's because and Tim, you've talked about this a million times, right? Like the promise [Julie Hoyer]: of the value of like having the thing, having the data and in current conversations with clients, [Julie Hoyer]: and I've seen it more like across the industry and articles, you know, there's this whole like, [Julie Hoyer]: well, what's the actual return on investment for AI? And I just feel like AI has become [Julie Hoyer]: this thing that really highlights to people that just because your data is there or you have all [Julie Hoyer]: this data, right? Like even though AI can access it and use all of it that you've spent so much time [Julie Hoyer]: and effort to create and surface, the output of AI is not inherently valuable to you. You know, [Julie Hoyer]: like it is finally like the mirror to that flawed thinking. And so it's interesting when you were [Julie Hoyer]: saying like faucet first design, I do feel like everyone is now afraid to just be like, oh yeah, [Julie Hoyer]: do another AI POC. They're like, no, I need to know that my faucet in my business is going to work,

1:00:22[Julie Hoyer]: not that you can go make plumbing in another room. Like I don't care about that plumbing in another [Julie Hoyer]: room. So I just, I really like that finally clicked for me in the conversation we've been [Julie Hoyer]: having, but I'm curious your thoughts on like AI being scrutinized for value and this idea of [Julie Hoyer]: like the data itself is not the value to the business. AI should be scrutinized for value, [Rob Collie]: you know, there's a lot of pressure to just go do AI, right? Because everyone, everyone can sense [Rob Collie]: the uncertainty of inaction, right? If we're not doing anything about AI, we are falling behind, [Rob Collie]: right? As a business. And so, but like what happens is the people who don't know what to do about AI [Rob Collie]: at all, they don't know how to apply it, but they're important. They're like board members or whatever, [Rob Collie]: they lean on executives and say, hey, you need to be doing AI, right? And the executive doesn't know.

1:01:26[Rob Collie]: So they turn to someone else and say, you know, you need to be doing AI. And then you end up in [Rob Collie]: these weird, crazy things. We've all heard these stories and they're real about people measuring [Rob Collie]: teams usage of AI tools. And by the way, they're always the off the shelf tools. So they're the [Rob Collie]: wrong things for most business uses, right? And making sure that people are using them, [Rob Collie]: because if they're not using them, then they can't go back up the chain and say, hey, we are doing [Rob Collie]: something about AI, right? If we're in a really, really strange place where no one knows what to [Rob Collie]: do with it, but the amount of pressure to do it is I don't think we've ever been in a situation [Rob Collie]: like this. Like BI wasn't like this, right? Like no one has ever under pressure to do BI, like, [Rob Collie]: unfortunately, right? They, you know, like the boards never came to their, you know, [Rob Collie]: C suites and said, you should be doing BI. Well, I feel like personalization was a little bit [Julie Hoyer]: like that, Val. Like we heard that from a lot of clients, right? Like our goal is to do personalization.

1:02:23[Tim Wilson]: You're like, but to what end? Like digital transformation. So like, I just think that [Rob Collie]: the semantic model case with an AI chat interface over the top of it is one of the [Rob Collie]: easiest wins for AI adoption that's actually going to add ROI. Like so for people who have [Rob Collie]: Power BI investments, if they've got good semantic models, like we're, we're putting in AI chat [Rob Collie]: interfaces. And honestly, they're so much better than dashboards. There's so much friendlier, [Rob Collie]: there's so much more user friendly, people actually get value out of them. And even the [Rob Collie]: people who built the semantic models who know it, the ins and outs, even I use the chat interface [Rob Collie]: to go and do all kinds of things for me to go research things for me that I wouldn't do otherwise.

1:03:11[Rob Collie]: It's not even like saving me time. It, it, it, I form better questions and kick my buddy off to go [Rob Collie]: research it for me. And again, it leans on the semantic model. It doesn't, it doesn't hallucinate. [Rob Collie]: It gets real answers from real systems and comes back and, and, and, you know, and I can sort of, [Rob Collie]: I can send my assistant back and, you know, to do more if I, if I want, you know, [Rob Collie]: under all this pressure, and again, as data and analytics professionals, it is one of the easiest [Rob Collie]: and most trustworthy and best places to start dipping our toes into AI is to start enabling [Rob Collie]: these sorts of AI chat interfaces that actually work, not the ones that come off the shelf from [Rob Collie]: the vendor. But if you build it, you build something yourself, like we've got this framework that we've [Rob Collie]: built, again, we have these two developers that have been doing amazing things for us, [Rob Collie]: like our clients freaking love this stuff and we love it. So I think it's one of those, it's not [Rob Collie]: the only thing. I do not want to say this is like the future of AI, but it is a great place to dip

1:04:12[Tim Wilson]: the toe into it that does add value. And it's right in our backyard. So I wanted to make sure we, [Rob Collie]: I got that, that little, you know, career advice. Yeah, it's a great place to land the plane to kick [Tim Wilson]: off for show two. Yeah. All right. Well, that, yeah, there's, there, I have so many more questions, [Tim Wilson]: so many more thoughts, but we also up against the clock. So this has been a fantastic discussion.

1:04:42[Tim Wilson]: And yeah, I think it's even in this discussion for me in the book, lots of light bulbs have gone on [Tim Wilson]: for me. So this has been enormously useful to me, at least, and it's all about me. But [Tim Wilson]: when we last thing we like to do on the show is go around the, go around the group and everyone [Tim Wilson]: share a blast call, something that something or maybe a couple of some things that might be of [Tim Wilson]: interest or amusing or entertaining or useful to our audience. And Rob, you're our guest. So [Rob Collie]: do you have a last call or two? I do. I have one that's relevant and then one that's kind of just [Rob Collie]: funny off, off topic. So the one I'll obviously provide you the link, but an old friend of mine [Rob Collie]: at Microsoft named Uli Homan wrote something on LinkedIn recently, where he talks about this new [Tim Wilson]: role he's calling the product engineer. We've had software engineers, software developers, [Rob Collie]: all that kind of stuff. We've also had product managers who like were like the designers of it [Rob Collie]: and all that kind of stuff. And he makes a very, I think, eloquent case for, I think,

1:05:44[Rob Collie]: where we're all going to land with this product engineer role. And so I think it's a worthwhile [Tim Wilson]: read and it's not very, it's not very long. And I think it lands the concept pretty cleanly. [Rob Collie]: Then what's the relevant one? No, you said it could be, it could be any link to anything that [Rob Collie]: we've ever liked, right? That's true. Right. So this is an article from 2005 on the McSweeney's [Rob Collie]: website. It's titled, a realistic assessment of how many 12 year olds I could beat up before they [Rob Collie]: overtook me. And it is such an analytical, dispassionate analysis that uses game theory [Rob Collie]: and assumes optimal behavior on the part of the 12 year olds. It's like, no, the 12 year olds [Rob Collie]: aren't going to get up in a nice line and let me take them one at a time. They're going to form [Rob Collie]: a circle, right? And they're going to know that groin kicks are their primary weapon against me.

1:06:42[Rob Collie]: And here's how big an average 12 year old is and just really just breaks it down and [Rob Collie]: eventually comes up with a very reasoned argument for why this certain number is probably the number [Tim Wilson]: that would take him. I love it. That's so funny. Oh my God, that's amazing. [Tim Wilson]: That's really funny. Julie, what's your last call? [Julie Hoyer]: Mine. Actually, I feel like this ties back to Rob, what you were talking about in your use with [Julie Hoyer]: AI agents. A colleague recently shared a article called AI May Be Making Us Think and Write Moere [Julie Hoyer]: Like. And it sparked my interest right away because I feel like multiple times on the show, [Julie Hoyer]: I've said like, but what about like group think and like the cyclical, you know, like [Julie Hoyer]: self-fulfilling prophecy. I'm like, if AI is then using our AI outputs to then continue modeling, [Julie Hoyer]: I'm like, it's all just going to converge somewhere. So anyways, this was very like fulfilling for me [Julie Hoyer]: to see that there's now been research that, yes, that's actually happening. They've pretty much

1:07:47[Julie Hoyer]: said that they're seeing all of this really come to a point where everyone's losing their unique [Julie Hoyer]: voice in writing. And because of that too, what everyone is putting out is becoming more [Julie Hoyer]: similar. So it is like this idea of the group think happening. And it's funny because they [Julie Hoyer]: say that AI solution, Rob, similar to what you did, is they were saying, you need to make sure [Julie Hoyer]: you have a very clear framework to ask your AI model to enforce like strict stylistic constraints [Julie Hoyer]: and make sure that you can extract your distinctive voice. So I thought that was funny. It's like, [Julie Hoyer]: AI is the answer, even though AI is the problem. And it was this whole, you know, echo chamber.

1:08:34[Rob Collie]: So it's an interesting read. Wow. That's not everyone's starting to sound the same, but a lot [Julie Hoyer]: of people are. You can read something now and you're like, oh, yeah, that sounds like AI. It's [Rob Collie]: pretty quick. So I'm going to resist. Val, what's your last call? [Val Kroll]: So there's a AI tie-in for this one. But it is very self-serving because it comes from some [Val Kroll]: of my co-workers at SIN. But my thing is, 30% of my last calls come from like UX collective, [Val Kroll]: like this publishing group on Medium. And this could have very well been on there. So it's a [Val Kroll]: Substack article, all that caveat build up. And it's called Designing the Front Door. And what [Val Kroll]: I thought was really interesting is it's like from this engineer and this designer at SIN thinking [Val Kroll]: about how you create the front door to some of the agents that are not the helper agents, [Val Kroll]: the chatbot, the little thing you see in the bottom right hand corner of your screen, right?

1:09:40[Val Kroll]: But they're calling it like the spotlight messenger. And what I think is really interesting [Val Kroll]: about the article beyond the solution that they landed on is how they use data and experimentation [Val Kroll]: and lots of research to figure out what is the optimal way to bring some of that to bear in a [Val Kroll]: way that doesn't harm other interactions, whatever. And it just feels like some of the research that [Val Kroll]: I did when I was at further with the consent management banners four years ago and thinking [Val Kroll]: about like, how do you surface this when there is no established pattern? And so it's an interesting [Val Kroll]: look at it with some fun examples. And it's a quick read. So it's a good one. So how about you, Tim?

1:10:17[Tim Wilson]: So I'm going to kick it old school with an ebook. It's on a github.io site called Moedels [Tim Wilson]: Domestified, a practical guide from linear regression to deep learning. So if you want to [Tim Wilson]: step away from AI and just kind of understand generalized linear models a little bit deeper [Tim Wilson]: or causal modeling, I have not read the whole thing. I've kind of sampled it. It's got kind of a lot [Tim Wilson]: of our a lot of the models are kind of ours available and Python. So you can kind of play with [Tim Wilson]: either one. But I feel like that maybe when I get overwhelmed by Claude and going back and forth, [Tim Wilson]: I it's a good little thing to dip in to say, maybe I should just give some deeper [Tim Wilson]: statistical knowledge and a good old fashioned ebook. It's a good way to do it. So that's how Tim [Tim Wilson]: touches grass. That's what you're saying. Yeah, that's what I'm not measuring the speed of sound [Tim Wilson]: waves through different materials. So you know, but you can, but you can't. That's the important [Tim Wilson]: thing. And can I just kind of throw one last little plug in here and you could edit it out if

1:11:34[Rob Collie]: you if it doesn't it doesn't fit. I just want to encourage people to go to fairgamebook.ai [Rob Collie]: and preorder the book. I mean, I genuinely wrote this to help people writing books doesn't make [Rob Collie]: enough money to ever be worth the personal sacrifice. Like and so I was really gratified [Rob Collie]: by y'all reading it in advance. And yeah, it's it's it's for our crowd is for leaders. [Rob Collie]: And I think it will actually help you not be so afraid. Like if you're having some angst about [Rob Collie]: AI, like my I felt like sitting down right in that book, my job was to help you feel better [Rob Collie]: about it. Like I actually turn it into excitement. So I genuinely hope that helps people.

1:12:15[Tim Wilson]: Well, and I will say so you you took it from me because because I was definitely going to [Tim Wilson]: plug the book again. So I will I will further endorse that because I did find it to be [Tim Wilson]: extremely useful. We had a proof of it. So it was like a super early version. So I mean, [Tim Wilson]: it was a proof so we we could not we could not copy anything from it. We could not highlight [Tim Wilson]: anything in it. So like the the the notes had to be like old school retyping brilliant things.

1:12:43[Rob Collie]: And you know, you sit down you sit down with cloud code. And it's wide open. You just [Tim Wilson]: you can do it. There's no protection anymore. Okay, I'm not going to say maybe there wasn't [Tim Wilson]: a way to throw the whole thing at AI and have it read it. Even when I'm making those clouds, [Rob Collie]: tell me, hey, you know, Rob, this isn't going to actually protect you, you know. [Tim Wilson]: All right. Well, Rob, thank you so much for coming on. On the one hand, you're like promoting a [Tim Wilson]: book, but the book is itself really useful. And this discussion, I think was extremely useful.

1:13:17[Tim Wilson]: So thank you for taking the time to come on like this. And really, I think y'all I mean, [Rob Collie]: again, I hadn't met y'all before now. I really think this is a I love your format. I love your [Rob Collie]: approach. I think you're doing great work here. So I really appreciate your the invitation. [Tim Wilson]: Well, awesome. So what you could do and any of our listeners could do or you could have an [Tim Wilson]: agent do is go leave us a review and a rating. Look at that. Look at that segue.

1:13:41[Tim Wilson]: Oh, I'm on it. Make Michael proud. [Tim Wilson]: How many reviews do you want? [Tim Wilson]: So if you listen on a platform that allows leaving reviews, listeners, we would love [Tim Wilson]: to get reviews and ratings that helps us out. If anybody, if you'd like a sticker [Tim Wilson]: for the show, you can go to analyticshour.io and fill in the little form there. We'll send you [Tim Wilson]: stickers. We also love to hear from our listeners so they can you can you can reach out to us on [Tim Wilson]: LinkedIn on the measure slack. You can just email us at contact at analyticshour.io. But regardless [Tim Wilson]: of how many faucets you've turned on, if the faucets first, if the plumbing's first, if you're [Tim Wilson]: deep, deep into cloud code already, or if you're now just inspired to get into [Tim Wilson]: cloud code, no matter what you do for myself, for Julie, for Val, keep analyzing.

1:14:45[Announcer]: Thanks for listening. Let's keep the conversation going with your comments, [Announcer]: suggestions and questions on Twitter at at analytics hour on the web at analyticshour.io, [Announcer]: our LinkedIn group and the measure chat slack group. Music for the podcast by Josh Grohurst. [Announcer]: Those smart guys wanted to fit in so they made up a term called analytics. Analytics don't work. [Tim Wilson]: Do the analytics say go for it no matter who's going for it? So if you and I were on the field, [Tim Wilson]: the analytics say go for it. It's the stupidest, laziest, lamest thing I've ever heard for reasoning [Julie Hoyer]: in competition. It's all semantics. Yeah, it is indeed. It's a challenge.

1:15:28[Rob Collie]: And I saw your, I saw what you wrote in the doc in response and I'm like, oh, yeah, so that's [Tim Wilson]: okay, I'm not gonna ruin it. I mean, I will try. I mean, it's not like we can't like [Tim Wilson]: touch on it at all. I just think there's like so much other like gold for us to hit on. [Tim Wilson]: As the moderator, I will say, you know what, I'll allow it, but you're going to be on a [Tim Wilson]: shot clock. Okay, I got it. I mean, we're going to, we're going to barely scratch the surface [Tim Wilson]: of everything I think we would want to talk about. And that's just a sign of a good show.

1:16:04[Tim Wilson]: So unless Michael Helbling is running the show, in which case it's because he has lost control [Tim Wilson]: into the terrible job. And Tony, feel free to put that in the outtake. We always bully him [Julie Hoyer]: into one more question, you know? [Tim Wilson]: Rock flag and data genes to crafters.