#280: Dashboards Must Die! Long Live Dashboards! with Andy Cotgreave

0:05[Announcer]: Welcome to the Analytics Power Hour. [Michael Helbling]: Analytics topics covered conversationally and sometimes with explicit language. [Michael Helbling]: Friends, analysts, and dashboard builders, lend me your eyes. [Michael Helbling]: I come to speak of dashboards, not to praise them. [Michael Helbling]: The charts we build may live beyond our hands, yet meanings often tuned within the data. [Michael Helbling]: Dashboards are loved, executives who click, marketers who scroll, and analysts who sigh.

0:40[Michael Helbling]: But comprehension rarely walks these halls and insights haunts the tooltips like a ghost. [Michael Helbling]: O judgment, thou hast fled to decks of slides and reasons lost. [Michael Helbling]: My heart is with the charts, and I must pause until it come again. [Michael Helbling]: Okay, well, as the Bard might say, aw, shit, here we go again. [Michael Helbling]: We're talking about dashboards. [Michael Helbling]: The staple consumable of our little industry, but always a little bit hard to define.

1:07[Michael Helbling]: And now with AI, are they maybe finished for good? [Michael Helbling]: I don't know, we've got a lot to talk about. [Michael Helbling]: So let me introduce my co-hosts, Julie Hoyer. [Michael Helbling]: You've built a dashboard or 500, I would guess. [Michael Helbling]: Oh yeah, well-versed. [Michael Helbling]: And Tim Wilson, I know this is your first time talking about dashboards, but I think you might be excited. [Tim Wilson]: should have checked what we were talking about before.

1:30[Michael Helbling]: Yeah, yeah. [Michael Helbling]: Well, we'll try to keep you up to speed because I know you probably don't have too many opinions about it. [Michael Helbling]: And I'm Michael Helbling. [Michael Helbling]: I'm super excited for our guest today. [Michael Helbling]: Andy Kotriev is the co-author of The Big Book of Dashboards and former technical evangelist at Tableau. [Michael Helbling]: He is the co-host of Chart Chat and writes the How to Speak Data newsletter.

1:51[Michael Helbling]: He's got more than 15 years of experience in data visualization and business intelligence. [Michael Helbling]: first honing his skills as an analyst at the University of Oxford. [Michael Helbling]: He has inspired and trained thousands of people with technical advice and ideas on how to identify trends in visual analytics and develop their own data discovery skills. [Michael Helbling]: And finally, Andy has a new book out this month, Dashboards That Deliver.

2:14[Michael Helbling]: Welcome to the show, Andy. [Andy Cotgreave]: Hello, everybody. [Andy Cotgreave]: It is a delight to be here. [Andy Cotgreave]: I'm looking forward to a lot of banter and possibly arguments. [Michael Helbling]: I like your style. [Michael Helbling]: That's not what our show is about. [Michael Helbling]: No, it's awesome to have you, Andy. [Michael Helbling]: And I think, you know, obviously you've got a new book about dashboards that's just coming out in like a week or so.

2:42[Michael Helbling]: I think maybe you're taking the line that maybe dashboards aren't dead, but let's jump into it. [Andy Cotgreave]: Yeah, let's do it. [Andy Cotgreave]: I'm very excited because if I was listening to one of your recent episodes about AI, and I don't know your voices by too well, but I don't know, Michael or Tim, one of you said, dashboards are shit. [Andy Cotgreave]: I think that was the line you said in the AI episode. [Andy Cotgreave]: That was my call.

3:07[Tim Wilson]: That was something I might say. [Andy Cotgreave]: So I'm looking forward to taking that one. [Tim Wilson]: So can we start by actually defining a dashboard? [Tim Wilson]: Because I feel like that is actually one of those things where people get into raging debates, and it's actually the underlying debate is what is a dashboard. [Andy Cotgreave]: In the big book of dashboards, we defined, and in dashboards to deliver, we define a dashboard as a visual display of data that is used to monitor and or facilitate understanding.

3:42[Andy Cotgreave]: 15 words. [Andy Cotgreave]: When we wrote the Book of Book of Dashboards, the definition got longer and longer and longer as we were trying to capture all caveats. [Andy Cotgreave]: In the end, we're like, it's such a vague term. [Andy Cotgreave]: We just reduced it to virtually nothing. [Andy Cotgreave]: Stephen Fuchs chose to write about 1500-word blog post, destroying our definition, even questioning in his blog post whether myself, Stephen Jeff, the co-authors of that book, had the authority to teach about dashboards.

4:10[Andy Cotgreave]: Thank you, Steve, because he had very specific things, but things like must fit on a single screen, must be interactive. [Andy Cotgreave]: But it's so easy to demonstrate that that is patently wrong, right? [Andy Cotgreave]: I wrote a whole chapter in the new book about this challenge of what a definition of a dashboard is. [Andy Cotgreave]: And in the end, what is a dashboard? [Andy Cotgreave]: It's a piece of wood or leather that used to sit on a stagecoach between the horse and the driver to stop water dashing [Andy Cotgreave]: onto the driver's legs.

4:40[Andy Cotgreave]: People made cars and they stuck that kind of thing to the front of the car and they put gauges on it and called it a dashboard. [Andy Cotgreave]: So it's a term that's borrowed from something else in etymology. [Andy Cotgreave]: and is a catch all for so many things. [Andy Cotgreave]: So we have a really loose definition of what a dashboard is. [Andy Cotgreave]: But if you're building a chart or a collection of charts or even a collection or even a single text table, very controversially, we can make an argument that it is a dashboard, even though many people might say it isn't.

5:10[Andy Cotgreave]: And we're fine with that. [Tim Wilson]: But you've said it's displaying information and promoting understanding. [Tim Wilson]: That does stop short, I think, of the interface to an underlying data system for deep exploration. [Tim Wilson]: pick on adobe a little bit but they've got analysis workspace and they will send which is is you can build your building visual displays of information and then you're sharing those so i think in some cases i think of anything it's a challenge for their product.

5:45[Tim Wilson]: It's built more to be kind of a. [Tim Wilson]: an interactive digging and drilling and ad hoc, but they also say you can use it to create a workspace for the display and understanding of data to end users, which I think it struggles at because that's not what its core basis is. [Tim Wilson]: Do you see A line there between [Tim Wilson]: displaying and promoting understanding and exploring shallowerly or deeply? [Andy Cotgreave]: Absolutely.

6:22[Andy Cotgreave]: So, I mean, you get into that question to me also touches upon the [Andy Cotgreave]: The audience is in that there are data analysts and engineers, and there are business people or non-data analysts, not necessarily business people, just people who are not data analysts. [Andy Cotgreave]: As a data analyst, then we have access and use the systems to visually explore data and just [Andy Cotgreave]: ask and answer questions and chase insights to find them.

6:49[Andy Cotgreave]: But much as I would love everyone in the world to be as skilled as all of us in exploring data, I've discovered that's not a real option. [Andy Cotgreave]: Some people are freaked out by data so they don't have time to learn data. [Andy Cotgreave]: So a dashboard is a window on a predefined, finite set of questions [Andy Cotgreave]: to allow those people to explore in a relatively narrow way the answers to those predefined questions.

7:18[Tim Wilson]: I feel like that's some of the copy that was in the larger, the longer definition that then got shortened. [Andy Cotgreave]: Oh, yeah. [Andy Cotgreave]: So the definition is short and vague, and then we've written two books about what that actually means. [Julie Hoyer]: So then within that broader definition of dashboard, I'm glad though, Tim, that you had us touch on the difference between what's a dashboard and [Julie Hoyer]: where is maybe like an analyst or an engineer like digging in because I think that's really helpful.

7:43[Julie Hoyer]: My other question then is within Bucket of Dashboards, do you talk about them Andy as like different categories of dashboards depending on the audience or like you said is it a [Julie Hoyer]: a table of text or, you know, does it have a lot of visualizations? [Julie Hoyer]: Is it a page long? [Julie Hoyer]: Is it being put on, you know, a single screen or it's a whole Adobe workspace with lots of panels? [Julie Hoyer]: Are there categories for format?

8:10[Julie Hoyer]: Are there categories for audience? [Julie Hoyer]: Are there categories for any other characteristic? [Andy Cotgreave]: So, we don't formally create that categorization in our book. [Andy Cotgreave]: In our book, we refer to Nick de Barra, who's written practical dashboards. [Andy Cotgreave]: He sort of inherited Stephen Fugh's teaching course and then has built it into his own. [Andy Cotgreave]: He's brilliant. [Andy Cotgreave]: He created a taxonomy of dashboards.

8:36[Andy Cotgreave]: I think it's 13 different types of dashboards. [Andy Cotgreave]: Is it an infographic? [Andy Cotgreave]: Is it used for monitoring ongoing problems? [Andy Cotgreave]: Is it used to monitor a short-term project? [Andy Cotgreave]: He's got a really good taxonomy. [Andy Cotgreave]: We chose not to go down that route, but that's a really useful reference point if people want it for types of dashboards. [Andy Cotgreave]: holy endorse next approach.

9:00[Andy Cotgreave]: Another aspect of the question was around form factor. [Andy Cotgreave]: Should it be a single page? [Andy Cotgreave]: Should it have interactivity? [Andy Cotgreave]: Should it have multiple views? [Andy Cotgreave]: It depends. [Andy Cotgreave]: It depends on the vast infinity of user requirements, what they need, how they're going to consume it, how much time they're going to consume it, and again. [Andy Cotgreave]: So we explore that, adapt in the book, but we don't categorize it.

9:26[Julie Hoyer]: Just because I'm thinking, Tim, when we talk about dashboards a lot, I know Tim has the point of view, and I generally agree, but one of the best uses of a dashboard we talk about for performance measurement. [Julie Hoyer]: And so, Tim, I know I [Julie Hoyer]: When you talk about that, do you believe that's the only way a dashboard should be used? [Julie Hoyer]: Because I think that's where we end up getting in debates a little bit.

9:50[Julie Hoyer]: Should it only be used for performance measurement where it's very clear, concise, it's against a target, it's a quick snapshot? [Julie Hoyer]: Because does it make you bristle to hear like, there's lots of categories and uses of dashboards, or are you okay with that? [Tim Wilson]: I think I take a simplistic view and I say this is, I just live in that one category and I followed Nick. [Tim Wilson]: He is brilliant. [Tim Wilson]: He's engaging a lot on LinkedIn, so I will further endorse like Andy, like Steven, multiple.

10:18[Tim Wilson]: Everybody who's going to get mentioned here are [Tim Wilson]: really, really sharp. [Tim Wilson]: So I see that as kind of being a narrow view that if we use them for just measuring performance, monitoring how it's going, having targets, it really works. [Tim Wilson]: I think some of the stuff that Nick and Andy have done is saying, yeah, there are broader uses. [Tim Wilson]: I think I've had an overly simplistic view and they make a, they spend a lot more time thinking about it and have a much more nuanced, [Tim Wilson]: view, although I do come to questions like [Tim Wilson]: data storytelling.

10:53[Tim Wilson]: I'll see people, none of the people that I've just mentioned, but we'll talk about how a dashboard should tell a story and it's data storytelling. [Tim Wilson]: And that's one of the ones that just cringes and makes me cringe and wanna run back to my, no, no, no, here's this one specific area that I think dashboards can be used. [Tim Wilson]: I'll put it this way. [Tim Wilson]: I think it's very, very, I can so readily defend a dashboard that is generally one screen [Tim Wilson]: with some caveats for monitoring the performance of a project or an initiative, a campaign.

11:27[Tim Wilson]: And doing that well is, to me, easily, easily defensible. [Tim Wilson]: When people head off into other areas and start making broad proclamations, they're kind of setting up things that they can then say, these are, it's bad for this. [Tim Wilson]: And I'm like, well, yeah, but you're talking about something that maybe isn't really what should be a dashboard. [Michael Helbling]: It's that time of year. [Michael Helbling]: Half your team is on vacation, your inbox is quiet, and maybe, just maybe, you're thinking about taking a little time off yourself.

11:58[Michael Helbling]: Well, with 5Cran, you actually can. [Michael Helbling]: Their fully managed data integration platform keeps your analytics humming even when you're out of office. [Michael Helbling]: I mean, you could be off hiking, surfing, just sleeping in, [Michael Helbling]: 5Train is syncing all your sales, marketing, and product data into your warehouse accurately, securely, and always on. [Michael Helbling]: It's like a reliable teammate that never takes PTO, so relax, unplug, stop worrying about late-night Slack alerts.

12:27[Michael Helbling]: Your data is not just covered, it's thriving. [Michael Helbling]: That's the 5Train difference. [Michael Helbling]: Check them out at 5Train.com slash APH. [Michael Helbling]: See interactive demos or start a 14-day free trial. [Michael Helbling]: That's F-I-V-E-T-R-A-N dot com slash A-P-H. [Michael Helbling]: Let's get back to the show. [Andy Cotgreave]: I'm desperate to come back to storytelling, but I want to talk about that, your definition of what some of the things you've explained that to, right?

12:56[Andy Cotgreave]: One of the things that really frustrates me is I've seen in lots of organizations, they go, we need to build a dashboard and the project or we need to do BI, right? [Andy Cotgreave]: And then the project is sponsored by the CEO or some of the C-suite person. [Andy Cotgreave]: And so they define and build a system monitoring process. [Andy Cotgreave]: So let's say it's sales. [Andy Cotgreave]: So they are tracking sales by quarter to target and they can drill down to region and product type, perhaps, right?

13:20[Andy Cotgreave]: Brilliant. [Andy Cotgreave]: That is a high-level sales aggregated dashboard fitting in that criteria you've said. [Andy Cotgreave]: But the problem with those kind of project is they are aggregated and are useless for the people, for the millions, the mere normal people who are actually trying to do granular day granular work at the more curve face. [Andy Cotgreave]: So a dashboard we had at Tableau, which was hugely used, was, I think we called it the Who's Hot dashboard, and it was used by the sales organization.

13:48[Andy Cotgreave]: And it would show individual leads or individual people in our CRM and the activity they've been doing over a timeline [Andy Cotgreave]: on the website or the activity related to Tableau they've been doing. [Andy Cotgreave]: You can see a dot-plot on one row was one person. [Andy Cotgreave]: You can be like, visited the website, visited the website, did some training, visited the website, logged on, downloaded. [Andy Cotgreave]: That dashboard allows the person to look at their opportunity list and be like, Michael has been doing a whole bunch of watched all the training videos.

14:19[Andy Cotgreave]: I'm going to call Michael up and say, Michael, you've been doing some training, you should be talking about Tableau. [Andy Cotgreave]: And that is a dashboard that allows people to monitor disaggregated data. [Andy Cotgreave]: And it is a fundamental use of a dashboard. [Andy Cotgreave]: It's the disaggregated data visualized in a single screen or in multiple screens in that particular example. [Andy Cotgreave]: But that's useless for the C Chief Revenue Officer, because he doesn't get in the weeds with that, right?

14:45[Andy Cotgreave]: And so you have that spectrum of dashboards [Andy Cotgreave]: the need to be included inside any data strategy. [Andy Cotgreave]: And I get really frustrated when it's like, hey, we've built the sales tracking dashboards, off you go, I can't execs. [Andy Cotgreave]: And they're like, what can I do with that? [Andy Cotgreave]: Nothing. [Tim Wilson]: But I think you nailed it when it's like, there is this idea, we have all this data and dashboard is kind of the, so therefore, if we just build the right dashboards, then that is how as an organization, we become data driven.

15:18[Tim Wilson]: I'm trying to square the circle. [Tim Wilson]: I do tend to say a performance measurement dashboard, which is what maybe could be considered a scorecard, but I've got some kind of beefs with scorecards. [Tim Wilson]: I think what you just described, [Tim Wilson]: I don't think I'm in the habit of saying this is a dashboard. [Tim Wilson]: I absolutely think building something that is not aggregated, that is specific, that is part of a process.

15:48[Tim Wilson]: What you just described, I would say there is an operational process by which leads are being monitored and when you define it, they need information displayed that they understand so that they can follow their [Tim Wilson]: individual lead follow-up process, them as an individual having multiple leads. [Tim Wilson]: I haven't been in the habit of putting the label dashboard on it, but I think I may start because it makes sense. [Tim Wilson]: I'm loving this.

16:19[Julie Hoyer]: I have a little bit of a question for the group then. [Julie Hoyer]: It's interesting because I do feel like what happens to Tim's point and the pain I've felt a lot of times with clients is they want a dashboard that has big lofty promises and that they feel like if I just surface the data for them that all the interpretation [Julie Hoyer]: an actual synthesis of what those individual data points mean will just happen and they'll know what to do.

16:47[Julie Hoyer]: And obviously we know that's false. [Julie Hoyer]: So it's interesting how much interpretation [Julie Hoyer]: should a dashboard do for you? [Julie Hoyer]: Like in practice, a lot of times when I talk to stakeholders about a dashboard, I have ended up making documents where we actually outline like, here's the outcome you're going for. [Julie Hoyer]: Sometimes Tim, I can get them to a performance measurement type dashboard. [Julie Hoyer]: Sometimes I can't, but at least I can take that and map that to say like, these are the things you're trying to achieve or you're trying to understand.

17:18[Julie Hoyer]: I can turn that into a list of very concise, specific business questions that [Julie Hoyer]: actually have answers that are impactful for them to understand and then try to get them to something that is usable. [Julie Hoyer]: In the dashboards, then I usually put headers of the question the data point is answering, but that's like baking in a lot again of interpretation. [Julie Hoyer]: How much interpretation it has to come from an analyst has to be baked into a dashboard for it to be a good dashboard.

17:49[Julie Hoyer]: That's kind of a, I don't know what was the place question, but that's where my brain's going. [Tim Wilson]: I'm so proud. [Tim Wilson]: I'm so proud of that question you asked, Julie. [Julie Hoyer]: I was worried I was like, Tim, come for me. [Tim Wilson]: No, no, it's just all over the place and, you know, I'm like, it's not just me. [Andy Cotgreave]: So Tim, I think your statement belies the reason we don't really care about definitions because some people describe what we call as reports, they might be applications, they might be dashboards.

18:14[Andy Cotgreave]: It's like, [Andy Cotgreave]: What are you trying to achieve with this data and what do the customers need? [Andy Cotgreave]: I'm very interested in telling the fact that the semantics of what we're doing isn't important. [Andy Cotgreave]: It's the end goal that is. [Andy Cotgreave]: Julie, coming on to your question, [Andy Cotgreave]: I think it's, which I've now just forgotten. [Andy Cotgreave]: Now I'm answering Tim's question, I've just forgotten.

18:41[Andy Cotgreave]: I have the answer. [Andy Cotgreave]: Sorry. [Andy Cotgreave]: Who does the interpretation? [Andy Cotgreave]: Right. [Andy Cotgreave]: This is brutally hard, right? [Andy Cotgreave]: Because are all your users at the same level of data literacy or data fluency? [Andy Cotgreave]: Because that will change the way you design some of that. [Andy Cotgreave]: What we talk about in dashboards that deliver, we've got this whole framework.

19:03[Andy Cotgreave]: And it's about application design, agile development. [Andy Cotgreave]: It's really about thinking about user stories. [Andy Cotgreave]: So go to your stakeholders, work out as an account executive. [Andy Cotgreave]: I need to see who's active on my website in order to make a high chance, or I call it has a high chance of turning into a sale. [Andy Cotgreave]: Those user stories help define the answer to the question. [Andy Cotgreave]: And there isn't a single answer to the question, Julie.

19:33[Andy Cotgreave]: I wish there was, or I've not found a single answer to the question. [Tim Wilson]: But so as Julie, you described it, and I'm flashing back to my early days at Search Discovery, and there was a, I think it was a monthly report that was getting built entirely in what was Google Data Studio at the time. [Tim Wilson]: It was basically a presentation being built inside [Tim Wilson]: a BI platform, kind of a free, and it was really, and it was every month, it was kind of a, what's the story we're going to tell, and let's build it with these multiple tabs.

20:09[Tim Wilson]: And I was, that was a lot of work to build something that was kind of a one-time delivery. [Tim Wilson]: And when you're saying what question is being answered, I feel like, and I think this might get Andy to where your user stories, is this a question that I'm going to ask [Tim Wilson]: every day or every week or on a recurring basis, in which case I need a dashboard as a good way to say, here's the answer to that question. [Tim Wilson]: Or is this a question that I'm asking one time?

20:40[Tim Wilson]: Like, hey, was there any impact from that big traffic accident that happened wherever or some, you know, the airline strike? [Tim Wilson]: What was the impact? [Tim Wilson]: Answering that question, [Tim Wilson]: should be kind of a one time. [Tim Wilson]: Maybe it's a different type of dashboard. [Tim Wilson]: I think there are times where I've seen the somebody ask this, we might as well build it so it's automated and they can just continually get the answer to that question, but they're not gonna keep asking that question.

21:13[Tim Wilson]: Now we've just created another bit of jumbled clutter that they can go to and have to wade through to get the answer. [Julie Hoyer]: And that kind of comes back to what we were talking about too of like, is this an actual, is that something that should live in an actual dashboard that a business non-analyst user is going and looking at and having to look at 10 different data points to come to the conclusion of the one-time answer that the analysts did?

21:39[Julie Hoyer]: Or are we saying like, that's the type of question and then the type of [Julie Hoyer]: work data visualizations, whatever that's best left to the analysts and the engineers because they had to go into the interactive tool, dig and find it and it's not easily repeatable in some single visual that should live in a dashboard. [Andy Cotgreave]: I totally agree. [Andy Cotgreave]: I think anybody building out data strategies needs to realise that dashboards are just one tool in a toolbox, right?

22:07[Andy Cotgreave]: I think a great example of this is, I've done work with the Cabinet Office, which is the arm of the UK government that works under the Prime Minister. [Andy Cotgreave]: They have dashboard teams, but they also have a team that builds data narratives. [Andy Cotgreave]: When there are big cabinet meetings or COBRA, our emergency crisis meetings, they don't build dashboards. [Andy Cotgreave]: They build out narratives, and there will be charts with explanation, charts with explanation, because they've recognized that in that use case, and every business has use cases like this, a dashboard isn't the right thing, it's a narrative.

22:44[Andy Cotgreave]: Some of the things happened in the White House during the COVID years, though with dashboards and also narrative builders. [Andy Cotgreave]: In the end, the narrative builders were more favored by the administration in the White House at that point. [Andy Cotgreave]: And I think it's really important, and I'm sure we'll go on to AI, but AI brings another paradigm of exploring data. [Andy Cotgreave]: So dashboards just exist within this collection of options to which the hard job is trying to find the right one for the right purpose.

23:14[Tim Wilson]: So is that distinction of narrative versus the dashboard? [Tim Wilson]: Is that getting back to the storytelling dashboard? [Andy Cotgreave]: Yeah, absolutely. [Andy Cotgreave]: I think generally, I mean, Steve, co-authors of the book is Steve Wexler, Jeff Schofer, and Amanda McCulloch. [Andy Cotgreave]: Steve R. I chapter in dashboards that deliver about dashboards of a story finding, not storytelling. [Andy Cotgreave]: And I think that's a really important distinction.

23:38[Andy Cotgreave]: About 10 years ago, data storytelling was the hype in the industry, and dashboards can tell data stories. [Andy Cotgreave]: It's like, well, dashboards can answer four questions, maybe five questions, but probably four questions is the most. [Andy Cotgreave]: You might be able to design a grid flow that goes over Zoom and filter details on demand, like Ben Schneiderman's thing, but it's not really a story, is it? [Andy Cotgreave]: Find the insights, and then there's a whole skill and paradigm of how to then tell that story, which [Andy Cotgreave]: We don't address in this book, but I might in future books.

24:10[Tim Wilson]: I can't believe you're actually already thinking about the next book when the first one. [Tim Wilson]: I must not have chewed you up and spit you out. [Michael Helbling]: I just like that you've got things to say. [Michael Helbling]: Before you touched on AI and I want to spend a good amount of time on that, but before jumping off, we touched on this process you define in the book of [Michael Helbling]: how to build a dashboard. [Michael Helbling]: I would love to just actually go through that in a more of a step-by-step fashion because we touched on it, but I actually feel like that's actually such a helpful thing that's right there in the book, but I want to give people kind of a little taste of that.

24:47[Michael Helbling]: Do you mind just kind of stepping through? [Andy Cotgreave]: Yeah, so the framework is largely the mastermind of Amanda McCulloch and listeners might know her. [Andy Cotgreave]: She was exact director at DataVis Society and just an exceptional talent in process. [Andy Cotgreave]: And so the framework is largely her brainchild. [Andy Cotgreave]: And it's just like basically, what happens when you're building a dashboard? [Andy Cotgreave]: First of all, there is a spark.

25:12[Andy Cotgreave]: That spark can come from many different places. [Andy Cotgreave]: But at some point, somebody goes, we need a data [Andy Cotgreave]: asset, and that is going to be a dashboard. [Andy Cotgreave]: So we talk about when or when it isn't a dashboard. [Andy Cotgreave]: After that, you've got to go and talk to your users. [Andy Cotgreave]: So that is about discovering and prototyping. [Andy Cotgreave]: We took an inspiration there from the double diamond design approach.

25:33[Andy Cotgreave]: Go and talk to loads of users, get their requirements, and then prototype with wireframing or with data. [Andy Cotgreave]: Oh, and there was a debate we had internally. [Andy Cotgreave]: But all the way through this framework, talk to your users. [Andy Cotgreave]: Your users are part of the development team. [Andy Cotgreave]: So you've got your prototype, and then the next stage of the framework is development, and that's slightly circular, because again, that's very iterative.

25:59[Andy Cotgreave]: Then you go to user acceptance testing, release and adoption, how do you train your users. [Andy Cotgreave]: Then before you loop back to the start and spark, the time you come to redevelop or rehash the dashboard is, do you still need it? [Andy Cotgreave]: You know, can you deprecate it? [Andy Cotgreave]: Because we don't deprecate enough dashboards and many, many dashboards should die. [Andy Cotgreave]: I'm not here to say they should all live, but that's the framework Spark prototyping release, no, development release maintenance.

26:29[Andy Cotgreave]: And yeah, we really... What was the prototyping? [Tim Wilson]: Prototyping with data versus without data? [Tim Wilson]: What was that? [Andy Cotgreave]: Should you wireframe [Andy Cotgreave]: a dashboard in the same way you would wireframe an application, a website. [Tim Wilson]: I would say yes without data, because as soon as it starts, it's so hard to start putting either dummy data or, yeah. [Andy Cotgreave]: I mostly agree.

26:58[Andy Cotgreave]: But once I got sent once to a client site in Rotterdam, and this was a couple of sales opportunities that they've worked with, Jan Willem Tulp, who's a really great designer, and they've prototyped this dashboard. [Andy Cotgreave]: And they wanted you to go and build it in Tableau. [Andy Cotgreave]: And I'm like, brilliant. [Andy Cotgreave]: So Fleet of Rotterdam. [Andy Cotgreave]: And they gave me this beautiful, sketched, with Mark and Penn dashboard of beautiful line charts and bar charts and sketch plots.

27:23[Andy Cotgreave]: It was gorgeous. [Andy Cotgreave]: And I built it in Tableau. [Andy Cotgreave]: Tableau. [Andy Cotgreave]: I say the American accent in the print. [Andy Cotgreave]: Tableau. [Andy Cotgreave]: All right. [Andy Cotgreave]: I built it in Tableau. [Andy Cotgreave]: So 15 years working for this American company. [Andy Cotgreave]: Tableau. [Andy Cotgreave]: Stick to the bridge accent. [Andy Cotgreave]: Anyway, I built it and it looked like garbage.

27:38[Andy Cotgreave]: And I was like, why does it look like garbage? [Andy Cotgreave]: And it's because when you prototype, you draw a beautiful line chart, right? [Andy Cotgreave]: Maybe a little... And when you draw a scatter plot, you go, oh, look, a perfect regression and bar charts. [Andy Cotgreave]: But then when you put real-day training, your line charts go... And then scatter plots show chaos and bar charts, there's just one outlier. [Andy Cotgreave]: So that I do not... Why framing is amazing, but...

28:05[Andy Cotgreave]: But when you wireframe a static website or data application, the interface doesn't change, but the data drives the way your dashboard will look as well. [Andy Cotgreave]: So in the end, the customer wasn't very happy, but they did learn that their data, the dashboard they were building, wasn't actually good use for the trends in their data. [Tim Wilson]: But that is getting to such this fundamental how people believe that their business, that the data is cleaner, things are, there are trends, there are patterns.

28:35[Tim Wilson]: I mean, I think that gets back to that coming from the top down where we were earlier when somebody says, we need to build a dashboard. [Tim Wilson]: And Julie, it's kind of to your point, there's this assumption that if we just have the data visualized, oh, we're going to see the scatterplot with everything in that one outlier. [Tim Wilson]: And that outlier is going to just magically tell us how to drive the business forward or [Tim Wilson]: you name it, and then it's like, oh, actually then it becomes the dashboard is a way for them just to understand that when users actually look at it, the first time they look at it, they say, I've never been able to see this data before, this is great.

29:10[Tim Wilson]: And the third time they look at it, they're like, this all just looks kind of like noise to me. [Tim Wilson]: It's what I asked for, but I'm not getting anything new from it, which I think goes back to your user story [Julie Hoyer]: The other thing I was going to ask about too in the data and design and like do you design it with data or not when you're prototyping, my brain actually jumped to [Julie Hoyer]: making sure that what they assume is possible with their data is.

29:37[Julie Hoyer]: Because a lot of times, I was saying, we go through, what are your most crucial questions? [Julie Hoyer]: What are you trying to achieve? [Julie Hoyer]: And we know we have all XYZ data, all sitting there waiting for you to go build this for me. [Julie Hoyer]: And I'm like, okay, well, we got to get really specific. [Julie Hoyer]: What you're actually asking means we would need to be able to have [Julie Hoyer]: this type of rate and this type of data point and this type of relationship.

29:59[Julie Hoyer]: And lo and behold, it wasn't designed that way. [Julie Hoyer]: So we either have to go take the time to redesign your data and work with the engineering team to get it to do what we're asking it to do and make sure that it can do that. [Julie Hoyer]: And I think people are usually shocked by that. [Julie Hoyer]: So when you were saying like prototype it with data or not, my brain kind of went there, is that something [Julie Hoyer]: that you commonly run into?

30:23[Julie Hoyer]: Is it kind of baked into the process? [Andy Cotgreave]: Yeah, we talk about that in the book as well because I worked in marketing for so long in a tableau. [Andy Cotgreave]: And the amount of time is like, we're doing a campaign. [Andy Cotgreave]: We just can use a build as a dashboard for the campaign. [Andy Cotgreave]: It's like, build you a dashboard. [Andy Cotgreave]: All right, what do you think? [Julie Hoyer]: This is the scariest question.

30:40[Andy Cotgreave]: Yeah, I'm a tableau pro, so you assume that's half a day. [Andy Cotgreave]: It's like, it's going to take me seven days to find the data, goddammit. [Andy Cotgreave]: Right. [Andy Cotgreave]: So you're dead right, Julie. [Andy Cotgreave]: So we talk about that in the book. [Andy Cotgreave]: Yeah. [Andy Cotgreave]: And so getting to the data early in the process, not at the, but still be wirefroaming, but getting to the data reveals the trends that are not in the data and whether you've got the data in the first place or not.

31:04[Tim Wilson]: So yeah. [Tim Wilson]: I know we're going to get to AI, but I've got to ask you, using the dashboard as a way to drive some data collection, I'm sure there's somewhere in the book on this. [Tim Wilson]: having the dashboard, but having placeholders saying, this date is not available. [Tim Wilson]: You have declared that it's important and you want it and you can get it. [Tim Wilson]: We can't not roll this out, but we're going to go ahead because this is a designed entity.

31:30[Tim Wilson]: We're going to put a box here and it says data not yet available. [Tim Wilson]: I like doing that because it's [Tim Wilson]: is, to me, I see it often as a constant reminder that this is something that you said was important, that you said you needed, but it's not yet available. [Tim Wilson]: But I could also be not thinking about something, and that's a terrible idea. [Andy Cotgreave]: We don't have any examples in the book of empty boxes, but we do talk about that in the framework section.

31:59[Andy Cotgreave]: Don't let Perfect be the enemy of published. [Michael Helbling]: I had one thought that, and you touched on this a little earlier, which is sometimes the dashboard needs to be deprecated. [Michael Helbling]: A lot of companies struggle with this for various reasons and end up with hundreds, maybe even thousands of [Michael Helbling]: data artifacts, dashboards, reports that are almost impossible to sift through and need to be maintained.

32:26[Michael Helbling]: It basically burns out the data team because they're simply like carrying a massive carcass of old dashboards. [Andy Cotgreave]: Yeah. [Michael Helbling]: I'm trying to be a crizzly. [Michael Helbling]: I'm grizzly-dragging a carcass along. [Tim Wilson]: Technical debt for dashboards is carcassness. [Tim Wilson]: Okay. [Michael Helbling]: Yeah. [Michael Helbling]: So to refine that into a question is, in your experience and then all the work you've been doing over the years, how do you advise teams how to go through and step through a process where they do that hygienic step of getting rid of the ones that don't work or refactoring or whatever?

33:07[Andy Cotgreave]: So I think one of the things you can do, we talk about doing is right at the start, if you're building a new dashboard, put in an end date. [Andy Cotgreave]: So at what point will this be deprecated? [Andy Cotgreave]: And you could define that. [Andy Cotgreave]: So then you can define that as nobody's looking at it, but we'll come back to that. [Andy Cotgreave]: Or it might just have a finite date because it's related to a campaign.

33:29[Andy Cotgreave]: Or it might be we will review. [Andy Cotgreave]: The business owner will sit down and do a review in a year's time or something like this. [Andy Cotgreave]: There are various aspects like that. [Andy Cotgreave]: At Tableye, we've got a great process because we're allowed to publish dashboards internally, but they all get deprecated after two years, or after two years, if nobody's looking at it, it just gets deleted. [Andy Cotgreave]: The builder, the owner gets warned and they can go and refresh it or just delete it, but we just flush them out.

34:02[Andy Cotgreave]: There is a peril in deleting a dashboard that only one person looks at once a year, because it could be that one person looking at that one dashboard once a year makes a $10 million decision, right? [Andy Cotgreave]: And it's hard to capture that. [Andy Cotgreave]: But if they are doing that and the dashboard is gone, they'll be like, where is my goddamn dashboard? [Andy Cotgreave]: You're like, oh, now we realize that user had a use case.

34:22[Andy Cotgreave]: So in fact, actually, one of the dashboards in the book by Michael Gathers, who works at an IndyCar team, [Andy Cotgreave]: His definition of dashboard success is the amount of support requests he gets, because every support request is somebody engaging with the dashboard and frustrated they can't do what they wanted to do with it, right? [Andy Cotgreave]: So that's a slight aside. [Andy Cotgreave]: But yeah, it's those kind of things you can build into the process early to deprecate a dashboard, and then good governance, monitor usage, and just [Andy Cotgreave]: Flush is the one, Flush is the sandbox published ones away.

34:57[Tim Wilson]: I will say there's the, if you ask somebody or using this dashboard, that's bad data because they'll, even if they aren't, they'll get a, they're like, oh, but I should, they'll either think I should be and I'm going to start. [Tim Wilson]: So they'll say yes, or they'll be like, well, I know somebody, I had somebody else build it. [Tim Wilson]: I never really got that much use out of it, but I don't want to hurt their feelings. [Tim Wilson]: And I don't want to be, I don't want them telling me that, why did I even request it?

35:23[Tim Wilson]: If there is not the ability within the platform to monitor usage, I'm a fan of dropping the big old red text at the top saying, this will be deleted. [Tim Wilson]: This will be on whatever this date. [Tim Wilson]: Know when you put that up and give them plenty of warning. [Tim Wilson]: It doesn't necessarily solve the one year. [Tim Wilson]: problem, but I had an analyst who worked for me years ago who just refused to, because the number of people she sent the report out to was vast and included some very high senior people, she was like, this must be important because I am spending hours of my week compiling this and sending it out.

36:01[Tim Wilson]: And I was like, but what are they using it for? [Tim Wilson]: She's like, I don't know. [Tim Wilson]: Look at the size of this distribution list. [Tim Wilson]: And so I finally told her she needed to start including in the email, we're going to stop sending this. [Tim Wilson]: And I was like, can you tell me [Tim Wilson]: who comes back, and then my point being, instead of trying to solve, think you're solving everything for everybody, there probably are some specific use cases in there, the people who are going to freak out, but they need one thing once a quarter.

36:28[Tim Wilson]: You don't need to be sending everything weekly so that once a quarter they can get it. [Tim Wilson]: But I love the planning and end date at the outset. [Tim Wilson]: People will set it two years out and they'll be like, it's two years. [Tim Wilson]: Two years will be there before they know it. [Tim Wilson]: Yeah. [Andy Cotgreave]: If they even release it in two years, hopefully they do. [Michael Helbling]: We keep dashboards like we keep gym memberships.

36:53[Michael Helbling]: There's a lot of for for data people like there's dashboards are exceptional and useful, but there's a lot of pain buried in there as well over the years. [Michael Helbling]: All right, Julie, you were about to say something, I think. [Julie Hoyer]: push off our AI conversation with one more question. [Michael Helbling]: Oh boy. [Michael Helbling]: But I'm sorry I have to. [Michael Helbling]: Okay, good. [Julie Hoyer]: Because we were talking about like that one person that's making a $10 million decision on a dashboard.

37:25[Julie Hoyer]: Have you seen, do you have thoughts on in practice, what are some of the best ways people should be sharing learnings that they're getting from dashboards, like across a team? [Julie Hoyer]: Or I think I had seen like, [Julie Hoyer]: You had part of the book talking about like there again and we've talked about different types of users right so they're looking at the data asking questions with different lenses different purposes that they need the answers for.

37:51[Julie Hoyer]: Do you have any guidelines for people or seen it in practice done really well like how do you communicate. [Julie Hoyer]: what people on the same team are gaining from a dashboard or people on different teams are getting from a dashboard. [Julie Hoyer]: And it's hard to say, will it ever get back to the analysts that's like, can I deprecate this thing or not? [Andy Cotgreave]: But I'm interested to hear. [Andy Cotgreave]: One thing I've learned over speaking to thousands of customers in my time at Tableau is that a refrain I hear so often is we spend the first half of every meeting arguing about the data.

38:30[Andy Cotgreave]: We need a solution to that. [Andy Cotgreave]: And I'm like, fine, that's true. [Andy Cotgreave]: But then I try and spin it. [Andy Cotgreave]: But what if that was actually a positive thing? [Andy Cotgreave]: What if you had a good robust data reporting system that told you [Andy Cotgreave]: average sales are going up, but mean and median sales are not doing the right thing, or they could have that reliable data that they trusted in so that they could spend half an hour arguing about the data, because then they're having a data-informed conversation rather than a data argument.

39:01[Andy Cotgreave]: Organizations that bring data into conversations. [Andy Cotgreave]: For a start, data is never clean. [Andy Cotgreave]: There are so many [Andy Cotgreave]: There are just so many problems in the data pipeline at every step in the way that data is now the perfect. [Andy Cotgreave]: So it is never going to be able to be the 100% informer of any decision. [Andy Cotgreave]: There is human intuition. [Andy Cotgreave]: There is just human politics and ego in making decisions.

39:25[Andy Cotgreave]: And so we should embrace that. [Andy Cotgreave]: And then it comes a little bit back in storytelling. [Andy Cotgreave]: One of our best European leaders at Tableau, he always brought in every all hands meeting for Europe. [Andy Cotgreave]: and Tableau, he would always come to each meeting with a brand new chart. [Andy Cotgreave]: I mean, he added a couple of people who would work on these every month. [Andy Cotgreave]: I thought that was really good because if you sat in a meeting every month and your head of sales or your head shows the same dashboard every month, it looks the same every month.

39:54[Andy Cotgreave]: Three months into watching those meetings, you just don't pay attention because it's like, oh, here's the bit where James shows the dashboard. [Andy Cotgreave]: Good God, it's the same as if it's boring. [Andy Cotgreave]: Whereas he would be like, okay, I've looked at the dashboard and I'm going to visualize one insight I've taken from that. [Andy Cotgreave]: You'd be like, oh, here's James telling an insight visually with data.

40:13[Andy Cotgreave]: So I don't have, there's no specific framework answer to that Julie, but I just do examples that are what I've seen, what I've seen success and what talking about that to reserve, right? [Tim Wilson]: I mean, counterpoint, the arguing about the data or the bringing a new... I've seen cases where if the argument is about the messiness of the data, why do these numbers not match? [Tim Wilson]: And then everybody can get caught up on what is the right number or if somebody brings a new charge and it's surprising, then people's tendency to say, well, did you exclude this?

40:47[Tim Wilson]: Did you do that? [Tim Wilson]: And in every case, there would be a... [Tim Wilson]: probably eventually some general consensus that, yes, maybe returns should have been filtered out, even if it doesn't material to change what that person is showing. [Tim Wilson]: Then everybody gets sent off swirling on trying to get to [Tim Wilson]: agreement on the definition and there's a lot of work and discussion and deeper understanding of the data not of the business and everybody.

41:18[Tim Wilson]: There's a sense that wow we did something because we had an argument and we resolved it and like all you've resolved is. [Tim Wilson]: some definitional shit, you haven't necessarily actually resolved anything that's going to move the business forward. [Tim Wilson]: I think that's where I struggle. [Andy Cotgreave]: Tim, I'm going to force the segue now, because if humans can't do it, how the hell would a janitor AI have that contact to know those things that even we can't work on?

41:48[Tim Wilson]: Well done. [Michael Helbling]: This man is a professional. [Tim Wilson]: Michael, he's coming for your job. [Michael Helbling]: But yeah, so that's the latest type of course in every space. [Michael Helbling]: We all have this mandate to leverage AI, use AI, where can AI do all this? [Michael Helbling]: So in the world of data and dashboards, yeah, what are you seeing in the space as it relates to AI? [Andy Cotgreave]: Let's talk about ThoughtSpot.

42:11[Andy Cotgreave]: They were early in natural language approach to querying data, and ThoughtSpot are great. [Tim Wilson]: This was hilarious if this is going to go away. [Andy Cotgreave]: That genius marketing campaign was dashboards like that. [Andy Cotgreave]: Come and do Natural Language. [Andy Cotgreave]: It's a brilliant campaign. [Andy Cotgreave]: I love it. [Andy Cotgreave]: You go to ThoughtSpot's homepage. [Andy Cotgreave]: and watch the demo and says, you can ask any question.

42:35[Andy Cotgreave]: When you've got a chart you like, you save it to a live board, which is what we call dashboard. [Andy Cotgreave]: Right. [Andy Cotgreave]: So on the one hand, generative AI, LLMs promise infinite answers to infinite possibilities [Andy Cotgreave]: Data related to Data2. [Andy Cotgreave]: That's fantastic, but I'm lazy. [Andy Cotgreave]: If I go ask the same question every week or every month, I don't want to type that question every week.

43:00[Andy Cotgreave]: I just want to open my phone, go to say Tablet Pulse, for example, which is really good at this, and it's just like, oh, my metrics are going up, my metrics are going down. [Andy Cotgreave]: Put my phone down and move on. [Andy Cotgreave]: I don't want to type that every time. [Andy Cotgreave]: And thoughts won't recognize that with their live boards. [Andy Cotgreave]: People want to go back to those things. [Andy Cotgreave]: So on the one level, people still want at a glance ways to monitor and explore data to facilitate understanding.

43:25[Andy Cotgreave]: Hey, that's a dashboard. [Andy Cotgreave]: Then the second aspect of AI [Andy Cotgreave]: is this context thing we've just talked about. [Andy Cotgreave]: If we can't solve this ASP challenge, then Generative VR isn't going to do that. [Andy Cotgreave]: There's a huge amount of efforts around semantic loads across the industry to try and define business data. [Andy Cotgreave]: We have been trying to do that for about 50 years. [Andy Cotgreave]: We haven't done it yet.

43:48[Andy Cotgreave]: So I mean, you know, there's more of a catalyst to get that shit done and get it done properly, but processes humans are lazy and under pressure. [Andy Cotgreave]: And then using generative AI as a data analyst, I mean, it's when I do a data project, [Andy Cotgreave]: to build a dashboard for this. [Andy Cotgreave]: So, okay, fine. [Andy Cotgreave]: Who's got the data? [Andy Cotgreave]: I've got to email that person and get them to give me the password.

44:13[Andy Cotgreave]: They haven't got a password, it's a local file, so they send me the copy of the Google sheet. [Andy Cotgreave]: Then I connect it somehow to the core data systems. [Andy Cotgreave]: They don't actually match properly, but you can't really do the matching on the thing, so I have to manually connect the data. [Andy Cotgreave]: Then I have to find another data set, but I have to broadcast the message, because who owns this data, right?

44:30[Andy Cotgreave]: Hey, I can't get over that stuff. [Andy Cotgreave]: That's human processes, because data is messy as anything, right? [Andy Cotgreave]: Right. [Andy Cotgreave]: So, so far, in this answer to the question, I'd be down on generative AI. [Andy Cotgreave]: I think it is unreliable for data analysis. [Andy Cotgreave]: It is unreliable. [Andy Cotgreave]: It doesn't understand business context, and it can't access the things away human can.

44:52[Andy Cotgreave]: It's not all useless. [Andy Cotgreave]: There are some things it can do pretty well. [Andy Cotgreave]: But in all my experience of trying to use AI to do data projects, I've been left [Andy Cotgreave]: wholly underwhelmed and really frustrated. [Andy Cotgreave]: I just thought I could have just done this quicker myself. [Tim Wilson]: There's this idea. [Tim Wilson]: If you probe when somebody, if they said, well, I need on the dashboard, I need to see [Tim Wilson]: sales training over time broken down by geography.

45:22[Tim Wilson]: You'd say, well, okay, you could ask AI that and maybe it could produce it, but aren't you going to always want to look at it? [Tim Wilson]: That's a lot more work for you to continue to ask to your first point, ask that question again and again. [Tim Wilson]: That's still not really what they want. [Tim Wilson]: I feel like there's this nebulous, optimistic, [Tim Wilson]: thing that no, the question, I'm not even going to ask, I'm just going to have an agent that is just going to tell me insights and I'm not going to have to ask it anything.

45:55[Tim Wilson]: Like that's sort of the path that's going to like just trying to completely cut out the human, which I don't feel like is all that [Tim Wilson]: different from past iterations of, that's what digital was going to do. [Tim Wilson]: All of a sudden, we're going to have all of this data, and therefore, this will be just a shortcut. [Tim Wilson]: We'll be able to do one-to-one marketing. [Tim Wilson]: Never worked out. [Tim Wilson]: There's a little bit of everything old is new again.

46:21[Tim Wilson]: There's this weird thing where people think this is the technology that if they really probe, they want it to just tell them what to do. [Tim Wilson]: And then actually they're like, well, no, just do it for me. [Tim Wilson]: It's like, well, what are you as a human doing? [Tim Wilson]: Do you have that low regard of yourself that you think that you should just be entirely replaced? [Tim Wilson]: Like somewhere thinking through, what do I want to look at?

46:49[Tim Wilson]: Where does that data come from? [Tim Wilson]: What does it mean? [Tim Wilson]: There is value in the friction, even for non-analysts to understand the nuances of the business. [Tim Wilson]: What is a monthly active user? [Tim Wilson]: What does that really mean? [Tim Wilson]: And that's not a data question. [Tim Wilson]: That is a, what do we in our business strategy think matters? [Tim Wilson]: And if we just were able to ask a chat, he'll just tell me Mao's over, tell me Mao anomalies.

47:21[Tim Wilson]: And I haven't actually taken the time to understand what that really means. [Tim Wilson]: So yeah, I'm similarly frustrated [Andy Cotgreave]: I've tried the exercise with customers. [Andy Cotgreave]: If you want the AI to be able to ask and answer any question, write down, so let's say it's to do with active users, write down as granular as you can the list of every single thing that has to be accurately defined or anticipated in order for an AI to know where that it can look across loads of data source.

47:48[Andy Cotgreave]: That list is so long. [Andy Cotgreave]: Then you talk about this digital was a thing. [Andy Cotgreave]: I'm going to tell you my favorite quote. [Andy Cotgreave]: Right? [Andy Cotgreave]: And ask you first, do you kind of agree with this? [Andy Cotgreave]: So millions of dollars are spent every year collecting data with the assumption that having the data solves the problems being studied. [Andy Cotgreave]: Nods, yes, does that seem reasonable?

48:09[Andy Cotgreave]: Millions of dollars being spent yearly collecting? [Andy Cotgreave]: Yes. [Andy Cotgreave]: Right. [Tim Wilson]: Oh, yeah. [Andy Cotgreave]: So second question, what year is that quote from? [Tim Wilson]: 1968. [Tim Wilson]: 68, says Tim. [Michael Helbling]: Michael. [Michael Helbling]: I mean, I don't have no idea, but I'm with Tim that it's probably a way longer to go than we even think. [Andy Cotgreave]: Well, it's from this book.

48:32[Andy Cotgreave]: I'm now holding up Graphic Methods for Presenting Facts by Willard Co. [Andy Cotgreave]: Brinton, printed in 1914, 110 years ago. [Andy Cotgreave]: He said, page one, we're spending millions of dollars collecting data and it doesn't solve the problems. [Andy Cotgreave]: Millions of dollars, right? [Andy Cotgreave]: And this whole book is about saying up a data culture, about being data analysts, creating curves and charts for executives and presenting data [Andy Cotgreave]: And this book gives me an existential crisis because we haven't solved the problem in 110 years.

49:01[Michael Helbling]: I was going to say that doesn't make me feel any better at all. [Andy Cotgreave]: But what I've come to realize is that there is a frontier. [Andy Cotgreave]: We have a bunch of data. [Andy Cotgreave]: And we have technology which can create a frontier of what that data and technology can do. [Andy Cotgreave]: And then since Brin and Stein, we've just pushed that our human ideas go beyond that frontier. [Andy Cotgreave]: So we will always be frustrated.

49:26[Andy Cotgreave]: Whatever happens in the next 20 years, the next technological innovations, they'll be frustrating. [Andy Cotgreave]: But at that point, AI will have walked its way into our lives and we'll just be using it and things will be better. [Andy Cotgreave]: The problems are not new, right? [Andy Cotgreave]: And I'll have to wrestle with that every day since I read this gamble. [Michael Helbling]: I do sort of see some really interesting use cases where people could leverage AI as part of the process.

49:51[Michael Helbling]: So in the discovery phase, [Michael Helbling]: Use AI to pull together common themes and make sure you're getting good visibility to everything you've discovered in all those conversations with users in the prototyping phase. [Michael Helbling]: Get AI to help decide, like, am I using the right kind of visualization here? [Michael Helbling]: What other alternatives might be good for this use case? [Michael Helbling]: Or in the development, help me extend my coding abilities or my data normalization abilities beyond what I'm capable of.

50:18[Michael Helbling]: And so that's sort of, I've got a friend, Donal Phipps, who commented [Michael Helbling]: talking to him. [Michael Helbling]: Two days ago. [Michael Helbling]: Augmentation over automation. [Michael Helbling]: And I think that's kind of the framework that as analysts, something we can all sort of say, okay, yeah, it's not going to replace what we do. [Michael Helbling]: We can't. [Michael Helbling]: But there's all these different places within a great model or context from the book, we can leverage AI to accelerate our efforts or augment the efforts that we're putting in.

50:50[Andy Cotgreave]: Automation, not automation. [Andy Cotgreave]: I think that's a brilliant three-word summary. [Andy Cotgreave]: The challenge is that that doesn't satisfy the business model. [Andy Cotgreave]: These tech companies are working on that because they are waiting for the massive enterprise scale solution and it isn't there because it can't be trusted. [Tim Wilson]: But it is funny how it goes that way. [Tim Wilson]: You start, is it going to kill dashboards?

51:10[Tim Wilson]: And you're like, no, no, no, you wind up with the defender saying, [Tim Wilson]: basically saying, well, it's useful for prototyping the dashboard and it's useful for evaluating the dashboard and thinking about what's going to go on the dashboard. [Tim Wilson]: And somehow you're like, see, I proved to you that GenAI is, and it's like, no, nobody's saying that it's not useful and valuable, but that's a different conversation about its use in the context of traditional analytic delivery of information.

51:45[Julie Hoyer]: I'm curious, do you think that using AI in the interpretation of data [Julie Hoyer]: would end up helping people. [Julie Hoyer]: Tim, I'm thinking about how you build a dashboard. [Julie Hoyer]: It has beautiful visualizations. [Julie Hoyer]: It has clear trends, lines, things. [Julie Hoyer]: You can see what's going on. [Julie Hoyer]: But at the end of the day, we always run into how do you help the business actually correctly interpret, like, this is time series data.

52:17[Julie Hoyer]: Or here are the caveats. [Julie Hoyer]: Or this is expected variation. [Julie Hoyer]: So variation, time series, those types of things. [Julie Hoyer]: Do you think that there is an opportunity that AI would be able to help people answer the questions around, like, is this a material change I'm seeing in the data? [Julie Hoyer]: Or, I mean, I'm guessing that's kind of far-fetched. [Julie Hoyer]: I don't even know if people would know to ask that question.

52:38[Andy Cotgreave]: Yeah, I think that there's huge opportunities and many people across the industry are chasing that. [Andy Cotgreave]: So Tableau Poults, [Andy Cotgreave]: is Traction Metrics, but uses a narrative science to just interpret the data and put it into text. [Andy Cotgreave]: You can paste charts into ChatGP to include in Haskit to explain the visuals. [Andy Cotgreave]: So that's two things. [Andy Cotgreave]: So the narrative based on the data is kind of bland because it's quite hard coded as text.

53:10[Andy Cotgreave]: It's like [Andy Cotgreave]: You're almost pre-defining what the paragraphs are going to say. [Andy Cotgreave]: As an end user, if you see that 10 times, you probably stop paying much attention to it. [Andy Cotgreave]: But then if you just get the AIs to interpret a visual, it's so reliable. [Andy Cotgreave]: I've been through that pain, and you just explain this chart, and it just makes stuff up. [Andy Cotgreave]: Because it's a probabilistic, stochastic approach, it just makes things up.

53:41[Andy Cotgreave]: It's hard to rely on it. [Andy Cotgreave]: Yes, it's got huge potentials. [Andy Cotgreave]: The pitfalls are huge. [Andy Cotgreave]: As data analysts, we spent decades trying to build up trust in our stakeholders. [Andy Cotgreave]: This is from Tableau Tyn from his video. [Andy Cotgreave]: I'm stealing his quote here, but it takes minutes to lose that trust. [Andy Cotgreave]: And if we give them tools that say sales went up when they went down, that would be a terrible indictment on what we do as analysts.

54:15[Andy Cotgreave]: opportunity currently high risk. [Tim Wilson]: I got to make my point here because I think there is a part where even the human work, put in front of an analyst and I had this happen years ago when I was in an agency and they had outsourced the client, big CPG, I think it was this was P&G, had outsourced the polling of the reports to offshore. [Tim Wilson]: And then they said, and this was going to have us not the, the, the on short agency was going to get less of the work.

54:45[Tim Wilson]: And they said, they're just going to send you the reports and you'll look at the data and like interpret them. [Tim Wilson]: And so I think there's opportunity for AI to move upstream with [Tim Wilson]: the business users to say, let's help you really formulate questions and ideas, and not, I'm looking at a dashboard. [Tim Wilson]: I mean, that's just, I die a little bit anytime someone's like, we've pulled all the data together, we've built all these dashboards, they're amazing, they follow all the bright principles, okay, this has happened to me multiple times in my career.

55:20[Tim Wilson]: Can you come in and Tim and tell us how to look at those dashboards and get insights from them? [Tim Wilson]: And I've gotten pushback. [Tim Wilson]: I'm like, let's not look at those. [Tim Wilson]: Let's talk about your business for a while. [Tim Wilson]: What's keeping you up? [Tim Wilson]: What are your challenges? [Tim Wilson]: What are your ideas? [Tim Wilson]: And then come in with a much, much narrower view of what specific question we need answered, maybe even helping refine what data would we look at and what should we be concerned about?

55:45[Tim Wilson]: And then arriving at the chart with a much, much more clear [Tim Wilson]: purpose, we're going to see if sales went up. [Tim Wilson]: When you say went up, are you clear? [Tim Wilson]: Are we talking about sales going up a material amount, an amount that doesn't look like it's just gauging as noise? [Tim Wilson]: I think huge potential for AI to patiently sit there and be introduced [Tim Wilson]: good friction to the process to say, you're not going to jump to looking at charts and then think AI is going to glean insights from it.

56:22[Tim Wilson]: It's like giving somebody a bag of rocks and saying, find the valuable ore in this rock. [Tim Wilson]: Well, they could bust them up. [Tim Wilson]: They could look at them. [Tim Wilson]: There's no guarantee that there's valuable ore in that rock. [Tim Wilson]: So you're starting kind of farther, too far downstream. [Andy Cotgreave]: Oh, well, I'll say that it sounds like you've read the first section of Dashboards to Deliver, which is all about... For the turn of hell we had on that.

56:43[Michael Helbling]: Damn, to bring it full circle. [Michael Helbling]: Andy, incredible. [Michael Helbling]: We do have to start to wrap up. [Michael Helbling]: And that is the new book, Dashboards that Deliver How to Design, Develop and Deploy Dashboards that Work. [Michael Helbling]: This has been so fun to have this conversation and I think very helpful and insightful. [Michael Helbling]: So thank you very much, Andy, for coming on the show. [Michael Helbling]: And now it's time for something I'm really excited about.

57:11[Michael Helbling]: We're going to take a quick break to hear [Michael Helbling]: here from Alfred of ours, Michael Kaminski from Recast, the Media Mix modeling and GLIF platform helping teams forecast accurately and make better decisions. [Michael Helbling]: Michael's sharing some bite-sized marketing science lessons over the coming months, and we're happy to have him on the show to help you measure smarter. [Michael Helbling]: Hope for to you, Michael.

57:31[Michael Kaminsky (recast)]: What exactly is power analysis? [Michael Kaminsky (recast)]: I get asked this question a lot. [Michael Kaminsky (recast)]: Power analysis is the process of trying to understand the limitations of an experiment prior to actually conducting. [Michael Kaminsky (recast)]: When we're designing an experiment, we face trade-offs. [Michael Kaminsky (recast)]: We want to generate the most useful conclusions as cheaply as possible.

57:46[Michael Kaminsky (recast)]: Power analyses allow us to understand this trade-off between the conclusions we'll be able to draw and the cost of the experiment. [Michael Kaminsky (recast)]: The most generalized form of a power analysis is a simulation, where we code up the statistical analysis we're going to do once the experiment concludes, and we apply that same statistical test to hundreds or millions of simulated experiments, showing us under what conditions we'll draw the correct conclusions.

58:06[Michael Kaminsky (recast)]: We might see that with the amount of lift we expect to see, we need a sample of at least 250 participants to draw the correct conclusion 99% of the time. [Michael Kaminsky (recast)]: But if we accept some risk and are okay if we only draw the correct conclusion 95% of the time, maybe we only need 100 participants. [Michael Kaminsky (recast)]: The main variables that affect our statistical power are the assumed underlying variation in the data, the assumed effect size generated by the intervention being tested, and the sample size.

58:30[Michael Kaminsky (recast)]: So what's the takeaway? [Michael Kaminsky (recast)]: Power analyses are simulation exercises that help you understand the limits of your experiments and their expected cost before you run them. [Michael Helbling]: All right, well, if you enjoyed that mini lesson, Michael and the team at ReCast have put together a library of marketing science content specifically for analytics power hour listeners. [Michael Helbling]: For everything from building media mix models in-house to communicating uncertainty to your board, head over to www.getrecast.com slash aph.

58:58[Michael Helbling]: That's www.getrecast.com slash aph. [Michael Helbling]: All right, well, one other thing we love to do on the show is go around the horn and share a last call. [Michael Helbling]: Something might be of interest to our users. [Michael Helbling]: And you're our guest. [Michael Helbling]: Do you have a last call you'd like to share? [Andy Cotgreave]: Yeah, I'll go. [Andy Cotgreave]: Let's go to the movies. [Andy Cotgreave]: Summer 2025.

59:17[Andy Cotgreave]: I recently, I'm not a massive Marvel Universe fan, but I did go and see First Steps or Fantastic Four First Steps. [Andy Cotgreave]: Why is that relevant to your users? [Andy Cotgreave]: Because they have some cool data displays in that film, right? [Andy Cotgreave]: In fact, the visual design of that movie is fantastic. [Andy Cotgreave]: And there are some really exceptional data displays. [Andy Cotgreave]: And so go check out the movie.

59:40[Andy Cotgreave]: And also, now I've pointed it out to the audience. [Andy Cotgreave]: you will never not see chance when they appear in movies. [Andy Cotgreave]: It's brilliant. [Andy Cotgreave]: They're everywhere. [Andy Cotgreave]: I'm a really good speaker. [Michael Helbling]: Awesome. [Michael Helbling]: All right, Julie, what about you? [Michael Helbling]: What's your last call? [Julie Hoyer]: Mine's in the entertainment space a little bit too.

59:56[Julie Hoyer]: A little different than my other last calls. [Julie Hoyer]: But honestly, I am a sucker for any kind of sports documentary, honestly. [Julie Hoyer]: And there was one I recently watched. [Julie Hoyer]: It's Power Moeves. [Julie Hoyer]: It's about Shaquille O'Neal coming and helping Reebok basketball and they could come back. [Julie Hoyer]: And I am really curious because, of course, when those come out on Netflix, it's so delayed. [Julie Hoyer]: So I was talking about last year, I think the launch of their first basketball shoe was earlier in 2025.

1:00:25[Julie Hoyer]: And I immediately was like, I want to know, is Reebok basketball really going to make a comeback from this? [Julie Hoyer]: Do people love the shoes, hate the shoes? [Julie Hoyer]: What are their dashboards look like internally? [Julie Hoyer]: What's their revenue doing? [Julie Hoyer]: I was just kind of like, did he make the comeback that he was trying to do? [Julie Hoyer]: So I'm kind of invested now to [Julie Hoyer]: try to dig in and see if.

1:00:44[Michael Helbling]: All right, so if you're from Reebok, reach out to Julie to give her the update on how things are going. [Tim Wilson]: Yeah, I want to know. [Tim Wilson]: Well, who's the kid from Maine who got drafted first in the NBA because he wears Converse? [Tim Wilson]: Anybody? [Tim Wilson]: Cooper Flagg? [Tim Wilson]: What is Cooper Flagg? [Tim Wilson]: His shoes are... I've heard the name. [Michael Helbling]: I don't know what shoes he wears.

1:01:06[Tim Wilson]: New Balance, I think. [Tim Wilson]: Yeah. [Tim Wilson]: All right, Tim, what is your last thought? [Tim Wilson]: Mine's like, right, totally in entertainment as well. [Tim Wilson]: It's also pandering to our new sponsor, ReCast, unintentionally. [Tim Wilson]: What are priors in MMM and why they're difficult to get right? [Tim Wilson]: But you need to. [Tim Wilson]: I don't know that it's on Prime Video just yet, but I'm sure it's going to be made into a movie.

1:01:36[Tim Wilson]: But it's a long piece. [Tim Wilson]: Marty Sanchez at Recast wrote it. [Tim Wilson]: Because I've lived in the surface level of how does Bayesian differ from Frequentist? [Tim Wilson]: Well, you have priors, then you adjust your priors. [Tim Wilson]: So it's a pretty deep dive that I kind of committed to reading it two or three more times and I'll still be a little bit confused, but it's really well written. [Tim Wilson]: That has not a knock on Marty, it's a knock on my understanding, but it's a good read on priors and media mix modeling and not the NBA or the Marvel Cinematic Universe.

1:02:13[Tim Wilson]: Yeah. [Tim Wilson]: And that was for the right person. [Tim Wilson]: Yeah. [Tim Wilson]: To Tim. [Tim Wilson]: What do you do for fun? [Tim Wilson]: Michael, what's your last call? [Michael Helbling]: Yeah. [Michael Helbling]: We're aware. [Michael Helbling]: OK, so over the years, there's been this conference in the US that's gone by a couple different names, Exchange, DA Hub. [Michael Helbling]: And it's always been one of my favorites.

1:02:36[Michael Helbling]: It's got a huddle-based format. [Michael Helbling]: So you're sitting down with and having these conversations [Michael Helbling]: with fellow practitioners, people from all these different companies. [Michael Helbling]: And so the signal-to-noise ratio is super good. [Michael Helbling]: You're not getting talked at by a bunch of speakers all the time, and it's coming back. [Michael Helbling]: It's called the Data Exchange Conference, and it's happening October 27th to the 29th.

1:03:00[Michael Helbling]: in Asheville, North Carolina. [Michael Helbling]: So I'm really excited about that. [Michael Helbling]: And I'm excited to see that conference emerge again, because I feel like we need something like that for business users, data users to be able to share insights and knowledge with each other in that kind of format. [Tim Wilson]: So that's my last call. [Michael Helbling]: We're going to be at the Biltmore Estate. [Michael Helbling]: It's not at the Biltmore, but a good question.

1:03:24[Michael Helbling]: I think there's another hotel or property that they're hosting it at, but Asheville is a beautiful city. [Michael Helbling]: And it's also right there, Asheville is coming back from those major hurricanes that hit Western North Carolina. [Michael Helbling]: So even going there and being part of it is also a part of bringing that region back from some major disaster. [Michael Helbling]: So you can even feel good about that too, and if you go.

1:03:48[Michael Helbling]: Okay. [Michael Helbling]: Obviously, we've been talking about dashboards and design of them and how to do that. [Michael Helbling]: And I'm sure you have thoughts and questions. [Michael Helbling]: We would love to hear from you. [Michael Helbling]: So please reach out to us. [Michael Helbling]: The best way to do that is through either LinkedIn or the Measure Slack chat group. [Michael Helbling]: or you can email us at contact at analyticshour.io.

1:04:11[Michael Helbling]: Andy is going to be out in the world. [Michael Helbling]: His new book is available in just like a week. [Michael Helbling]: So go to Amazon or wherever you buy books and check that out. [Michael Helbling]: I'm sure that you will see him and bring your books with you so you can get him to sign them when you see him out in the world. [Michael Helbling]: And Andy, thank you so much. [Michael Helbling]: This has been such a fun conversation and such a needed one, I think, right at this moment in time.

1:04:35[Michael Helbling]: My pleasure. [Michael Helbling]: Thanks, everybody. [Michael Helbling]: And no matter what you do out there with your dashboards, remember, I think I speak for both of my co-hosts when I say, keep analyzing. [Announcer]: Thanks for listening. [Announcer]: Let's keep the conversation going with your comments, suggestions, and questions on Twitter at, at Analytics Hour, on the web at analyticshour.io, our LinkedIn group, and the Measure Chat Slack group.

1:05:01[Announcer]: Music for the podcast by Josh Crowhurst. [Charles Barkley]: Those smart guys wanted to fit in, so they made up a term called analytics. [Charles Barkley]: Analytics don't work. [Michael Wilbon]: Do the analytics say go for it, no matter who's going for it? [Michael Wilbon]: So if you and I won the field, the analytics say go for it. [Michael Wilbon]: It's the stupidest, laziest, lamest thing I've ever heard for reasoning in competition.

1:05:24[Julie Hoyer]: I was waiting for Carlos to come back. [Tim Wilson]: Oh no, that's... Even though I'm the one who gets stuck doing that, I start panicking, like, as Michael Frapping up, saying, what? [Tim Wilson]: I know there was something. [Tim Wilson]: Oh. [Tim Wilson]: Yeah. [Tim Wilson]: I literally have to write them down during... I forget to do that. [Tim Wilson]: That would have been a... Oh, yeah. [Tim Wilson]: Damn it. [Tim Wilson]: Someday we'll figure out how to actually do intros with asynchronous arrival.

1:05:52[Michael Helbling]: Well, you know, I wanted to make sure to get some shop talk about book publishing in there, you know. [Andy Cotgreave]: All right. [Andy Cotgreave]: Not out of that cool club. [Andy Cotgreave]: Last thing, I'm just going to tell my daughters to ensure that. [Andy Cotgreave]: Yeah, that's totally fine. [Andy Cotgreave]: You're all good. [Tim Wilson]: Just scream it for where you're standing. [Andy Cotgreave]: Yeah, that's the American way.

1:06:13[Andy Cotgreave]: I'll be back in a minute. [Michael Helbling]: Nobody respects what I'm trying to do here, so, you know. [Tim Wilson]: Rock flag and dashboards are not dead. [Tim Wilson]: That's the verdict.