Will AI Replace Doctors? You're Asking the Wrong Question

0:04SPEAKER_00:I don't know about you, but listen in, we're all incredibly time poor. There are not enough hours in the day, and I love AI. I think AI is an amazing tool to make my life easier, to make me better at doing what I'm doing. But a lot of people are really scared about AI. They're scared to have a go. There's a lot of mystery around it. And as normal, the NHS is slow to the table. Um, welcome to WhyMatics Matter. My name's Simon Fraser, and I'm had the pleasure of being joined by Yasmin Hyatt, who is a trainee who is mastering her specialty fantastic, but is also spending a huge amount of time mastering AI.

0:53SPEAKER_00:Thank you, Yasmin.

0:55SPEAKER_01:Hi, thank you so much for having me on.

0:57SPEAKER_00:Tell me a little bit about your journey and where you where you're at in your career at the moment.

1:02SPEAKER_01:Yeah, so um I am an anaesthetics CT4 trainee. Um I am currently working in Yorkshire and Humber. And last year I took an Oopier, where I've always had an interest in AI, but I didn't have the opportunity to really look into it. Um whereas my Oopie year gave me that opportunity to do an AI clinical fellowship at the Improvement Academy. And I enjoyed it so much that I've continued on with that. So I've now split my work between clinical and the AI work uh at the improvement academy.

1:35SPEAKER_00:Fantastic. And you enjoying that?

1:37SPEAKER_01:I'm I'm loving it.

1:40SPEAKER_00:Wow. Um so in terms of learning about AI, I want to be provocative. Will AI replace doctors?

1:48SPEAKER_01:So that is very often how it's phrased. And I think there is a bit of a paradigm, predominantly driven by AI vendors, to almost replace clinicians. So if you look at a lot of the research in the AI healthcare space, it's always talking about how we can compare AI tools to how doctors perform. And I think fundamentally, that's actually the wrong perspective to look at it with.

2:12SPEAKER_00:Okay.

2:13SPEAKER_01:You know, these AI systems are not there to replace us, much like when we replaced coffee in World War II, as we ran out of stocks, it was typically called Ursatz coffee. And I think of AI systems that are designed to replace doctors as Ursatz clinicians. Um, so yeah, I think fundamentally that's the wrong way to look at it. And how we really need to look at it is from a systems-based perspective, to see how AI can work with doctors to create a system that performs what each individual in that system can do.

2:46SPEAKER_00:Fantastic. Okay. So if it's not going to replace us, actually, what is AI? You know, we sort of we throw all these words out. What does it mean to maze a clinician?

2:59SPEAKER_01:Yeah, so AI is a term that's very often bandied about, and I think sometimes we forget what it actually means. So AI stands for artificial intelligence. Um, and really we have to pinpoint exactly what we mean by intelligence. The typical definition that's been given is that AI is essentially tools that can mimic human intelligence, but ultimately when you boil it down, it it just finds patterns between data sets. So that's fundamentally what we mean by artificial intelligence, is statistical tools that can find patterns in data sets.

3:34SPEAKER_00:And so how can how is it being used in clinical practice? Is it being used freely, safely, wisely? What's what's your view?

3:46SPEAKER_01:Uh I wouldn't say it's it's not commonplace at the moment. It's starting to make its way in. I think we're already seeing it go into other fields a lot more quickly. Um, and I do think this highlights perhaps a cultural difference between technologists and healthcare professionals. So, as healthcare professionals, we are evidence-based practitioners. There is a 17-year gap between research and implementation. And it's a 17 on average, absolutely.

4:17SPEAKER_01:Um, and it's a culture steeped in hierarchy. So, you know, we understandably so, because we want people who have the best experience to look after our patients. And technology is very different, it tends to be disruptive, um, almost a move fast and break things if we're gonna pull out that quote directly. Um, and perhaps compared to healthcare, definitely a lack of accountability, not to be very cynical about it. Um, so it's how can we bridge the gap between the two?

4:47SPEAKER_01:And I think that's really the crux of the challenge here as to why we've not seen it implemented much. Um, but there is tools coming in, and most of what we've seen so far is in the administrative space.

5:00SPEAKER_00:Okay, tell me a bit more about that.

5:03SPEAKER_01:So the obvious example would be Copilot. Um, so it's been shown to save a lot of time. So it can take meeting minutes, it can help with documentation, um, providing summaries for large documents and to pull out really key bits of information. Um, so it's quite easy to see how it can help with efficiency gains there. Um, but there are some tools that I think we have to be careful with that seem administrative on the surface, but actually they really aren't.

5:32SPEAKER_01:They're more medical devices. And a good example of that would be ambient voice technology.

5:38SPEAKER_00:Okay, so you've mentioned two words, ambient voice technology and device. Just explain that a bit more.

5:46SPEAKER_01:Yeah, so uh ambient voice technology, a good example of that would be a tool like Heidi. So these are tools quite often used in GP practices. So they are coming into primary care. It listens in on consults, um, and then it can it provides a summary of what that consultation was. So the obvious benefit there is you know, you can focus on the patient. That's what we're trained to do. Um and you can, you know, rather than sitting down and trying to put all those notes on your computer, you can actually listen to the patient and look at them.

6:20SPEAKER_01:Um, so patients tend to really enjoy that, clinicians enjoy that. Um, but you know, the main benefit is usually touted as a time-saving benefit. When you really look at the data, though, it's not as obvious as it initially appears. Um, because you still have to read that note that AI has generated, make sure it's not made any mistakes and it's not missed anything critical of?

6:46SPEAKER_00:I I heard, I saw a quote, I don't know if it's true, but you know, well, you read there was IKEA have replaced 6,500 jobs with AI, but they did not lose the staff. They actually give them better jobs and they actually enhance. And and does does the ambient technology maybe take one of the most painful bits of being a clinician, the documentation, the letter writing, actually give back some to the patients? I know it hasn't really reduced time, but do you think it people are getting better consults?

7:22SPEAKER_01:I think that's the real benefit here. And uh, you know, when we think about things like KPIs or, you know, we want quantitative data, but the quantitative data doesn't really show the full picture. Actually, it's qualitative data that gives the stories and it gives the right questions to ask. So I think what you're showing me is that you're thinking about the broader picture of like, actually, what is it that clinicians really want? And they want to look after their patients.

7:47SPEAKER_01:And I the real benefit is that reduction in burnout. So burnout is not just being very busy, sometimes it's not doing the job that you want to do. And tools like this can help help with that, absolutely.

8:00SPEAKER_00:I I heard a clinician who introduced it in New Zealand um quite freely. Um, but obviously, these some of these tools are properly validated and secure to use. And he introduced it to a psychiatrist, the psychiatrist said, I think you've just extended my career by 10 years because I just hate the admin, and it's a bit where it's actually a feel of God's whether it actually materializes, but it's changed the psyche, the view, and it's got a bit of this bring back.

8:31SPEAKER_00:So ambient technology is, I suppose, the bit that everybody started to be aware of, and there's some trusts that are really launched, and some were like, What is that? What's ambient? I don't even know what it is, and and there's still like some trusts are still on paper. I remember in my trust, um, I was the uh director of medical education, and we introduced electronic patient records. What for me, one of the best things. I know people listen might be called my god APR, but actually, we we got APR when COVID hit, so we had access to all the records and everything.

9:08SPEAKER_00:And um without the bringing of that technology, but going back to my role, there was a business case put in which said we would lose 10 FY1s to counterbalance the business case cost of paying for um the EPR system, and then six months down the line, somebody said, Simon, where are we gonna use these 10 posts? And I was just like, No, you really don't get this, you know. And people are terrified, particularly in admin, I'm gonna use my job, but there's so much work to do there.

9:44SPEAKER_00:Should we be embraced? So we've let's stick with the admin side. So you talked about how we can use, you know, and obviously we've we've all got a core pilot in HS. You know, the the government have invested in core pilot and they're building safe enterprise spaces. Um and then we can talk maybe about I think what you talk about, palot palotitis, uh, all the different things that don't manage, we'll spend a bit of time there. But if we go back to um admin or generative AI, somebody listening who's maybe a bit scared or hasn't really dabbled, what what can they do?

10:24SPEAKER_01:So how they can use it in their practice.

10:29SPEAKER_00:You know, I've got my inbox full, I've got huge documents to read, I've got reports to write, I've I've just got so much to do, and it has to come out of this box for those two those one finger at a time typing.

10:45SPEAKER_01:So I think the most obvious benefit from it is that kind of summarization. Um so if you've got huge documents that you need to look through and you just want an overview, these tools, especially these large language model tools, can be fantastic at summarizing information. So when I say summarizing, I think that's the real key here is that's different from judgment. And it's a question between attention and intelligence. So it has exceptional ability at taking huge data sets and summarizing it for you.

11:17SPEAKER_01:So, yeah, summarizing big documents. Um, these like Copilot, for example, you can listen in on your meetings. So, similar to what I was talking about with ambient voice, it can transcribe meetings for you. So I had that experience when I was working on a recent uh on some interviews last year where I recognized, oh my goodness, how challenging it is to transcribe interviews. Oh my god.

11:45SPEAKER_00:It's just a pay typist to man listen hours and hours.

11:49SPEAKER_01:Absolutely. And you know, I am I have to say, I'm a very fast typist, and it took me three hours to do a 45-minute interview. Whereas when I used Copilot, it was, I would say, about an hour, which was still painful, but you can see how much of a time saving it can it can give you. And because of the fact that it's intelligent, or you know, it's using AI, it can pick out key bits of information. So copilot is just one example, but there's lots of tools that can help you to use AI as an assistant.

12:22SPEAKER_01:So it's almost like a personal assistant that you can use.

12:25SPEAKER_00:What's what's your favorite one?

12:27SPEAKER_01:The one I like to use. So personally, I love using Claude. Um, so that's that's the tool I really, really like because it's great at what it does. So it is one of the best large language models out there, if not the best. Um and it also uh it it minimizes the amount of sycophancy some of these tools can have. So a lot of the tools might, you know, I like the fact that it can be a good thing.

12:52SPEAKER_00:Hey, yes, I mean you're great.

12:55SPEAKER_01:Exactly. So it it challenges me sometimes or makes me, you know, if I provide it with a piece of work I've I've done, it will find ways to challenge what I've written. And I I quite like that.

13:07SPEAKER_00:So one of the key things I I I am a big user of AI and I've learned a lot, and you and I, you we've we've chatted about this um at different times. And we talk about it's what you ask AI, isn't it? It's how you've got to train, you've got to train the system to know you. I'm a bit like you, I love Claude, and Claude knows me well. I I I had a divorce from Chat GPT, and I had to sort of migrate my personality and likes and voice style into Claude.

13:38SPEAKER_00:How do you have a healthy relationship with your Claude or your co-pilot? How do you get understand each other? What's your top tips?

13:47SPEAKER_01:That's that's a good question. Uh I I just recognize what it's good at and what it tends to be less good at. Um, so like I said, I don't like to use these tools and rely on them to perform judgment calls.

14:00SPEAKER_00:And so your judgment's yours.

14:03SPEAKER_01:The judgment's always going to be mine. And I think there is a corollary here with clinical AI. So with these ambient voice tools, you know, there's a big talk about how they're actually just summarizing the consult. But really, what you decide to include in a consult note is a judgment call, right?

14:22SPEAKER_00:So like saying how you how you bring that consultation to life and what your narrative and your thoughts.

14:28SPEAKER_01:Absolutely. So when you ask, let's say, a tool like Heidi to shorten its clinical summary, it's taking out bits of information and including other bits of information. And we all know as a medical student and as an F1, you write reams of notes. Um and as you become a consultant over time, you start shortening the notes and you just include those really key tidbits of information. So, really, these tools are summarizing, but they're actually performing judgment calls.

14:57SPEAKER_01:And I personally don't like using it for that reason.

15:01SPEAKER_00:So you've got to train it. And one so I I've trained a lot of my AI software to recognize um our teaching voice um and narrative and stuff. And I've included documents and copy, and it keeps learning. And you've got to keep, I think you've got to have a conversation. Um, and I think you've got to be really clear, and I think you and I talked about this. I've been looking at the um co-pilot studio, so there's some really fantastic learning videos, and they have the GCSE format, which is what is the goal, what you're trying to achieve, what is the context?

15:43SPEAKER_00:So, you know, I the goal is I want a detailed summary of all of this feedback. The the context is I am a medical education center manager and I'm looking after um training program. The source is giving them the source file of the data that take the information from here, and this is my expectations. I expect this to be summarized in no more than 150 words with clear bullet points, which I can share with the team written in a neutral non-judgmental tone.

16:20SPEAKER_00:Now that sounds like quite a waffly prompt, but what what I've learned to do is build projects and have specific projects where we train it in the background with those prompts. Now I haven't got onto skills. I don't know if you've seen if you've started with skills yet in Claude, where you can actually convert. So what it does, it takes Holland the skills and see how they're working out. You can actually now save them as skills. Have you got that for yet?

16:49SPEAKER_01:I've not I've not used Claude for that reason yet, no.

16:52SPEAKER_00:No, I'm gonna have to get find somebody to um I think that's the next level. So yeah, I get his daily LinkedIn post of the cutting edge, and and I think what it highlights is the pace this moves, and you mentioned hierarchy earlier. Okay, who's the best in the space? The inverse hierarchy. Yeah, you know, my daughter's 22, she's used AI throughout her whole degree in maths as a learning partner and and stuff. So how are we going to given that it takes 17 years from research to implementation, how are we gonna catch up in the NHS?

17:38SPEAKER_00:You know, they've put digital as a real pillar of the future. What's your thoughts? What are you hearing?

17:46SPEAKER_01:For me, I think that highlights the fact that we need people who can bridge the gap between those two cultures. And I I think the people that can do that are not going to be the AI vendors or computer scientists, although they're very important. It's actually going to be clinicians. So the reason for that is you can perform, you can build these tools, but if they're solving problems that don't exist, what is the point of the tool? It's about the problem that you have.

18:14SPEAKER_01:So if you want to get a nail into a piece of wood, you want a hammer, you don't want a screwdriver. But very often what happens is that the AI vendors come with tools that are screwdrivers, and you bought this tool now, which was an expensive tool, it looks very fancy, but it really doesn't solve the problem that you have. So what you need is people on the ground who find these clinical problems, who might have a generalized understanding of what AI tools can do what, and then you can identify the right tool for the problem that you have.

18:47SPEAKER_01:That's what we need is from within, people that are empowered and have a basic understanding of these AI tools to recognize the problems and what might be able to solve them.

19:00SPEAKER_00:I totally agree. I I remember being one of the first few residents doing medical education to a significant level 20 odd years ago. And I think people with the split roles of being clinical, research-based, and AI-based are what's going to really help us. Um and we need to hold the hand of people in organizations who are very nervous and risk-averse. Come with that, we're very risk-averse to this. So, what what stuff's happening nationally? So you mentioned medical device and stuff, you know.

19:37SPEAKER_00:So, can I just as a clinician download an AI app and put pati start using for patient care? Is that a reasonable, safe use of it's a loaded question, isn't it?

19:51SPEAKER_01:Yeah, I I felt a little twitch in my sort of question. I was like, oh no. So no, I've done a workshop.

19:59SPEAKER_00:I've done a workshop, and somebody said, Oh, yeah, I put all my ECGs through Chat GPT. Yeah. I was like, really? I don't know. Where's the patient information going? Is that valid? Got help us when people listening. How do you do this safely? And how do we actually not just be scared and actually start getting some proper pilots at work?

20:19SPEAKER_01:Yeah. So first and foremost, if you're using um medical device AI, the reassuring part is that it tends to have um safeguards against you know using that patient data to train the model. So generally, medical AI is designed for medical use. Um so they're not trained on patient data, they've already been trained elsewhere, so you you can be assured that there are safeguards on confidentiality. Whereas if you're downloading these apps, so if I'm using Chat GPT or Claude, these open source models, there is a real confidentiality risk.

20:55SPEAKER_01:So you have to be very careful in what you put into those tools. Um, and there are ways that you know, you might think, oh, these are trained on vast quantities of data. Who can sift through it to find the patient data? You can. Um, and you know, one of the most important things we learn in medical school is to maintain patient confidentiality. So don't do that. Um, also, you have to recognize that these aren't like a lot of the general purpose tools are not designed with healthcare in mind.

21:25SPEAKER_01:Um so they use vast streams of data. This can include medical journals, but it can also include Reddit forums, which I'm sure you know is very much of variable quality. So you don't want to base your medical opinion or medical thoughts on a tool that's using Reddit sub forums to tell you what to do.

21:46SPEAKER_00:Okay. So, and can you give us an example of stuff that you've seen that is starting to work? Uh or and I know you've talked about there there's pilot fatigue.

22:00SPEAKER_01:Uh any ones that you see might be promising for the future, particularly around clinical reasoning and so um I'll give an example um of a company called Annalise, and I think they pro developed tools that are excellent, but you can see what I mean by the pro identifying the problem being the crux. So one of the tools that they've developed is a CT head algorithm. Um and a trust in Yorkshire was struggling with the sheer amount of CT heads that have been requested out of hours.

22:35SPEAKER_01:They didn't have enough radiology consultants or registrars on call. So they had to outsource the reporting to uh you know these overseas. Exactly.

22:45SPEAKER_00:Local companies.

22:46SPEAKER_01:Very expensive. So about 60 to 72 pounds per scan. And most of these scans, 80%, in line with the nice national guidance, come back as normal. And no matter how hard they tried to reduce the demand on these CT requests, they couldn't because it was who wants to be the doctor that misses a brain bleed? You might know this patient is probably going to be fine, but you don't want to be the one missing a brain bleed.

23:12SPEAKER_00:And you don't want to be criticized, you don't, you know, it's defensive medicine, it's defensive investigation, isn't it?

23:17SPEAKER_01:Absolutely. Um, so they found a solution rather than trying to reduce demand, how can we optimize the process? Um, so they used Analisa's CT head algorithm, or they have a plan to use it. Um, and these algorithms are fantastic in that if it says it doesn't have anything, you can be 99.999% sure sure that it's not going to have any findings whatsoever. So, in a way, you can delay reporting those scans till the next morning, where an in-house radiologist can report them.

23:52SPEAKER_01:Um, and then other scans that might have some less urgent findings can be reported less urgently by these locum uh companies, and therefore you're still saving money. And anything with an obvious urgent scan, you get reported straight away. And with something like that in place, you could save a lot of money over time. So we're talking in the realm of hundreds of thousands of pounds a year. Um, and you can use that money to then hire more radiologists who can then take up that job in the following morning to report their scans.

24:25SPEAKER_01:So I think that's a fantastic use case example. And to highlight the fact that it's not a problem with the AI vendor or the algorithm, Annalise also has an excellent chest x-ray algorithm. It's been implemented in many trusts, but a lot of the doctors say it doesn't really help them with their clinical reasoning very much. Yes, there's still some benefit, um, but it tends to highlight a lot of things that are probably normal. There's a lot of false positives.

24:51SPEAKER_00:Normal variants, yeah, yeah.

24:53SPEAKER_01:So, how much of an impact is is different.

24:57SPEAKER_00:But if you're being scanned in the middle of the night and people caught and you've got something that's really worrying, now hopefully your radiographers would pick up on it as well as they're processing it and flagging it, but ability to make sure that nothing gets missed. I I heard a resident do a project in Wales, and it was um linked to the chest x-rays, but it was more looking at follow-up. So it was actually how many suspicious x-rays prompted proper follow-up and that that closing the loop.

25:29SPEAKER_00:And when you put a human in, we drop things. Probably that they missed three cases that follow-up wasn't properly, you know, the report wasn't action and stuff. And we started using these in the loop, but keeping the human in the loop loop for the judgment of decisions, the balance, and it has to make patient safety better.

25:51SPEAKER_01:Absolutely. And it it boils down to that fundamental difference between attention and intelligence. So with attention, it AI tools tend to identify problems that might not exist, but it's very nuanced. And with intelligence, as human beings, we tend to focus on one thing and we might get distracted not recognizing there's another finding. So when you bring the two together, you know, people that uh AI tools that are more likely to have false positives and uh clinicians that might have false negatives, they can compensate for one another.

26:27SPEAKER_01:And it goes back to saying that it can create a system that outperforms what each individual can do.

26:33SPEAKER_00:So the benefit of both of them is amplified. That's brilliant. Well, thank you for your time. Um before I summarize, I've got one question for you. Um, I like to talk about marginal gains. Marginal gains are those small improvements you make or learn in the last year of your own career and learning. What is your one key thing you've learned and developed as an individual?

27:04SPEAKER_01:As an individual. I think it's the main thing I've developed over the last year or so has been persistence and it goes back to that marginal gain of I was very used to, I think, when I was 18, 19, 20, of thinking I can just work really hard at something and then see the benefits very quickly. And I think what I've learned over the last year is actually I need to do a little bit of effort every day, and then you won't see the benefits till later. But it it's it's been a big learning point.

27:40SPEAKER_00:Yeah, I love that. I love that. My my 1% module gain from our discussion today has been AI can't replace human side of judgment, which comes with experience, but it can free us up when we're distracted, and it can free up a lot of that admin burden so we can do more what we love, which is speaking to patients, to the team members. Yasmin, um, thank you for your time. Thank you for all the amazing workshops you run with us on AI. I think it's such an exciting space.

28:18SPEAKER_00:Thank you for being on Why Medics Matter, where we talk about stuff that's really relevant and important to you. So if you've enjoyed this, um please click on subscribe and follow us as we will look to bring more interesting topics that resonate with you. Take care. Thank you.