E38: What About the Coach? Mixed-Methods Study Assessing the Experience of Coaches in a Peer Surgical Coaching Program

0:03You are listening to The Operative Word, a podcast brought to you by the Journal of the American College of Surgeons. I'm Dr Lillian Erdahl, and throughout this series, Dr Tom Varghese and I will speak with recently published authors about the motivation behind their latest research and the clinical implications it has for the practicing surgeon. The opinions expressed in this podcast are those of the participants, and not necessarily that of the American College of Surgeons.

0:32Hello, and welcome back to The Operative Word. I am your co-host, Lillian Erdahl, and I am joined today by Dr Eilidh Gunn, who was kind enough to make things work from across the pond, to tell us about her research, “What about the coach?”, which is a mixed-methods study assessing the experience of coaches in a peer surgical coaching program. So welcome, Dr Gunn. I thank you so much for inviting me onto The Operative Word. It's a great privilege to be here.

1:02Well, we're pleased to have you here from the University of Edinburgh. And, also to hear that you completed your PhD. You know, partly, this research was during the time that you were completing a PhD and having a surgeon with a PhD to really delve into questions around how to teach surgeons. I think it's important to have somebody who kind of understands, the inside track of how it is to train in surgery, and give us feedback about how we might improve surgical training.

1:35I also, wanted to just highlight that this work was done, in the Sabermetrics lab run by Dr Yule. And I had to look up what Sabermetrics is. So surgical Sabermetrics, according to the website, is the digitized measurement of surgical performance. So, you know, it's interesting to really break down surgery and think about it from that perspective. Yeah, it's a great aim that’s true. Yeah, this is a great collaboration between the safer surgery lab, Ariadne Labs, at Harvard and the Surgical Sabermetrics lab at the University of Edinburgh, with Professor Yule having a history at Ariadne labs and the safer surgical group.

2:20before coming over to Scotland. And I was his PhD student. So surgical Sabermetrics really tries to utilize license and performance analysis and enhancements that have been learned through professional sports. And that can have, you know, there's many tears and I have many colleagues who started their own PhDs in far more kind of digital and technical fields of analysis than Than what we discuss here coaching and but really what that can mean is either looking at performance enhancement through behavioral interventions such as coaching, which is what we're going to discuss today.

2:55And also looking at, digitizing that performance and looking at metrics. So that could involve looking at all sorts of different types of data, whether that's video, whether that's from sensors placed on the surgeons body, and whether it's through teamwork analysis and combining psychological techniques to look at performance, such as, MGT environment and really is to try and help maximize surgeon performance as best can to help an individual achieve their best.

3:27Yeah, well, and and the benefits, to surgeons of getting feedback, I think are pretty well-established. But, I appreciate the, the mindful approach to figuring out the different types of feedback that a surgeon needs and how to get data about the performance of both the individual and the team to really use evidence-based feedback and, you know, mentoring and teaching and coaching kind of all, require us to have a good understanding of what the individual needs in order to improve.

3:57And so I really appreciate kind of understanding how do you help an individual improve and having guidance. And I think that that is going to come out a little bit as we hear more about the coaching program. So I thought we might start by just sort of talking about the SCOPE project. And, you know, your research looks at the coaches themselves, but I think our audience would like to hear probably a little bit about the coaching project, just to understand what the coaches were doing and, you know, understand their mindset.

4:28Thank you so much. So the SCOPE program was created back in 2018 by Dr Smink, and Dr Yule, and with, Dr Pradarelli who was a research fellow at the time, and was designed to create a structured framework to allow, you know, a attending level surgeons to coach themselves. So a peer level coaching program. And it was thought that this was very powerful because especially my understanding is in the United States, you know, attending surgeons practice autonomously.

5:01And so oftentimes they're running their own surveys for, you know, multiple decades and doing it, you know, and they're already experts in their field right? They’re already doing an incredible job. And they're, you know, competent. But I guess the question was more, about how do you continue to maintain that level of excellence, over the course of your career? And there was quite a well known article, that was published in The New Yorker by Atul Gawande that, asked about that, just described his experience of having a coach and how it enhanced his technical skills.

5:34And so then based off the back of that, there was a consideration about how traditional CME offers the opportunity to go to courses and conferences. But how could we integrate the concept of coaching done by either musicians or sports professionals within the operative environment? And how to make that accessible? And having that opportunity between peers seemed like an ideal chance, you know, to have two people within the same field and to have their own areas of expertise, a share and exchange practice, but using a structured framework.

6:07And that's really what SCOPE was, key components where, you know, you'd have your surgeons and they'd be in pairs. Allocation was open between the department, how that would come about. And then you'd have a allocated coach surgeon and a coachee surgeon. The description of the coach surgeon being the person that would observe and then provide feedback and do constructive analysis, with the coachee being the surgeon being observed and with the opportunity for them to switch roles.

6:33If that was useful, the SCOPE program being having a pre-meeting with your coach, you know what you want them to improve or what skills do you want to look at? Things that might be interesting beyond technical skills? Could be non-technical skills. So communication, leadership, teaching and training those kind of things which are hard to elicit. Even though you, instinctively as a doctor, In general, as a surgeon are practicing those skills, maybe tens if not 100 times in the operation.

7:03So you can discuss that and then you'd have your period of observation. Then you would do kind of a paired, analysis and constructive feedback of that performance. Following the observation by the coach surgeon. And usually outside of the operating theater at a later time to allow for like kind of decompress and not have, you know, and trying to get the patient discharged and seeing the next patient, you know, so you could have some cognitive distance from that.

7:29And then following that you can then action plans to decide, how am I, what am I going to do next? Or how are we going to do this next. And you can pick a case that would reflect those goals again and then continue that cycle. Sorry this is quite a long answer. No, that's a great that's a great explanation. And a lot of things that we put into practice in early training come to mind when you talk about that. You know, I think we're building on these principles that we use.

7:58But with someone who has already achieved competence and, you know, maybe is looking to achieve mastery or determine, you know, where it is that they need to continue to grow and improve. And for adult learners, to think about identifying their own learning goals, but with some, some help and assistance makes a lot of sense to me. And of course, you're, you know, all of your work towards your PhD and thinking about how surgeons learn and improve, you know, I think comes into play.

8:31I was also thinking about the, idea that learning doesn't stop when you complete residency or fellowship, when you go out on your own, as you mentioned in the United States we’re very much, sort of released to be independent surgeons after our training. And I think that there's been a lot of discussion around, you know, how do you find support during that transition time and how do you find ongoing support? Because this work is difficult and challenging and, simply because you understand how to do it and have the technical skills does not mean that you stop needing mentorship and support, or that you stop learning and growing.

9:11Yeah, I think I think that's very true. And I think that all training pathways are a little different, or slightly different in the UK. I think there is the same feeling about especially early, what we call early years consultants in the UK as well. That kind of feeling that is lifelong. And jave, say, you know, Dr Smink himself took part in the SCOPE program and is really a believer. And in that kind of lifelong learning and grow. But something we discuss in the paper, this concept of growth mindset, which has been identified also in other kind of coaching research as well amongst surgeons, the idea that you can kind of foster that kind of belief or that attitude to practice.

9:50Yeah. And and one of the things that you talked about is, you know, this is a big time investment for the coaches and for the hospitals. mean, you have a, you know, surgeon peer who's not, generating clinical work during that time because they're really focused on mentoring and training. And so I think having that that idea that it's important for our profession, to continue this and making that investment, I really appreciate from leaders like Dr Smink saying, you know, it is important for hospitals to set aside time for peers to help one another continue to grow and improve.

10:30So that the SCOPE program was, across multiple hospitals and, you know, over a period of two years is kind of the the time period that you were looking at and you identified, several potential coaches. Tell me about the process of kind of identifying the coaches and, and how you were going to learn about the coach experience. Yeah. So this built upon a work that had been done by Dr Pradarelli in his research time and Dr Trey Sinyard, you know was the MPH research fellow at Ariadne labs who is my co first author on this paper and really what they had learned is Dr Pradarelli had managed to expand it to these eight centers within the greater Boston area.

11:18And the initial feedback from these early iterations of the program had identified that, you know, maybe both, we initially or most people, including myself, when I started my own coaching research, thought that the coachees, you know, would be the people who would report that they benefited from taking part. Well, that's the hope, isn't it? You know, that's why this long coaching program. But I see that kind of some of the qualitative data that have been gathered by Dr Pradarelli suggest that it was more bidirectional and actually obviously, incredibly, they have a massive respect for Dr Caprice Greenberg, who is very much like global expert in coaching in surgery, and has published many wonderful papers and also made done a publication, that described this bi-directional feedback when they did their own qualitative analysis of research.

12:09So a lot of coaching work, and also one of the issues, as you've also described in your last question, was that some of the feedback was that the time afforded both financially and, you know, with regards to just physical time, it was quite a lot. And therefore, recruitment might be an ongoing challenge for the SCOPE program moving forward. And also the idea that there was a potential if the coaches did benefit, then it would help level the playing field, if you will.

12:37You know that there was mutual learning for both. And this was interesting to us in Edinburgh and why we chose to collaborate together was I was keen to try and introduce or a trial doing peer coaching in Edinburgh as part of my PhD project, with a potential to look at the SCOPE program. But it was again a question I got asked a lot was that, you know, we'll surely only half of the, half of the team is going to benefit from doing this. So it really went ahead and created some research questions, which we had a well shared research questions that both Trey and I had that we’d like to answer.

13:12So my understanding was that the coaches and pairing was done within the departments that took part in the program, and that it was done very much, it was a they were given the autonomy to be able to decide who is going to coach and who wasn't, though there is a SCOPE guide that was created bit try to encourage like, you know, swapping roles, and also seeing not always the senior surgeon being the coach and the junior surgeon being the coaches. They’re, sort of encouraged to like, you know, exchange roles or move against the more kind of didactic rate...sorry, you can’t see hand movements in a podcast, but the didactic kind of balance that you'd expect in senior and junior.

13:51Yeah. Well, and and again, I think, what I have learned as a teacher more than anything, a teacher and a mentor is how much I benefit, personally and how much I learn. So, so I love the idea of advancing that culture to say, you know, it turns out that, learning is not unidirectional. It's not top down. It doesn't follow hierarchies or anything like that. But, that actually you can have rich learning that flows, you know, bidirectional in this case because we're talking about a dyad, but, you know, sort of, up, up and down and, and around a team working together, it's not always one person teaching another by sort of role definition.

14:36That's extremely well put. And thank you. Yeah. It's you know, you can tell why I was excited to talk to you about this work. I really enjoyed reading it. And so once you kind of identified your research question and then you had to reach out to coaches and design a way to answer the questions. So tell me a little bit about your study design. Yeah. So basically we initially decided that, you know, to do this we would be best placed to try and, you know, reach out to those surgeons who had participated as coaches, in across the multiple centers, in the SCOPE program.

15:18And what we decided to do was, take a mixed methods research approach, which involves using quantitative and qualitative elements to kind of create, a kind of a detailed and rich snapshot of people's experiences and opinions. And it can be used quite a lot across health care. But we thought this is specific, really useful because we had, unanswered questions where there was a previous set normal or, you know, research that demonstrated what the coaches had experienced.

15:56So we wanted to explore this, and we felt that we wanted to do get in depth feedback of doing interviews because it was obviously Covid. It was done via zoom at that time. But we also wanted to try and see if we could get some, you know, metrics. And we used to just say and we created, a survey, questionnaire which was anonymized so that people could feel they could be honest. And we included in that that a Net Promoter score, which is a kind of a used within business and marketing.

16:25And you say on a score 1 to 10, you know, how much are you, like, how likely are you to recommend, you know, in this case the SCOPE program to a friend or colleague. And so we sent that out to all of the eligible participants who who had served as coaches. And we created the interview guide and also the survey guide that was with myself, Dr Sinyard, Dr Smink, and Dr Yule. And we also with regards to the methods got a insight and expertise from Dr Emily Cummins who is an extremely experienced mixed methods researcher in health care.

17:06And this really was really vital to us to like elevate things and also because it's not...the key is you obviously can gather your two types of your data, but the real, expertise that is required is how do you co-analyze those and like how you use one to reflect the other. And there's a lots of different, you know, methodologies as how to do that with the way in which you integrate those data together. And she was really instrumental in helping us do that with our data analysis and look at things with regards to the categories or concepts that we were able to identify following that process.

17:45I appreciate you kind of explaining the importance of having an expert. I do think that sometimes qualitative data is seen as something that's not gathered scientifically. You know, it's been hard. I think as, as we, bring these types of analyses into the surgical literature, particularly in the realm of education, to gain traction. And as we were discussing before we started recording, you know, the, the mixed methods and the qualitative analysis have a long history of study behind them.

18:19These are not really new methods, but, sometimes they're new to surgeons. Who have looked more at clinical science and sort of only, quantitative analysis. And I was wondering, in doing this, who conducted the interviews and how do you kind of structure so that when you're doing, semi-structured interviews, you ensure that you have consistency in capturing those data? So it was my coauthor, Dr Sinyard that, conducted all of the interviews. He is obviously, based in the Boston area, was able to conduct those, and we created an interview guide.

18:57He'd also had training and semi-structured interview techniques. And before undertaking the interviews. And I think that that he said that was particularly helpful. And having done my own, later on in my own PhD, I think it is really helpful to have these kind of signposts to come back to. Obviously, you don't want to be too rigid because you can get these really rich elements of experience that come through, and that's part of the benefit of the mixed methods, structure.

19:24You can really get in depth and, you know, insight into the experiences in that, you know, in that iteration of the coaching program from participants. But I think that that was really helpful. And it was also helpful when we created the guide to have input from multiple team members, you know, to get different perspectives. And I think also being mindful of time and just the person, the individual who's donating their time to the research and making sure that you don't rush things along but you don't, extend things to as well is quite important.

19:59From my own experience, which is obviously not the same as this study. Yeah. So we've talked a lot about the background and hopefully our listeners are still with us, but I think it's important to understand that the context and the approach. So tell me, what everybody really wants to hear about, which is the results. What did you find about the experience of the coaches? Okay. So overall, we didn't have any preconceived numbers. That was something I should have said.

20:25We didn't have like, when we did the analysis we didn't use, like, a set template or a structure. And that was because what we're doing was exploratory. For all we knew when we were analyzing, it could have come back that nobody had a good time, and then it wasn't worthwhile, which is obviously kind of was not was the null hypothesis. So when we looked and, what we did was we were able to create, these kind of conceptual categories, which is what we did with regards to the analysis technique that we used.

20:55But basically what we found were overall is that, you know, when we're looking at it, is that what we identified were...the coaches...the first was called affective attitude towards participation. So basically the coaches that were taking part were motivated. And were, you know, keen to engage with the coaching process. And they kind of at least stated either their own, you know, desire for their own practice enhancement or to contribute to the development of others.

21:24And also like reflection on maybe how their own career progressed or things that they would want to benefit from moving forward. And then when we actually went in, they said, okay, well, were are there any benefits or Dr Sinyard said, were there any benefits? You know, they described they actually based on observing their colleagues, how some of them had listed practice change. And this was also identified when we looked at their survey results as well.

21:53And also that they felt they had improved their teaching skills, but also a sense of professional fulfillment. You know, when you have that idea of helping someone whenever, you know, even myself very well, great. You know, for you, helping someone develop a skill, it can be rewarding to see that and then progressing the way that they want or learn. Yeah, yeah. I mean, I think that and those things were quite, you know, we thought that they were, were what we were hoping for that.

22:18And I think that really it was great to see that coming out of the interviews, but also coming out in the questionnaire as well, that people really felt that their performance had improved as well as that of the coaches. I love to see that observing their peers, led to improvements in their own practice. I can remember myself, you know, being in some surgeries where two surgeons were present two faculty surgeons especially, you know, complicated, procedures in, in highly specialized surgeries.

22:49And, it was amazing to see that both surgeons were typically focused on the same section of the operative field. You know, there's a lot of sort of shared knowledge and experience that comes across when they're together, working. And so, you know, I think the observation at that level, practice is very different than what, you know, a junior trainee or even a fellow sees when they watch another surgeon. Yeah. I think it's like, very much like, I don't know, I think, well, it's very much like the art of surgery.

23:22You know, that you're describing, like when it goes, you know, up to the next level of like, okay, someone does something. And I think that I have to agree. And it was great to hear people say, well, they thought about how someone had done, you know, either non-technical skill or had managed, or even, you know, logistics, that it was helpful to see how they'd done that. And I think that that does show it's those kind of moments and things that you identify, you know, they're very hard to capture, like, outside of doing some like coaching or giving or just even by giving someone the opportunity with a very simple set structure to get that observation opportunity as well, whether it's in video or whether it's in person.

24:07Yeah. And, you know, I think that that thinking about all the different ways that we have continuing learning, in our careers led you to talk about, CME, sort of comparing their experience with coaching to CME. What did you find? So, I mean, obviously we can you know, discuss about the fact that there is, you know, potential bias and the responding, you know, cohort of people who wanted to do interviews with us. But most people felt that the when compared to existing, CME activities that they, it was they preferred or it was beneficial comparatively.

24:45You know, we felt this might be because of the fact, like based on some of their interview content, that, you know, the acquisition of the knowledge was immediate or directly applicable, sorry, applicable to their, their own, operative environment. There's active participation, you know, as opposed to passive learning. You know, you have to be engaged to be a coach and you have to be engaged as well to be a coachee equally. And but then also when that was explored again, there were some upsides and downsides to people saying, well, if you made this official for CME, then it becomes your more like hurdle based or objective based.

25:23And is that a benefit or not? That's maybe for future research. But yeah, I think it just yeah, I think it's environmental especially, but also the kind of the focus of coaching itself might be making it more appealing to some people. And, and part of what you were looking for was, you know, is this a valuable use of the coach's time. So I think whether it gives you CME credits or not, having it be valuable learning time for the coach seems to support it.

25:53That that it's worth continuing these programs, which again I think is is part of the question or part of the driving, need to do this research to understand whether it's really worth taking so much time to do an activity like peer coaching. Well, it's reassuring that, you know, the majority of respondents did feel like that. You know, it was worthwhile use of time. You know, a 15 out of the 19 thought so, and the net promoter score also is reflective of that.

26:25But I think that this leads us very nicely because I'm aware of time, just on the fact that, you know, navigating the logistics of coaching was a one of the main findings in regards to barriers, if you will, to participating in the future. And, people often find that, you know, scheduling and time were a, a big challenge to actually taking part in the sessions. And some coaches did mention, you know, financial cost, but felt that the benefit outweighed that.

26:56But I think that, you know, it's something that definitely needs to be considered for future research as to how to overcome those barriers to like. As many people as possible, the opportunity to take part. Yeah. Well, and I want to be, careful, how I talk about the effects on surgical culture because it's from the perspective of the coaches. And, but I was, thinking also about the investment of the organization in that time and potentially the cost of it.

27:26And, and the coaches, talked about the flattening of the hierarchy and perhaps increasing the psychological safety for junior team members. And I wondered if you could dive a little bit more into how they thought this experience achieved that or, you know, led to that culture change. The I think this was, for me personally, a surprising finding in a way. Because we had obviously had discussions and there is some really there's a great paper that was written about, challenging surgical culture by Burns et al, which is part with Caprice Greenberg as well, looking at the wider effect of coaching programs on surgical culture, but basically what we found is that when we discussed it with them, they talked about the fact that, you know, everyone could benefit from a coach.

28:15And, you know, one of the participants said, you know, they're everyone is kind of put on an equal basis if they're taking part in a coaching program and it's, you know, you're able to give each other feedback. But also, I think that the other thing is that, you know, it was kind of other things that we discussed is also things that might be challenging that you are able discuss with your colleagues with regards to like logistics or shared experiences or, maybe more challenging aspects of the procedure that have been done.

28:49And then with that, you, this may be me just, you know, suggesting, like talking about, you know, how they've overcome, you know, operative or technical challenges and, and maybe allows for that kind of camaraderie that you might not have another opportunity to get necessarily in another way. And, you know, with that kind of structure, although I'm sure many people have their colleagues, maybe they have mentors within their departments that they can meet with and have, like casual, you know, coffee or watercooler kind of conversations in their own time.

29:22But I think that that's maybe what coaching allows for is that kind of dedicated time. To work with someone who has the same challenges or in the same environment that you are and the coaches have then been able to identify that as being constructive and even people whose experiences weren't ideal in the program still felt it was of value. As like, I think it was described as a master class learning experience. So that was great. When I read that in the transcript. And so the master class, I may be rambling a bit, so please forgive me.

29:58No, I think that there's the work is very rich. And again, I think that's what I appreciate about particularly the qualitative, portion of the mixed methods. And if you'll bear with me, I'm going to read a quote from that that was in the paper. One of the surgeons said, “We have a good culture in our group, but coaching adds to that. That adds to the fact that I'm there for you and you're there for me. And I think it is a big thing that goes beyond the nuts and bolts of throwing a suture better or something like that.

30:29It's just adding to the culture of support amongst the faculty, which I think is really key.” And, and I think if we can leave our audience with anything, it's the idea that nobody should do this job alone, regardless of how much mastery they have of the technical and performance aspects of surgery. And so that that, more than anything, was the message that came out of this for me. We're all lifelong learners. And, the benefits of having peers who can help you continue to grow and learn really came through in this paper.

31:04I think that, that's a great summary. And it's, I'm really grateful that that’s the message that we were able to convey as a research team, when we were putting together, this work. Well, thank you, Dr Gunn. I hope that our listeners will consider coaching. That was again a message from the paper. Many of the coaches wished that they had had that experience early in their career. And, so as we continue to look at the SCOPE project or other opportunities for coaching, I, you know, I hope that people will consider it.

31:36Yeah, it's great. And, you know, you can still sign up, you know, get access to the materials if anyone is interested. And I can forward a link on if people would like to learn more about the SCOPE program. And I’d also like to thank Dr Erdahl and JACS for having me today and for accepting our research paper. Thank you for listening to the Journal of the American College of Surgeons Operative Word Podcast. If you enjoyed today's episode, spread the word on social media by using the hashtag #JACSOperativeWord.

32:09Subscribe to The Operative Word wherever podcasts are available, or listen on the American College of Surgeons website at facs.org/podcast.