Bayesian Borrowing in Phase 3 Trials

0:00Judith: Welcome to Berry's In the Interim podcast, where we explore the cutting edge of innovative clinical trial design for the pharmaceutical and medical industries, and so much more. Let's dive in. Scott Berry: All right. Welcome everybody. Back to In the interim, I am your host, Scott Berry, and I'm joined today by my co-host, uh, Kurt Veley. Dr. Kurt Veley is here, and, uh, Kurt needs no introduction, uh, as he is a, a regular on the show. Uh, we have a interesting topic and, and the topic I'll, I'll, I'll introduce the topic a little bit by telling you a true story.

0:41Uh, and Kurt is related to this story. For those that didn't know, Kurt and I were fellow graduate students at Carnegie Mellon. And, um, when I was a 23-year-old, first year graduate student at Carnegie Mellon, I, uh, called my dad. And of course, um, the, the aspect I called him is I called him and informed them. Him that my wife had taken a pregnancy test and it was positive. And uh, I don't know if many of you in the world have similar interactions with your father in when such a thing happens, but Kert Viele: I think, the answer is no, Scott.

1:26Scott Berry: I think the answer's no. Uh, the, the question, the first thing he said after I said, hi, I said that, uh, by the way, I did say, uh, we took the pregnancy test and the pregnancy test says it's 99% accurate. And I tell him this, and his first question to me was, what was your prior probability? Honestly, that's, that's what he asked me. And, um, uh, I, I, my response was, and uh, and, and Kurt knows this well, we, we were, I was a married graduate student making a stipend.

2:04I said the test was $15. Dad. The prob, the prior probability was pretty high, and his next comment was, oh, congratulations. That's fantastic. Uh, and he was excited, uh, for, for this whole thing. So they, the, the point to that is if our prior probability is low. The 99% means something different about the the, at the end of the day, the question is, is Tammy, is my wife pregnant or not? Um, in the, in that, based on this experiment where the prior probability makes a huge difference, and many of you have probably seen the rare disease examples where if you have a low prior probability of something and you take a diagnostic test and it says positive, you're still probably unlikely to have the disease.

2:57Uh, so in this circumstance, if, if, if I had a 1% chance that she was pregnant and the test was 99% effective. Meaning if she's pregnant, 99% chance it says yes, if not 99% chance it says no. I think they're actually more accurate than that. But, um, if I had a 1% chance, I still, it's only 50 50. She's pregnant in that circumstance where if I started with a 10% chance, it was over 90%. She's positive. The point to that is the prior probability of that changes my interpretation of the result of the experiment.

3:41So the topic for today is using Bayesian borrowing. And that's a case of Bayesian borrowing of what my dad did faced with the result of a clinical trial. Uh, and the diagnostic test is a clinical trial. A clinical trial is the same thing based with that information. The prior probability was a huge part to his interpretation of, uh, of the result. There are cases now, and the Bayesian guidance is out where we use Bayesian. Probabilities going into the trial to interpret the result of, of the clinical trial working with the FDA.

4:24And today's topic is doing that. What does it look like? What are the characteristics? What's the behavior? Have we done this before? So that's the topic for today, Kurt. Kert Viele: All right. Sounds like a good topic and it sounds like we're talking about, um. So we, we've done this in a couple contexts, and it sounds like you're gonna talk about the second one. We've done this in context where we're only borrowing on, you know, a clinical trial.

4:54There are two arms, there's a control, there's novel treatments. We've done situations where we've borrowed only on the control arm. So you're comparing to an existing drug. There's lots of data on the existing drug, and you're trying to bring that in. That's one problem you might solve. Um. It sounds like today we're gonna talk about the, um, what we'll actually consider the more complex version. Kind of a newer version of we're borrowing on both arms where we have some idea about what the treatment effect might be based on history.

5:30And now the question is, what does our current trial have to do in order to. You know, this is a journey from we know nothing in order to, we know the drug works if we're partially there based on what we've already seen. Um, what else do we need to know? How much more evidence do we need to get over that bar? And I know that we've seen this in the context of. You know, completely external trials. We've seen it in the context of, you know, people run one trial and get p equal 0.06.

6:04And now the question is what do we do? Um, so where, where do you wanna head from here, Scott? Scott Berry: Yeah. Yeah. Uh, good. So first of all, just to, uh, and, and by the way, this is a, this is a shout out to anybody who, uh, listens, watches, uh, in the interim, if you have questions. Throw 'em our way. We're, we're, we're looking for interesting topics. This was something that somebody, uh, contacted me and asked us, have we ever gone to FDA using Bayesian, borrowing?

6:39And it was a little bit like, whoa. Yeah, this is kind of what we do. Uh, and I was a little, almost surprised by the question, but then it, it made a ton of sense that, that now with the Bayesian guidance out and they were interested in the, this interaction and what does it look like? The, the other part to this is the Bayesian guidance make, ref makes reference to Bayesian trials where you're controlling type one error or you're not controlling type one error.

7:07Her and I was asked by somebody else, a brilliant researcher, actually, uh, Dr. Bijoy Manan about, uh, he's a stroke neurologist in Calgary. Uh, what's the difference? Uh, that sort of thing. So I think we can tackle all of those things today. So maybe we should set it up with a case where this may make sense or cases where we've done this before. Uh, just to, just to give people an example of, of it. Now first maybe we should set up what is done in a Frequentist standard clinical trial is you calculate for that trial the probability of a type one error.

7:49Um, uh, and it's only data in that trial that goes into that calculation. And we say, if the probability of achieving the data in that trial. Given the null is less than 2.5%, we're gonna call it a successful trial. By the way, in the circumstance of the pregnancy test, the assuming she's not pregnant, the probability of it saying pregnant is 1% statistically significant. In that case now, that that might be how we interpret it. Now, here's a circumstance and we're, it's not uncommon that a company, a sponsor, has run a trial and the trial is, uh, not convincing for approval, but has some positivity to it.

8:43Maybe it's a p value of 0.07. Uh, some cases it's actually been a p value of 0.025, but yet FDA is not convinced. Regulators are not convinced. So let's take a case where there's positive data and a a p value of point, uh, a one-sided 0.04, Kert Viele: And I, I think Scott, Scott, to this, we should add another situation this happens is somebody runs a phase two. And gets really positive results, but it's not quite there. And their question is, Hey, haven't we already fulfilled at least part of the bar for approval?

9:22Same kind of idea. We're partway there, but we're not fully there. Scott Berry: So company A has run a trial that's not statistically significant, but it's close, and maybe they even went to FDA and say, will you approve us? Now, I've never been at the FDA, but I can imagine them thinking hard about it, actually thinking the drug probably works, but it hasn't met the thing. We can't approve it yet. So company A is in that situation. Do you tell them you have to go and run a trial ignoring all of the things we know from that first trial and you have to jump a 0.025 hurdle and we, we ignore it.

10:05Company B has no data and they have a treatment for the same disease. They wanna run the same trial. They're asked to run a trial with a 2.5% type one error. Um, are, are the companies asked to do exactly the same thing? Because a ton of scientific sense that that first company a can utilize the information from that trial and maybe they don't need as large of a trial to demonstrate that the drug works. So that's a circumstance that we're talking about today where you have prior probability going into a new trial that the treatment's effective and you want to use that to ana analyze the new trial.

10:51Uh, a and it could be that first trial is a phase two trial. It could be that first trial, uh, you know, was a biomarker trial that just happened to show clinical benefit there. There's a lot of ways in which that information comes about. Another circumstance and we did a complex innovative design trial where this was a, a public presentation of a circumstance where a company had demonstrated benefit in two types of epilepsy, and they wanted to investigate a third.

11:27There's a commonality in the diseases. There's a commonality in the endpoint, the mechanism of the disease. You're taking a treatment that's been shown effective in a related but slightly different disease, and IT benefits, clearly benefits. These other two, and now you're going into a third, should the standard be. A trial in that subtype of disease 0.025 type one error, or should there be some sense of borrowing from the others and, and for example, in this case, it was showing about a 30% reduction in seizures in both of the other types.

12:05If all of a sudden relatively early in this third type you're seeing a 30% reduction in seizures, you're pretty, pretty sure the drug works because of that prior probability. Yes, these are cases. The third case, uh, talks a lot about it is you might have a conclusive proof of benefit in an adult population. There are rare circumstances where pediatrics have the same disease. It's very rare. It's hard to run a trial. Do we just ignore them? You might run a trial and borrow from the adult in some way.

12:38So there's a lot of circumstances that you go into a phase three confirmatory trial with prior information about the efficacy of the treatment that's in that trial. Kert Viele: And I, you know, one thing to add to this, you know, these kind of situations, they've come up in the past and there are often ways that regulators have made the bar a little bit easier given this prior information. And that may be, you know, one trial versus two. It may be, hey.

13:10We'll change something about the, you know, what you have to do in your phase three trial. What we're really talking about today is trying to formalize this process. So the question is, if you wanna apply all the statistical rigor that we usually apply to a trial. You know, ordinarily this is, you gotta do 0.025 all the time. Is there a middle ground where we could basically formalize and say, here is how we're gonna analyze it. We're still gonna keep all the statistical rigor, but we're gonna recognize that you do have this, your partway there.

13:46Scott Berry: Okay, so suppose we have this prior probability. And we have a prior distribution about a treatment. And, um, we're going in and we're running a new experiment now, and we're gonna collect data in the new experiment. And the Bayesian approach is, we're gonna use this prior in some way, and we'll talk about different priors in that dynamic discounting and the, those aspects of it. But we're gonna use that prior in some way with the new data.

14:18Now explain. Why that can inflate type one error and what type one error means and why. It's a little bit disconnected in the, the way we're going about doing this. Kert Viele: So what's gonna happen here is suppose that, you know, my, my, whatever my prior evidence says is I think I have a 10% effect, 10% beneficial effect on something when I come in and compute type one error. What I am saying is, let's assume that the control and treatment arms are exactly equal to each other.

14:55Tell me Scott Berry: In in the new experiment, Kert Viele: In the new experiment. Yeah. Yep. So we got our historical evidence on one side. We got our current trial on the other, which I'll refer to it if there's no effect in the current trial. Now, this is something we don't think is true. We have prior evidence, so we at least are less likely to think it's true, is the more precise way to say it. But if it is true, what's gonna happen is that prior evidence, it's gonna bias things upward.

15:23It's gonna say, Hey, if I didn't see an effect in the current trial, I'm gonna average this in some way with the beneficial effect from the past. I'm gonna. Pull things up a little bit, I'm gonna basically say, you know, maybe I have a two or 3% effect. The net result of that is that little bit of bias makes you more likely to claim efficacy if, if, if, if all of this is, if you don't have an effect in the current trial. So the key issue is. What is the value of that assumption?

15:57And this is where we get into the guidance and you know, do we control type one error? Do we not? We will often do that calculation and say, you know, if this is true, here is the type one error. The the results of the type one error for various kinds of designs, and they're gonna be hired two point half percent because of this bias from the past. The argument that you make is this assumption that I'm worried about. The null hypothesis is true. I've already partially disproven it.

16:32So the value of that type one error calculation is less in that case. And that's where we get into, I know where you want to go. Discussions of with and without type one error. Scott Berry: Hmm. So if, uh, and. If you are gonna use that prior distribution, uh, to it, and, and what might that look like is we're gonna use the prior, use the new data and say we need a 97.5% probability that the treatment's beneficial after the combination of the two, that that's gonna be successful.

17:06We may go to FDA, and that's the criterion. Now, if we weren't using prior distributions. That probably controls type one error at 2.5% for using a flat prior that that Bayesian analysis is similar to a frequentist analysis. Uh, and so the type one error would be controlled for the new experiment, and that would satisfy in the guidance where it says you are controlling type one error. Because really all we're using is the new experiment. Uh, because we're not bringing any prior, once we bring that prior and we combine them together now, uh, as, as Kurt says, to get 97.5% with the combination, the new trial doesn't have to be 97.5 by itself.

17:56That might be 93% is enough. Combined with the prior to say overall combined, we're 97.5. If you evaluate only the new experiment, it might have a 7% type one error in that circumstance, Kert Viele: Yep. Scott Berry: and that's where the FDA might agree. Okay? The data you're using is reliable, it's relevant, it's appropriate. We agree that you can use that for approval as a combination. Then they're approving within only this modular new experiment, the type one errors above the traditional 2.5 and maybe it's 7% in that circumstance.

18:44So that's kind of the difference between a Bayesian analysis that controls type one error for the new modular experiment and one that doesn't, Kert Viele: Yeah, and I, I think we're. Scott Berry: about a lot. Yep. Kert Viele: we're, we're talking about the combination. I, I always tend to view this if, if I were doing things from scratch and I'm gonna do two trials, um, is that combination good enough? And the question is, you know, I'm basically designing my new experiment when the original, the historical data is historical.

19:17It's in the past so I can condition on it. Scott Berry: So let's, let's, um, and, and we'll, we'll leave as a separate topic. Where we're borrowing only on the control arm. Yes, that can inflate type one error. It's a different kind of thing about the new temporal differences in the new trial. And so we're, we're, we're talking about the circumstance where we're going in with an informative prior about the treatment effect, and you're having discussions with FDA about that.

19:49Kert Viele: So I'm gonna ask you, I'm gonna derail this slightly, and I haven't told you I'm about to ask this, but what about situations, let's Alzheimer's sepsis, where nothing has worked in the past that potentially should give us a negative prior? How should people view that? Scott Berry: Uh, I think it's appropriate and I think it's, it's been an unspoken thing probably at the agency. And there was a time not long ago where there were 25 straight failed phase three trials.

20:24Kert Viele: Which is unlucky, by the way. They should have won one by Scott Berry: Somebody should have. Yes. Uh, and that, by the way, the 26 trial, the FDA may be weary of that, that if all of a sudden the 26 trials shows P value of 0.02, maybe they're the lucky one. Uh, and you can imagine a, a skeptical prior, which is talked about in the guidance, in that circumstance. Uh, and maybe they even ask for a higher threshold of that, uh, circumstance.

20:53So that's kind of a different prior going in where historically the scenario is, is, is, has been nothing works, uh, uh, in that scenario. Kert Viele: Yep. Scott Berry: so you carry the burden of the disease with you as you, as you go into the agency, which I think is quite, quite reasonable, Kert Viele: And you don't wanna discourage research in these areas and so on. It's just a really hard problem. I wasn't expecting an answer for it, but we, we should be honest about it exists in the other direction too.

21:24Scott Berry: yep. Okay. So what this may look like, um, in the interaction. So we're in a circumstance, and by the way, we get the challenges of this. So, for example, we might get sponsors that come to us that ran a phase 3 trial, and it was a P value of 0.3. Barely observed positive differences. They find a subset of 30% of the population that, wow, it's, it's nearly significant. And they want to take that subset and ignore everything else and go to the FDA and borrow from that subset in a new trial, and it gives them a jumpstart for approval.

22:13Okay. And now they've cherry picked from the previous trial, and by the way, they can explain why. Of course, that's the relevant subgroup and all of that, and they want to go to FDA and borrow from that. And there are circumstances where you and I, they come to us. We'd say if we were at FDA, we wouldn't accept that. Because we believe this is largely about multiplicity, and now you can't pull that out and create a prior that's only based on that.

22:45If you wanna use a prior, based on the entire trial, that might be something to discuss and, and have that. But to self-select data and only use that data going to the FDA, you're probably gonna get a no. And by the way, historically that's happened to companies and they come back and say, FDA doesn't do Bayes Kert Viele: Yeah, that's always frustrating. Scott Berry: right, right. And it's had nothing to do with Bayes It's, you're, you're misusing it in ways that we, as Bayesians would say, no, we, we, I would not accept that circumstance.

23:23So there's a huge part of the data. You're going into the FDA. What are you using? Uh, has it been, you know, self-selected? Is it all of the data out there? Is it relevant for the Nutra? So there's a huge part of this, of the science. Science is hard and, and you go in. So if we go in with a circumstance where we believe we're bringing to them a, a relevant prior. Based on the previous information that becomes part of the discussion with the FDA.

23:54Is this reasonable, Kert Viele: Well, and it becomes a, it becomes a major part of the discussion with the FDA. I think we spend as much time on data as we do on the methodology at this point. It certainly is the main reason. As you said, FDA says no. Um, a lot of times we're not told where this subset came from. It's, Hey, I want to do a trial in this population, and this is the data I have on this population. And, you know, at some level that's a red flag at this point that I need to ask the question.

24:25You know, you said this is the data we have on this population. What's the data? What's the, what's the. What's the world actually look like so that I can assess whether this makes, whether this, you know, is it cherry picked? How much is it cherry picked? All of these things come into play. Scott Berry: Hmm. Yeah. The, the dog that didn't bark. What are you not telling us? Uh, because we get told the rosy part of it. And what are you not telling us and what, what data are we not using?

24:53Um. And so for example, in the case of the complex innovative design program where we've got these two other indications and they don't have a third indication that failed, for example, if they did, you'd want to include that in a hierarchical model. Or there's a, a large phase three and we're using the entire estimate that comes from that to borrow for the new trial. And we go to the agency and we're presenting them how we're doing the borrow. So I'll give you one example of a case that the device was approved.

25:25The Watchman device is a left atrial appendage device that's been approved by CDRH, uh, at FDA. And they or originally ran a trial that was inconclusive and the name of that trial, the original trial, uh, I'm gonna get this wrong. Um, the original trial was. Uh, protect af. And then they ran a second confirmatory trial called Prevail. And the Prevail trial borrowed where we discounted the first trial by 50% in a static way. We'll just discount it 50%.

26:06And by the way, this, this went through FDA, uh, went through Bram Zuckerman's group. Uh, Bram is. Is a brilliant scientist and understands what it means to borrow the implications for the new trial. More than once, these types of things have gone through the cardiovascular device division at the FDA absolutely brilliant scientist who was involved in this particular case in, in that borrowing. So we go in and we propose that level of borrowing and show.

26:42What happens in the new trial? The most important thing to show, and I had a I, I had a client discussion yesterday with the same thing is what example trials of the new trial, when is it successful with the borrowing? And then how does that compare to if we didn't borrow? So for example, if we're, we, we've got a. Uh, time to event analysis where we're borrowing data. If we don't borrow, we need a hazard ratio of 0.7 or better to win with borrowing.

27:20We show that if it's between 0.7 and 0.77 with the borrowing from the historical data that showed 0.7, those are still successful. Kert Viele: The field goal. Scott Berry: That, yes, the Bayesian field goal. And so you show them the, I think the most of we, sometimes we, we revert to operating characteristics. If the truth is 0.7, our trial's 93% powered. We, we, we would tell the FDA or we would say, if the device doesn't work, the new trial has an 8% type one error.

27:55And we, you, we talked about what that means is that additional trials are being demonstrated with the final analysis at the end being Bayesian as a combination of the trials being successful. But I, I think that's hard for the clinicians at the FDA That's hard for a, a proxy for Bram Zuckerman to understand. Okay, what am I agreeing to? In that show 'em a example of a final trial that the final trial shows an estimate of 0.76. The Bayesian analysis set gives amine of 0.72 and a confidence interval that's less than one, that this would now be approvable.

28:36They can see exactly what they're agreeing to that. So that's one of the things we show them, is depending on the result, we also wanna show them if the new trial shows a hazard ratio of one, it doesn't say, oh, the the treatment works. They would not, they, they would say, wait a minute, wait a minute. You know, we, we don't agree to that. We don't want this Bayesian thing to flip that. So what is it flipping? Kert Viele: So we've had a couple situations, I can think of one in oncology with rare cancers where this was like your epilepsy example where we're trying to extend the label.

29:15And so we're borrowing across indications. And one of the things the FDA was particularly concerned about. Is, remember, these are all rare. We may not enroll very many. They never wanted a situation, and I should say ultra rare. I mean, we're gonna get a handful, but they didn't want us coming in and saying, you know, we got zero responses in four patients, and we're gonna say you should approve. On the basis of the prior, we need some, we need it to work somewhere, which makes perfect sense.

29:45We need some evidence that it's doing something. And so we've had those discussions. You know, show us the minimum that we've had. I remember one a while back it was, we were borrowing adult data to pediatrics and again, we were gonna get small sample sizes. And the FDA actually said explicitly. We would be willing if you got 15 responses out of, and I don't remember how many, uh, kids we were gonna be able to enroll, but basically if you can get 15 out of 25, we'll be happy to accept that, that we think that's enough given the adult data and everything.

30:24And so we basically back solved, you know, what would this do, and then showed that this had reasonable operating characteristics as well. But that was the way the interaction went in terms of. Here's the data that we think would be convincing to us. Scott Berry: What, and and so through this interaction, we're showing them the new power. We're showing them the type one era of this new. Modular experiment with the Bayesian analysis that happens at the end.

30:55We're showing them example trials. We've had people at the FDA say, okay, your new experiment has a 8% type one error rather than 2.5. In that, what if we just say you can run a a, a new phase three trial without Bayesian borrowing and you get an alpha of 0.08. That, that, that we go to, that, you know, in, in many circumstances, that gives the same success and failure of approval that the Bayesian analysis creates at the end of the day. Kert Viele: Okay, you're asking me a trick question.

31:37So this, this one gets a little nuanced here. Um, and I think the, the answer does depend on are you borrowing only control? Are you borrowing control and treatment? Are you borrowing dynamically or are you borrowing statically? Um, I'm gonna, I'm gonna certainly focus on the dynamic borrowing, which I think we prefer to do. Um, and I'll have to talk about what that is 'cause I don't think we've really brought that up. The dynamic borrowing the idea is if you're walking in with prior information from a previous trial, suppose your current trial, it's ongoing and it just looks different.

32:13So the rates are different than what you saw in history, the treatment effects, what you saw in history. The notion is, you know, should I continue to believe this prior or should I go, Hey, something different is going on there. And if you got into the math, what you're really saying is you have a prior where maybe the current trial matches history and maybe it doesn't. Is there, there's an aspect of that to your prior. So what happens when things don't match?

32:41You would borrow less. So you start to give less weight to history, and this is a mitigation strategy. When you have mismatch, you borrow less, you're less prone to make these false conclusions. Doing that is far better than just increasing the alpha because increasing the alpha, basically you're saying, I'm always willing to take this extra error regardless of what happens and you don't have any of these mitigation strategies. So I think it's really important that you go, you know, I, I'm, I'm changing the alpha because I have evidence that it works and I should be able to discount that evidence appropriately.

33:22If the new data says it's not right. Scott Berry: The, the other nice thing that come, well, two parts that comes from the Bayesian part that's a little bit different than just saying, oh, you get 0.08 alpha, is that there was an explicit incorporation of the strength of the previous data that generated that eight. Kert Viele: Yep. Scott Berry: what you got in the new experiment that gave you a final answer of 97.5% or 99, whatever that is. So it's not arbitrarily, oh, we think you get 8%, you get 11%.

33:58It's the combined data that tells you what that implied value is. But the other part is if you just take 8%, one sided, your estimate at the end of the day. Is not utilizing that information. The confidence interval doesn't match the result of the trial, uh, in that, and the estimate can be as important as the final result in the trial, which is a combined estimate. So there, there are, I, I, I believe there's lots of advantages of using the Bayesian approach to, to generate the final combined analysis of all of the data that we know.

34:40Jumping a regulatory hurdle. Kert Viele: And a lot of times that can be even borrowing good information. You may actually pull the point estimates down. We did that actually, it was the same rare cancer trial. It had nine groups in it. One of the early data sets it, it's a blockbuster success, but it estimated a 60% rate. And you know, the, the control was estimated to be about 10. So this is a huge treatment effect. But when we did all the borrowing.

35:13It didn't raise the 60% our prior. It certainly moved it. It. It was positive information, but it only estimated it to be about 48. 'cause it pulled it back and said, look, we don't think it's that good. There were also aspects, it was the highest group among many, and we've talked about that in other podcasts. But anyway, what was really cool about that is they continued to do follow up and they continue to enroll a few more patients. And lo and behold, the final data came out pretty close to 48%.

35:44So I mean, it, it, it will pull things down. Scott Berry: Okay. So, uh, uh, we've touched on a number of these issues. Uh, I'll give you an example. Of a case where this happened. Now, we, we have a number of examples of trials running right now at the FDA where there's been agreed upon Bayesian analysis of this. We we're going to them, uh, I'll give you an example of one where there was this agreement. The trial ran, and actually the data's been publicly disclosed, um, uh, because it went through an advisory committee meeting.

36:19So it, the, the treatment is called REBYOTA® And it was a Ferring company, Rebiotix And the treatment is for c Diff infection, and this REBYOTA® is now approved by Seaberg. In it and they had run a phase two trial. All of this information is public. Uh, it's a wonderful thing about the advisory committee meetings is that a huge amount of this is made public and we're doing less advisory committee meetings now. Now, and this is sort of negative of that, a whole separate topic.

36:53Um, but so the information's available. In the public register on this. So they ran a phase 2 trial, and this is a treatment for recurrent c difficile infection. This happens when individuals might be treating for cancer or a different disease. Um, they take antibiotics. It kind of destroys the gut biome. And c diff, which might live in all of us, takes over and, and now flourishes. Is, and it causes it, it causes debilitating GI issues. Diarrhea can actually be fatal, uh, in it.

37:35Now what they might do is give you another course of antibiotics and hope it sort of cures it. The recurrent aspects is, it doesn't. And the, and it's, this is a horrible, a horrible disease. The treatment is actually a fantastic treatment. It's a fecal matter transplant. They, they take feces from a healthy gut biome and through an enema. They, they give it to the individual, and the hope is that this restores the GI tracted healthy. Probiotics, healthy gut biome to to fight off the c diff infection.

38:17So they ran a phase 2 trial that was not statistically significant, showed a benefit of about 15% in preventing recurrent disease. This is a fantastic trial where there's only one endpoint. Uh, sure Mortality's looked at, but the only endpoint is does the disease come back? Yeah. You know there, there isn't a vast scale, there's nothing. There's one endpoint in the disease. Does the disease come back and the proportion of time it comes back, and if it doesn't come back within two months, you're in great shape.

38:51It's very unlikely to come back. It's almost a new circumstance of it, but if it does come back, then you're dealing with this. This case, they ran a phase two trial and showed benefit. They're now running a phase three trial is a rare disease and. These. Interestingly, it became very challenging to enroll this trial because there were outfits using FMT Fecal Matter, transplant Open Biome, for example, that patients could get, and this was not being shut down.

39:26The FDA could have shut down all these things, but they decided to, at their discretion, not, uh, but it never had never been approved before. So there's a circumstance where there's a belief this is beneficial. This phase two trial is showing now they're trying to run a phase three trial and challenging to get to 0.025 in this circumstance where I think largely the scientific community believes this works. So they agreed to borrow from the phase 2 trial.

39:54In the phase 3 trial, and have a combined analysis of those two trials meeting 97.5 with the combination of the two trials, the phase three trial read out, and it by itself, if you analyze only the phase three trial would've been about 93% probability. The treatment works in and of itself. That experiment didn't jump. The 0.025 hurdle combined with the phase 2 trial in the pre-specified analysis, pre-specified to the phase 3 trial, but not phase 2 Um, said 99.1% probability of the treatment is beneficial when you combine together the two trials.

40:45In the way that we worked with the FDA, um, uh, in creating this combined prior with the phase three was 99.1, which had jumped the 97.5 hurdle. That's the standard for clinical development. It given the rareness of the disease. It's not a two trial hurdle or the hurdle. We, we don't know whether we're in the one trial, two trial world of FDA in this circumstance, but given the rare disease, it very much fit into the, the one trial level of, of, of ev, of evidence there.

41:22This went to panel. Uh, public disclosure. And of course, the, the interesting thing is the statisticians were a bit of the hurdle as to whether this is appropriate. The statisticians split their vote one to one for it, but it, it was voted 13 to four that it demonstrated efficacy. Uh, in this circumstance, the treatment has been approved by FDA, it's on the market. And an interesting thing about it is that the FDA label for REBYOTA® Gives the 99.1 that came from the Bayesian analysis.

41:57There are no P values in the FDA label, but it has Bayesian posterior probabilities that it describes the combination of phase two and phase three as the estimate of efficacy that comes out of that, the treatment effect, relative difference estimate. Um, is based on that. And interestingly, at the FDA advisory committee meeting, the FDA statistician presented the posterior distribution based on the combined, showed a picture of the posterior and talked about the probability it's above zero, the probability it's at least 5% better, the probability it's at least 7% better.

42:41And this, this was. Based on FDA's analysis, the best summary of efficacy we have to present to you. Advisory committee meeting. And that picture of, of that stood on the screen for, uh, uh, a long time as here's what we're trying to, to. To make a decision on today, which was, which was sort of cool, so you can go forward and they do make reference in the new draft guidance of Rebi OTA as a case study of that, and the data's publicly disclosed for your scrutiny.

43:16Kert Viele: And the publications available. Scott Berry: And the publications available. It spends more time talking about, uh, other aspects of it than the, the pure Bayesian part. Uh, the adaptive design reports in there, the Bayesian, uh, uh, models in there. All of the pieces available, Uh, for it, which. Kert Viele: a lot of our best stuff is in the supplementary material, right? Scott Berry: Yes. Yeah, exactly. I would've loved to have stood up and, and explained to them, uh, uh, all of the Bayesian machinery, but, uh, that was not deemed to be the right thing to do, uh, in the circumstance.

43:49Uh, by, by the way, we're, uh, it'll be another podcast that we talk about, that particular example, a lot of, of, uh, Anna McLaughlin, Joe Marion here at Berry Consultants. Uh, Lindy Banky at, at rebi was the, the, the clinical lead of this and the one who spoke to the advisory committee meeting about she's fantastic, uh, sort of thing. So lots of people involved in, in that particular case, um, uh, with it. Okay, so have we, we missed any parts to what it looks like going to FDA good circumstances.

44:31Circumstances by the way that we say no to a client. No, we're not, we're not gonna go present that we, we would say no if we were at the FDA. We're not gonna present a Bayesian analysis of this. Uh, but we've done this probably couple dozen times. Um, over the years, probably more than that. We've been told no by the FDA by the way. Uh, we've presented what we thought was a very reasonable scientific argument for that. We've been told no, we've been told yes.

44:57We've been told, uh, can you change it in the following ways, uh, in that, so interaction is not uncommon in those, Kert Viele: We, we've been told to do it when we didn't propose it. Scott Berry: Yeah. Yeah. Uh, the rebi actually circumstances was a suggestion of the FDA, uh, Bram has made that suggestions. Other, other, uh, uh, scientists at the FDA had have suggested, oh, you should explore Bayesian methods, um, in the circumstance. Of course, going back to the original introduction of this, the, the home pregnancy test, uh, is now one of your colleagues that was, uh, that was Nick Berry, who is now a 34-year-old scientist here at Berry Co Consultants.

45:41So, uh, the Kert Viele: Thank you for making me feel old, Scott. Appreciate that. Scott Berry: I know, I know. Uh, as you, you, you babysat the youngster, Kert Viele: so so we should, I think the new podcast, we should always start with a story of view of Brad from grad school. I Scott Berry: Yeah, we have some of you too. So, uh, turnabout is, is fair play, uh, uh, in all of this. Yes. Alright, well we hope you enjoyed this little, uh, uh, Bayesian interlude here, which we hope is a more common thing with the FDA guidance.

46:18And until next time, we will be here. In the interim. Kert Viele: Thank you, Scott.