An Interview with Distinguished University Professor Gordon Guyatt, OC, FRSC on GOBSAT, Guidelines, Living Documents, and Network Meta-Analysis (Part One)

Author(s): Scott Douglas Jacobsen

Publication (Outlet/Website): In-Sight: Independent Interview-Based Journal

Publication Date (yyyy/mm/dd): 2019/09/08

Abstract 

Dr. Gordon Guyatt, OC, FRSC is a Distinguished University Professor is the Department of Health Research Methods, Evidence, and Impact at McMaster University. He discusses: professional research in the fall of 2018; hopes of a reduction in the timelines of publication of guidelines; reducing communication time and update time; message to skeptics of medicine in the mainstream; other professional areas to explore; early hypothesized applications of network meta-analysis or NMA; limits to pairings of NMAs; 2010s as the decade of NMAs; and the integration of NMAs into guideline methodologies.

Keywords: Canada, evidence-based medicine, Gordon Guyatt, medicine, network meta-analysis, NMA.

An Interview with Distinguished University Professor Gordon Guyatt, OC, FRSC on GOBSAT, Guidelines, Living Documents, and Network Meta-Analysis: Distinguished Professor, Health Research Methods, Evidence, and Impact, McMaster University; Co-Founder, Evidence-Based Medicine (Part One)^{[1],[2],[3],[4]}

*Footnotes in & after the interview, & citation style listing after the interview.*

*This interview has been edited for clarity and readability.*​

1. Scott Douglas Jacobsen: With regards to professional research for the fall of 2018, what are some of the new projects arising? What are the new questions being asked?

Gordon Guyatt: I will tell you some, what I would consider highlights. So, clinicians nowadays are relying heavily on guidelines. So, there is a medical electronic textbook called up to date that provides recommendations.

That is probably the number one resource in the world, certainly in North America, and provides recommendations. There are specialty societies in heart and lung and kidney and everything else. Young and older clinicians are paying great attention to these guidelines.

Over the last 20 years, there have been standards developed for trustworthy guidelines. There was an older model of guidelines, which was affectionately referred to as “GOBSAT”.

Jacobsen: [Laughing]

Guyatt: GOBSAT means good old boys sitting around a table.

Jacobsen: [Laughing] That is right.

Guyatt: That is how guidelines used to be developed but now we have a science of producing guidelines. I have been fortunate to be involved in the development of some of those standards. So, you have these up to date standards. It produces recommendations quickly. I do consult for them, trying to make them more evidence-based.

So, they do a pretty good job, but the nature of what they do, they do not adhere fully to the standards of trustworthy guidelines. Pretty good. Then some, not far from all, but some of the specialty societies adhere to the standards of trustworthy guidelines.

But they put together a team and it is a big production and it takes them a year to get there to get their guideline together, and then they’re so exhausted that they have to wait another two years before they start the process again.

So, the guidelines, many of them become quickly out of date. So, the question is, “Can one have trustworthy guidelines that are also updated when there is new evidence that is quickly updated?” So, there is also a science of pulling together the literature.

We call it systematic reviews. Again, I have been privileged too, as that science has developed over, starting a little farther ago, maybe 30 years ago. We know how to do that. I have been involved.

So, it was a systematic review out there. A new piece of evidence, we can update it quickly. So, it is good to know. Then a colleague who trained with me from Norway, who was interested in the whole guideline endeavour. He said, “Somebody’s got to do it, so that we follow the trustworthy guidelines standards. We update quickly, you and your team show we can update our systematic reviews quickly.”

We do have a process for producing this the trustworthy guidelines quickly. I said, “Pierre, you are right, but who’s gonna listen to us? We are not up to date. We are not the American Thoracic Society or the American Heart Association, who is going to listen to us?”

So Pierre said, “What if one of the top journals put out our new recommendations and published it as their endorsed recommendations?” He persuaded the BMJ to buy into the idea. So, for the last couple of years, we have been producing what we call BMJ rapid recommendations. So, a new study is published, we think it’s practicing changing information. We get our systematic review done in a matter of several weeks.

We put our guideline panel together and target within 90 days of the publication then we have our updated recommendation. Ideally, it would come in that time frame, as we think we have done our part. BMJ [Laughing] has been a little slow.

But producing these BMJ rapid recommendations, it is exciting. We have provided the ability once before to say that success will be when and with a particular activity, we become redundant. It felt good at the time, 20 years later, when we start to become redundant; it does not feel quite so good.

So, I shouldn’t be worn. But hopefully, these BMJ rapid recommendations someday will be redundant, when they went out of date then they can bump up their standards of maybe making theirs more trustworthy, and or the sub-specialty societies realize that they shouldn’t be putting out these guidelines every two years.

They should be making them living guidelines and updating. But until they do that, and we become redundant, it is fun leading the field and providing a model that this can be done, and how it should be done.

2. Jacobsen: With some software, people do open source. So, you have these, as you phrase it “living documents,” but software. So, people have continually updated and improved, basically, algorithms to perform a specific task, which is an idea built into that.

By analogy is interesting, if you were to smooth out some of the rough edges of process, and, into 2020 say, what will be your hope in terms of not only reduction of the 90 days, but also in terms of big journals, like BMJ, in terms of their process of publication?

Guyatt: They could. It is tough for them. But I could tell them ways that they could do it. It requires resources. Even the top journals have limited resources and require a level of commitment and devoted staff, which is resource-intensive, it couldn’t be done.

It would be better if it didn’t need to be the journals. So, the specialty societies have their ‘why we gave up on them’ is they have their bureaucracy. As they pass through their board before it gets out, they need a way of streamline.

Because they should all have teams our BMJ rapid recommendation team ready to update continuously. It started, the journal publication process slows things down. They have their bureaucracy. They need to streamline that to get their living guidelines out quickly.

So, that is what we would hope would happen in the future. That is still the model we want to provide that will help them do that.

3. Jacobsen: Can there be a way in which to use something like an open-source platform to reduce, for instance, communication time and update time?

Guyatt: We have something called the system of developing these recommendations. It has been around for 15 years or so. We call it GRADE. So, it is a way of looking at the quality of the evidence and the strength of recommendations. It is this graded approach that is a crucial part of trustworthy guidelines. Pierre and his colleagues in Norway have come up with a nonprofit company that they call MAGIC, for “making great the irresistible choice.”

Jacobsen: [Laughing].

Guyatt: It includes the MAGIC app, which is an electronic platform.

And we think this platform is there. As soon as we finished, we have it on the platform. It can be disseminated. It is out there. As a link anybody can go to, but before we can put it out, we would likely have to get the pass to go through the BMJ peer review process.

Jacobsen: Yes.

Guyatt: We could do it for specialty societies and put it out in the same way. But it has to go through their process. So, we have got to the electronic communication part of it. We have got that. It is that the bureaucracy is saying, “Yes, you can disseminate it now.”

4. Jacobsen: This brings to mind something like a mild peripheral question. When people are excessively skeptical about the scientific process, especially in medicine, often, I get the sense that they do not understand how difficult getting good-quality evidence and robust guidelines of research out are in the end analysis. What will be your short message to them?

Guyatt: These are people who are skeptical about the whole medical scientific endeavour?

Jacobsen: Yes.

Guyatt: The fraud is extremely unusual – unusual, but it happens. Overwhelmingly, scientists are interested in making the world a better place. Where they need to be cautious is financial conflicts of interest, researchers being attached to their own research; everybody thinks that their study is the greatest.

That is the problem. So, there are dangers. But there is such a thing as high-quality science, which, as you say, is challenging to produce, challenging to summarize. But there are trustworthy sources of evidence. They have to get some help in recognizing them.

5. Jacobsen: Is there anything else that in the professional areas we want to explore?

Guyatt: Trouble is, I am not sure what; I am what’s called a methodologist. So what turns me on is not the latest discovery of this, that, and the other thing, which might be the general audience thing, but advancing the methods.

So, at the expense of not necessarily being the most interesting finding, we came up with systematic reviews, which take all the best evidence and have a scientific standard for putting together all the best evidence.

Now, we know how to do that and with a set of rigorously. But then we summarize it, we can get each outcome for death, heart attack, stroke. We have a summary that says, “Here is our best estimate of the effect of the treatment on mortality, sorry it does not affect mortality, on stroke it reduces stroke by a small amount. Sorry, it affected myocardial infarction. Oh, there is a big effect in reducing myocardial infarction,” so hard at times.

So, we have got the system and it is called meta-analysis. Meta-analysis is the statistical approach, where you put all the evidence together, and you come up with the best estimate of effect.

But you could compare two treatments. But now often we have six treatments, or sometimes 12 alternatives. So, for instance, there are probably 20 drugs out there. When you are depressed, there are 20 drugs out there.

You’ve got rheumatoid arthritis, we have these new biologic agents, there are 10 of these biologic agents. Which one is the best? Or what are the best?

What are the collections of ones that do better than others?

We didn’t have a way of doing this. In the last decade, the statisticians have come up with something called network meta-analysis, where you can simultaneously compare all these treatments.

This is new science. We are figuring out how to do it well, how to interpret the results. The results are coming out in these huge tables that are completely uninterpretable to anybody. How do we take that? How do we take that and summarize it in a way that is true to the data and is still useful to the condition?

This is all an adventure to get these networks. This network meta-analysis optimized, and to find ways of having an output that is true to the data and still makes sense so the conditions in our health are helpful to the conditions and to the patients. Our group was involved in that process. That is exciting for us.

6. Jacobsen: What are some of the early applications hypothesized with regards to network meta-analysis?

Guyatt: So here’s one of the things. It is an initially misguided approach. We thought that this will tell you the best treatment; seldom is there a best treatment, so it needs to be reframed as, “Here are the three that you might want to consider for these reasons. You probably do not want to even think about these other three.” So, your patient may fit best with one of these three, which have their merits.

So, we are able to say, “Here, of these dozen things out here, here’s the two or three that you might want to take that you are that are thought to be best for your patients.” So, there is an explosion of these network meta-analysis, providing that advice.

So, without this approach, it was much, much more difficult when you have a dozen things out there. You have this paired comparison with A versus B, and then another C versus D. But the drug companies all compared to placebo, they do not have too many comparisons of these things. When they do you have A to B, B to C, but he hasn’t been compared to D, and so on.

Jacobsen: Right.

Guyatt: So, how do you make sense of this? The network meta-analysis allows one to make sense of it.

7. Jacobsen: Is there a limit to the pairings in the network meta-analysis?

Guyatt: No. There is no limit to network meta-analysis.

Jacobsen: That is exciting.

Guyatt: The limits are when you have a net we have meta-analysis with a dozen treatments or 15 treatments, the output is this…

Jacobsen: …[Laughing]…

Guyatt: …gigantic, take A versus B, C, D, E, F, G, H, I, J, K, and CB vs. blah, blah, blah. Anyways, these giant tables reach outcomes that are extremely difficult to make any sense out of. Also the limitation, the arithmetic, the statistic goal, work and handling of any numbers comparisons.

It is more difficult to make sense with the more comparisons there are; the more difficult it is to make sense of the network that emerges. Our work is in making some things about the statistics and how to do that best, but also interpreting and making sense of the whole thing, and interpreting in a way that makes sense to clinicians.

8. Jacobsen: With regards to evidence-based medicine, could the 2010s be considered the network meta-analysis decade?

Guyatt: Yes, yes, yes, yes. There are other things. There are other things that I hope will be part of the next decade. But yes, definitely, meta-analysis itself, the first was a huge advance. This network meta-analysis definitely takes things forward.

Jacobsen: So if these updates to these rapid-fire guidelines happen, and if we do them in 90 days, we send them off to the journal; and they have their own margin of error.

Guyatt: 90 days is supposed to take into account the journal processing.

Jacobsen: [Laughing].

Guyatt: We had one or two that have made the 90 days, but not quite. We are supposed to do ours in 60 days. They give them 30.

Jacobsen: So then 60-120 days. The 60-120 days’ rapid-fire updates. So, basically getting, somewhere between six and three of these per year, given that timeline.

Guyatt: But we are working on some of them simultaneously.

Jacobsen: [Laughing] So, even more.

Guyatt: But, our capacities are limited, 6 to 9 per year.

9. Jacobsen: That is cool. So, you are doing this in the 2010s, with a network meta-analysis, and you are having these guidelines updated nine to 12 per year. How integrated is network meta-analysis into this guideline methodology in terms of producing them?

Guyatt: Good question, makes it more challenging. We have had two. We have done definitely closer to the 6 per year. So, we have done a dozen or so of them, which have involved network meta-analysis.

So, but again, we are getting better at doing the NMAs quickly. I must admit, the NMAs, the Network Meta-Analysis, that we have done has already involved a few treatments. We can try to do one quickly with a dozen treatments.

We’ll get there. But we need more experience before we can take that on. But we have done them with relatively small networks, and we have done NMAs in the updating process.

Appendix I: Footnotes

[1] Distinguished Professor, Health Research Methods, Evidence, and Impact, McMaster University; Co-Founder, Evidence-Based Medicine

[2] Individual Publication Date: September 8, 2019, at http://www.in-sightjournal.com/guyatt-one; Full Issue Publication Date: January 1, 2020, at https://in-sightjournal.com/insight-issues/.

[3] B.Sc., University of Toronto; M.D., General Internist, McMaster University Medical School; M.Sc., Design, Management, and Evaluation, McMaster University.

[4] Credit: McMaster University.

License

In-Sight Publishing by Scott Douglas Jacobsen is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. Based on a work at www.in-sightpublishing.com.

Copyright

© Scott Douglas Jacobsen and In-Sight Publishing 2012-Present. Unauthorized use and/or duplication of this material without express and written permission from this site’s author and/or owner is strictly prohibited. Excerpts and links may be used, provided that full and clear credit is given to Scott Douglas Jacobsen and In-Sight Publishing with appropriate and specific direction to the original content. All interviewees and authors co-copyright their material and may disseminate for their independent purposes.