When the closure of the National Guideline Clearinghouse (NGC) hit the news, the ECRI team who worked on the contract felt as though we had lost a dear friend. After all, we had a 20-year history with NGC. That’s a long relationship by any measure. Yet, we had little time to mourn because we had an organizational commitment to disseminate evidence-based guidelines into practice.
ECRI’s decision to create its own guideline repository was welcome news to many members of the guideline community. However, it left some people wondering how our new repository would compare to NGC. I’m here to tell you that the ECRI Guidelines Trust® is not a reboot of NGC. NGC was a groundbreaking initiative, and it should hold its own place in guideline history.
We launched the Guidelines Trust last November with 300 guidelines; a significantly modest number compared to the 1,400 guidelines housed in NGC when it closed. Some former NGC users were curious as to why our repository didn’t duplicate the content included in NGC. That’s a fair question.
The game begins
If you grew up playing board games, Monopoly* may have been a game in your stack. You may recall that the key to the game’s success is to acquire many properties—and to build multiple houses and hotels on them. If people are measuring the value of the Guidelines Trust based solely on its volume of guidelines, they are not realizing the full utility of the site. Here’s why.
Five hundred (35%) of the guidelines represented in NGC did not meet the Guidelines Trust inclusion criterion related to currency. We require that guidelines be published within the last five years. That’s important for providers and researchers who rely on up-to-date guidelines. Simply put, ECRI will not develop or host content from outdated guidelines. Doing so goes against our mission to promote safe and effective patient care. It’s not an option to disseminate old or outmoded practices that can potentially harm patients.
Some people wondered why we didn’t just copy NGC’s content over to our site. Well, even though ECRI developed all of the NGC guideline summaries, that work was done under contract to the U.S. Agency for Healthcare Research and Quality (AHRQ). The content belongs to AHRQ—not ECRI—nor anyone else for that matter. We have no right to recycle or repackage it.
Instead, we started from scratch—reaching out to guideline developer organizations seeking permission to represent their content on our site. It has only been four months since our launch, so it’s not surprising that our content is in the hundreds, and not the thousands. That’s okay—it’s growing daily. After all, we haven’t made our way around the entire “Monopoly” board yet.
All guidelines are not equal
Yes, we know there is strength in numbers, but in guideline development we value strength of evidence. Clearly, not all guidelines are created equal. So, having lots of guidelines to choose from is less beneficial than knowing which guidelines are trustworthy—and the best fit for your clinical or research needs. Practitioners tell us that they do not want to weed through numerous guidelines to find the best one. We aim to aid this process by vetting and scoring the guidelines for you.
We do this using the widely accepted guidance for guidelines—The Institute of Medicine (IOM) Standards for Developing Trustworthy Clinical Practice Guidelines. We develop TRUST Scorecards for those guidelines based on a verifiable systematic review of the evidence, assessing them against the extent to which they adhere to specific elements of the IOM standards, which include:
- Composition of Guideline Development Group
- Systematic Review of Evidence
- Foundations for Recommendations
- Funding Source Disclosure
- Disclosure and Management of Financial COIs
- External Review
The Scorecards allow our members to see how well individual guidelines scored across these areas. Why is scoring guidelines important? Preparing guidelines for digitization is a goal for a majority of guideline developers; however, not all developers have a clear direction on how to get there.
Last month while on a call with a group of guideline developers, the discussion turned to implementation. A developer asked how to decide which guideline to select for inclusion in an electronic health record (EHR) when multiple competing guidelines on the same topic existed. It’s a great question that comes up quite often, yet there is no clear or simple answer. Many factors can make up your selection criteria, such as:
- The strength of the guideline recommendations
- Whether the recommendations are actionable
- Whether the recommendations can be measured to assess their impact
Also looking at the totality of the guideline with respect to how closely it adheres to the IOM Standards may be another factor to add to your list of considerations.
During another meeting with guideline developers, it became obvious that there is no consistency in the way developers are approaching meeting the standards. Some developers have an arduous and almost continual process for managing intellectual and financial conflicts of interest (COI) for their guideline workgroup. Others have check-ins at certain time-points within the development process to determine changes in workgroup members’ COIs.
Involving patients in the guideline process also differs by developer. Some developers are still challenged with how to efficiently incorporate the patient perspective in their development process. The guideline update process also varies by developer. Some are updating more frequently than five years. Others use a committee to nominate topics for potential update, regardless of timeframe. There is also wide-ranging variability on the way literature searches are performed, as well as the staff responsible for performing them. Given some of these differences, it may be beneficial to develop guidance to help developers meet the standards, or at least get closer to them. This effort could be a collaborative taken on by a number of developers representing a range of specialties, along with methodology organizations supporting the guideline development process.
We recognize that developers must keep pace as the guideline field evolves, and must do so with limited resources and a high bar for excellence. If we can begin to develop resources to assist with meeting the standards, which can offer some consistency in approach, perhaps we can next collectively focus on developing practical guidance to help inform implementation, dissemination, and measurement.
After all, no one ever said Monopoly was a short game. Just one worth playing.
We encourage you to take a “CHANCE” and register for the Guidelines Trust to see how it can benefit your long game.
*Darrow, Charles (2006). Monopoly: the property trading board game. Hasbro / Parker Bros.