Top Ideas & Best Uses of AI / LLMs in Clinical Practice Guidelines Development

Best Uses of AI LLMs in Guidelines Development

This article will examine a number of ideas and top suggestions for incorporating LLMs and AI, including Chatbots like GPT4 or Llama, into your clinical practice guidelines development process and workflows. As I’m sure you’ve heard, AI is improving and expanding rapidly, and that this is a moving target, so we’ll build upon this series with additional ideas and best practices for using AI and LLMs for guideline authoring and development as time goes on.

Supporting Literature Searches and Systematic Reviews – Finding Publications

AI has been used in literature reviews for many years. AI can be used to both support the search itself, as well as summarizing and categorizing the results. There are numerous benefits, including quicker and more efficient evidence generation, grouping and prioritization of references based on contents, and more efficient reviews with better quality control. There are companies that specialize in this, but you could also check out specific GPTs like Sholar GPT or Consensus.

Extracting Key Clinical Data

LLMs and chatbots have proven to be invaluable in extracting data. This includes data extraction from published literature, extraction from in progress word documents, or even extraction from PDFs! There are many different use cases of AI for extracting data, but one could be to identify inclusion/exclusion criteria, PICO-related fields, action types, or more.. The “Data Analyst” GPT works well, and a brand new GPT called “Clinical Data Extractor” from Indigo Solutions is now available.

Associating Keywords and Codes with Guidelines and/or Specific Guideline Recommendations

This specific use case straddles both the guideline development process, as well as the dissemination and implementation use case. But one concept would be to utilize an LLM to analyze a specific guideline manuscript, including the guideline recommendations. From there, keywords could be assigned at the document/guideline and/or recommendation level. These keywords could be used to support many activities, including further mapping of those keywords to clinical terminologies, such as ICD10, CPT or SNOMED codes.

Support Image and/or Flowchart/Algorithm Creation

While it may take a bit of trial and error and fine tuning with prompt engineering, Chatbots like ChatGPT can be great tools to support the creation of images or algorithms. Specific GPTs include, but are not limited to DALL-E and image generator, and a more specialized GPTs for flowcharts is called Diagrams.

Helping During the Authoring Process

This one may seem obvious, but one of the best use cases we’ve found for Chatbots during any sort of publication/authoring, including guidelines development, is using them to help with the writing process. This could include rewording or phrasing key bits of information, simplifying complicated concepts, finding synonyms, and more. You can even use it for copy editing or as a very basic fact checker. GPTs to check out include Copywriter, AI-humanizer, and Write for Me.

Making Life Easier During the Public Comment Process

One of the biggest hassles guideline developers deal with is addressing hundreds of comments from external reviewers during the peer review or public comment process. I know society partners who have received literally thousands of comments on a single guideline manuscript! LLMs/AI can make this process easier by summarizing the comments and sentiments into more manageable and bite sized chunks. In fact, we are Guideline Central, along with a team from the Harvard Data Analytics Group, have developed an AI-enhanced clinical guidelines development peer review and public comments tool that we will be launching later this year. It will be available to all guideline developers for free, and will incorporate AI to help streamline the process of responding to comments. Stay tuned for more details!

Did we miss something? If so, let us know how you are using AI during your medical guidelines development processes, and what has/hasn’t worked well. For now, this concludes our list of the best ideas for integrating LLMs and AI into your clinical guideline development processes. We will expand on this series in the coming months, including a similar list of best practices for utilizing AI for guidelines dissemination and implementation.

Are you interested in learning more about how AI or LLMs / Chatbots can support your guideline development, or even guidelines dissemination or implementation efforts? If so, contact us today to learn more about our clinical guidelines AI solutions.