A new proof-of-concept study published today in the American Journal of Infection Control (AJIC) reports that artificial intelligence (AI) technologies can accurately identify cases of healthcare-associated infections (HAI) even in complex clinical scenarios. The study, which highlights the need for clear and consistent language when using AI tools for this purpose, illustrates the potential for incorporating AI technology as a cost-effective component of routine infection surveillance programs.
According to the most recent HAI Hospital Prevalence Survey conducted by the Centers for Disease Control and Prevention, there were approximately 687,000 HAIs in acute care hospitals in the U.S. and 72,000 HAI-related deaths among hospital patients in 2015. About 3% of all hospital patients have at least one HAI at any given time. The implementation of infection surveillance programs and other infection-prevention protocols has reduced the incidence of HAIs, but they remain a risk, particularly to critically ill hospitalized patients with inserted devices such as central lines, catheters, or breathing tubes.
Many hospitals and other healthcare facilities have HAI surveillance programs to monitor for increased infection risk, but they require extensive resources, training, and expertise to maintain. In resource-constrained settings, a cost-effective alternative could help to enhance surveillance programs and allow for better protection of high-risk patients.
In this new study, researchers at Saint Louis University and the University of Louisville School of Medicine evaluated the performance of two AI-powered tools for accurate identification of HAIs. One tool was built using OpenAI’s ChatGPT Plus and the other was developed using an open-source large language model known as Mixtral 8x7B.
The tools were tested on two types of HAIs: central line-associated bloodstream infection (CLABSI) and catheter-associated urinary tract infection (CAUTI). Descriptions of six fictional patient scenarios with varying levels of complexity were presented to the AI tools, which were then asked whether the descriptions represented a CLABSI or a CAUTI. The descriptions included information such as the patient’s age, symptoms, date of admission, and dates that central lines or catheters were inserted and removed. AI responses were compared to expert answers to determine accuracy.
For all six cases, both AI tools accurately identified the HAI when given clear prompts. Importantly, the researchers found that missing or ambiguous information in the descriptions could prevent the AI tools from producing accurate results. For example, one description did not include the date a catheter was inserted; without that detail the AI tool could not give a correct response. Abbreviations, lack of specificity, use of special characters, and dates reported in numeric format instead of with the month spelled out all led to inconsistent responses.
“Our results are the first to demonstrate the power of AI-assisted HAI surveillance in the healthcare setting, but they also underscore the need for human oversight of this technology,” said Timothy L. Wiemken, PhD, MPH, an associate professor in the division of infectious diseases, allergy, and immunology at Saint Louis University and lead author of the paper. “With the rapid evolution of the role of AI in medicine, our proof-of-concept study validates the need for continued development of AI tools with real-world patient data to support infection preventionists.”
Additional details about the study include:
- Both AI tools were used with retrieval augmented generation, an approach that improves the quality of prompting through a knowledge repository that gives the AI tool additional context. In this case, the repository included material from CDC’s National Healthcare Safety Network, a tracking system for HAIs.
- The ChatGPT Plus tool developed for this study, HAI Assist, is available at the OpenAI GPT Store for people with a ChatGPT Plus subscription.
“HAI surveillance is a critical responsibility for infection preventionists, and our community needs every possible tool to help us ensure the safety of our patients,” said Tania Bubb, PhD, RN, CIC, FAPIC, 2024 APIC president. “This study suggests that AI-powered tools may offer a cost-effective means of improving our surveillance programs by assisting infection preventionists in day-to-day work functions.”
