Woojin Kim, MD

ACR DSI Chief Medical Officer

Musculoskeletal Radiologist, Palo Alto VA Medical Center

Glendale, CA

In a paper by Google in 2015, the authors state, “It may be surprising to the academic community to know that only a tiny fraction of the code in many ML systems is actually devoted to learning or prediction.” They illustrate this using a now-iconic figure, depicting large boxes labeled "Data Collection," "Feature Extraction," "Serving Infrastructure," "Monitoring," etc., surrounding a small black box in the middle labeled "ML Code" to demonstrate the complex infrastructure required for real-world machine learning (ML) systems. [1] Similarly, adopting AI in clinical settings is challenging with complexity beyond the accuracy of models that many research papers and AI vendors focus on. IT solutions must demonstrate enhanced clinical workflows and deliver tangible value to patients and providers.  

In healthcare, the value equation is often described as the quality of care (including outcomes, safety, and service) divided by the total cost. [2] This framework underscores the intricate relationship between quality, safety, and informatics in the lifecycle of clinical IT systems – from development through implementation and ongoing use. Westbrook and her colleagues identified three key lessons from their research for maximizing the potential of health technology to enhance patient care.  

1. Anticipate and mitigate new error types. While new IT systems can improve safety, they may also introduce unforeseen errors. Whether implementing a new system or transitioning between systems, healthcare organizations must establish robust monitoring to identify issues, develop agile response strategies, and cultivate a continuous learning and improvement culture.  
2. Prioritize the human element. Successful health IT implementation extends beyond technology to support its users. With increasing staff burnout and workforce shortages, organizations should design intuitive human-technology interfaces optimized for real-world scenarios, provide comprehensive training and ongoing support, and regularly solicit and incorporate feedback.
3. Problem-first, technology second approach. Resist the temptation to apply a single technological solution across multiple contexts. Instead, begin by clearly defining the problem, evaluate various options to find the best fit for each issue, and implement solutions iteratively to allow for adjustments. [3] 

As health IT becomes increasingly ubiquitous, maintaining a strong focus on quality and safety is paramount. Conversely, a deep understanding of IT systems is crucial to ensure that quality and safety remain at the forefront of healthcare delivery. The ACR Quality and Safety + Informatics Conference provides a unique forum for quality and safety professionals to collaborate with informaticists, driving improvement and positive change in radiology. This year’s conference will be held September 19-21 in Washington, DC.  

On Day 1, we will begin with a keynote by Ashwini Davison, MD, FACP, FAMIA, the CMIO - Enterprise Imaging Data Strategy for Amazon Web Services, followed by sessions on QI leadership, maintaining quality through workforce shortages, learning from failures and redesigning EHR to drive QI, learning from ACR’s Learning Network, and managing errors and learning from failures. On Day 2, we will begin with a keynote by physician leader and board-certified radiologist Rick Abramson, MD, MHCDS, FACR, followed by sessions on data access sharing and usability, clinical implementation of AI, building your AI team, health system merges, the role of AI in equity, and what quality can learn from informatics. Finally, on Day 3, I will give a keynote, followed by sessions on empowering local AI teams through the ACR toolset, information security, and policy and regulation of AI.  

By building a community focused on setting and achieving shared goals, you can enhance patient outcomes. Join us at the 2024 Quality and Safety + Informatics Conference for dynamic presentations that foster collaboration and drive improvement and positive change in the radiology field. Register to attend in person or virtually to earn up to 18.5 CME credits.  

Woojin Kim, MD | ACR DSI Chief Medical Officer | Musculoskeletal Radiologist at Palo Alto VA Medical Center

References 

1. Sculley D, Holt G, Golovin D, et al. Hidden Technical Debt in Machine Learning Systems. Advances in Neural Information Processing Systems 28 (NIPS 2015). https://proceedings.neurips.cc/paper_files/paper/2015/file/86df7dcfd896fcaf2674f757a2463eba-Paper.pdf. Accessed August 11, 2024. 
2. Smith TM. What is value-based care? These are the key elements. https://www.ama-assn.org/practice-management/payment-delivery-models/what-value-based-care-these-are-key-elements. Accessed August 11, 2024. 
3. Ellis LD. How Health Care Leaders Can Optimize IT Systems to Improve Clinical Quality and Patient Safety. https://postgraduateeducation.hms.harvard.edu/trends-medicine/how-health-care-leaders-can-optimize-systems-improve-clinical-quality-patient-safety. Accessed August 11, 2024.