ACR DSI Use Cases in Artificial Intelligence are detailed specifications that allow developers to create AI algorithms that assist radiology professionals in disease detection, characterization, and treatment. They are built around a framework
that brings radiology professionals, industry partners, and other stakeholders together to develop, validate, deploy, and monitor artificial intelligence (AI) algorithms in medical imaging and the radiological sciences. The following
details the ACR DSI Use Case development process.
Define Use Cases:
ACR DSI works with radiology professionals, including diagnostic and interventional radiologists, radiation oncologists, and medical physicists, to define and prioritize use cases around highly relevant clinical scenarios in which AI may
help improve patient care. Designed to guide effective algorithm development, ACR DSI Use Cases include narrative descriptions and flowcharts that specify the goals the algorithm should meet, the required clinical inputs, how it should
integrate into the clinical workflow, and how it should interface with both human end-users and an array of electronic resources, such as reporting software, PACS, and electronic health records.
In the interest of considering a broad range of possible use cases, the DSI solicits use case proposals from the radiology community. There are radiologists, medical physicists, developers, and others who propose ideas that become DSI use cases and eventually XML modules (within PACS or VRS) for seamless algorithm integration into the clinical environment. Use case panels choose the highest value proposals for collaboration, sharing their clinical expertise while ACR provides administrative support to create the use case.
ACR DSI leverages two ACR open frameworks, Technology-Oriented Use Cases for Healthcare AI (TOUCH-AI) and Computer Assisted Reporting with Data Science (CARDS).
It uses these frameworks to deploy an array of tools and common data elements that radiologists can use to create annotated data sets for training, testing, and validating algorithms, integrating algorithm outputs into the clinical
workflow, and monitoring algorithms’ performance in clinical practice.
Algorithm Training and Testing:
ACR DSI makes tools available to help radiology departments create training and testing data sets around its use cases. ACR DSI maintains a public directory of institutions that have created such data sets and assists in connecting these
institutions with developers who want to use the data sets to build algorithms around its use cases.
ACR DSI Use Cases include data elements that use statistical tools to analyze algorithm performance across multiple sites prior to FDA approval. ACR DSI’s validation service, known as ACR Certify-AI, offers developers a secure method to
statistically validate their algorithms and gather performance data ahead of the FDA pre-market approval process, going far beyond the demographic scope of the original training and testing data sets.
ACR has longstanding relationships with numerous health care IT vendors and has worked with them to deploy tools, such as ACR Select and ACR Assist,
into the workflow of radiology professionals and other physicians. ACR DSI Use Cases build on these interfaces to seamlessly integrate AI algorithms into clinical practice.
Built for each use case, ACR Assist–AI modules leverage these pathways to create standard application programming interfaces (APIs) that integrate algorithm outputs into existing reporting software, PACS, electronic health records, and
other electronic resources. ACR DSI creates unique identifiers for each AI algorithm developed around its use cases and maintains a public directory of these algorithms. It also manages a directory that tracks which reporting vendors
have incorporated which models into their reporting platforms.
ACR DSI uses these data elements to collect specific information about the algorithm’s effectiveness from both radiologists at the point of care and metadata about the exam, including the equipment manufacturer, the field strength or number
of detectors, and other relevant examination parameters. ACR DSI collects these data in the ACR National Radiology Data Registry, and the ACR DSI algorithm
monitoring service, ACR Assess-AI, sends effectiveness reports to developers for FDA post-market surveillance and for adaptive learning to improve algorithm performance in the field.