While AI has a wide range of applications and potential in radiology, the fact remains that most radiologists aren’t using AI in practice today. While AI appears to be everywhere, and press releases tout the potential of new algorithms daily, the fact remains that most radiologists currently don’t have access to AI applications for routine patient care.

Whether or not you are impressed by AI thus far might depend on your perspective. If you were expecting AI to have made the radiologist’s role obsolete by now, you are likely disappointed (sorry Geoff Hinton! ). If you were hoping for more clinical support, you might be cautiously optimistic. Why? Because recent research shows that some AI tools consistently perform as well as physicians in limited applications, but you might remain wary of ethical concerns or fear the algorithm “black box” that could obscure the decision-making behind the outputs.

Personally, I am enthusiastic about the role AI can play in supporting radiologists or possibly even eliminating some of the routine care we have to provide, making us available to bring even more valuable services to patients. If you are like me, you have likely concluded that AI applications are evolving slowly and unevenly, held back by data availability for algorithm training and by the inevitable challenges of developing a new technology that has not made its way from bench to bedside.

What Is the Role of Define-AI Use Cases?

To date, no other national medical specialty organization has developed AI use cases for developers so they can create AI algorithms that are relevant and valuable to healthcare professionals. The first-of-their-kind use cases in the newly released ACR DSI Define-AI Directory are building a framework to facilitate the development and implementation of AI applications that are poised to help radiology professionals in disease detection, characterization, and treatment.

The directory of use cases enables data scientists to produce radiology algorithms that:

1. Address relevant clinical questions to improve patient care
2. Support standardized pathways for implementation in the clinical workflow
3. Use standard data elements for creating training, testing, and validation data sets across multiple institutions
4. Contain data elements for monitoring performance in clinical practice using clinical data registries

What Do Use Cases Tell Developers?

The Define-AI use cases guide developers on the healthcare problems to solve, establishing clinical context and input and output specifications.
To demonstrate why these use cases are valuable, let’s examine one that may generate lots of interest in the development community because of the populations affected.

Scoliosis detection. It is estimated that in the U.S. more than four million people have scoliosis with a lateral spinal curvature/Cobb angle of 10° or more1. While identification of the curve apex and vertebral level is crucial for denoting the curve type, a developer might not realize that other measurements are also critical, and that those measurements — and the findings related to them — would have value to clinicians and might speed diagnosis.

Because of subtle patient differences in the clinical significance of scoliosis with the same lateral curvature, the ACR DSI Data Science Subspecialty Panel experts developing the scoliosis detection use case specify that algorithms for scoliosis evaluation provide not only the Cobb angle for the lateral curvature but also the pelvic tilt, because together they would be more valuable in helping physicians evaluate scoliosis than detection of a single parameter alone. An algorithm supplying these data points will immediately help clinicians with tasks that are typically time-intensive to perform and hard to reproduce.

For an algorithm to be trusted, the panel also determined that having good training data is important. Developers should train algorithms for scoliosis detection based on spinal examinations with images from the upright, supine, flexion, and extension views, with weight-bearing and non-weight-bearing patients.

Since other data elements (which sometimes go unreported) might influence clinical management in later years, developers should include them in algorithms as well, and the scoliosis detection use case makes that clear. Once AI developers can successfully perform the primary functions of detecting Cobb angle and pelvic tilt, the scoliosis detection use case suggests adding additional data points — including grading the patient’s Risser score (a classification based on the level of ossification of the iliac crest apophyses) and determining the triradiate cartilages as open or closed — as these observations and annotations would also be useful to radiologists.

To enable clinicians to easily track progress over time, algorithms can also add value by returning difference values of these measurements from prior imaging. That specification is part of the use case as well.

From breaking down the scoliosis detection use case, it’s easy to see why structured use cases will be extremely valuable to AI algorithm developers. They drill down into the details of specific inputs, provide specifications for training and testing datasets, and elaborate on additional features the algorithm should include in its outputs to AI users to increase the algorithm’s clinical value.

What Are Some Other Examples of Use Cases?

Here are two other highly relevant clinical scenarios found in the Define-AI Directory where AI can have an immediate impact on patient care:

Pneumothorax detection. Rapid diagnosis of a pneumothorax in patients with acute shortness of breath, those having recently undergone thoracic intervention, and patients in the ICU on a ventilator, is critically important. Unfortunately, diagnosis can be delayed because the clinical situation is difficult or not suspected, and radiographs are not immediately interpreted by a radiologist. At times, detection of a subtle pneumothorax on radiographs can be difficult as well.

AI algorithms to detect a pneumothorax could aid in treatment of this potentially life-threatening condition by prioritizing these patients for the radiologist. Algorithms will be able to speed diagnosis and early intervention and could prioritize worklists, queue up diagnostic tools for the radiologists, and alert referring providers of the pneumothorax condition.

Classifying risk of breast cancer based on micro-calcification volume and morphology. The average risk of a woman in the U.S. developing breast cancer sometime in her life is about 12 percent², but classification of calcifications as benign or malignant remains challenging. Women undergoing biopsy of suspicious calcifications often have a benign diagnosis. Furthermore, we are learning that not all ductal carcinomas in situ will progress to invasive carcinomas.

AI algorithms based on this use case will classify micro-calcifications and may perform as well or better than humans at classifying the risk of invasive cancer. They may be able to prevent false-positive biopsies without a significant associated risk of underdiagnoses.

Change Is Coming

Ultimately, a good algorithm must both be clinically relevant and readily integrated into radiologists’ standard workflow to be well accepted by the market and provide value to patients. Structured use cases make sure developers have the information they need to make that happen.

Next month, a DSI blog by Dr. Patti will detail the efforts of the ACR and the experts on the ACR DSI Data Science Subspecialty Panels in developing standards and supplying developers with radiology problems amenable to AI solutions — including the almost 50 use cases in the Define-AI Directory.

We're now on the cusp of discovering the healthcare innovations that may be unlocked by continued advancements in AI technology. The use cases in the TOUCH-AI Directory will aid developers in creating AI tools that can be integrated into medicine and will help physicians tackle important problems — offering more people access to high-quality healthcare and supporting the radiologists that provide that care.

Change is coming, and Define-AI use cases have the potential to make a difference in ensuring that AI in radiology works to the benefit of our patients.



By Bibb Allen, Jr., MD, FACR | ACR DSI Chief Medical Officer; Diagnostic Radiologist, Grandview Medical Center, Birmingham, AL


1. Scoliosis in Children, Scoliosis Statistics, Clear Scoliosis Institute, accessed September 30, 2018. https://www.clear-institute.org/learning-about-scoliosis/scoliosis-statistics/

2. Noone AM, Howlader N, Krapcho M, et al. (editors). SEER Cancer Statistics Review, 1975-2015. Table 4.17. National Cancer Institute. Bethesda, MD. http://seer.cancer.gov/csr/1975_2015/