Artificial intelligence (AI) has the potential to aid radiologists in numerous ways, evidenced in part by the extensive list of radiologist-defined AI use cases compiled by the ACR DSI. Despite this huge potential, there are challenges facing the development of clinically useful AI models.

Many of the challenges stem from the fact that imaging data features vary from facility to facility. This occurs for various reasons. While some are well understood, such as variability in equipment manufacturers or patient populations, many are hard to interpret. We do know that AI models trained on data from a single facility are biased toward that facility’s population and do not generalize well across facilities. In practice, this means that AI models trained on data from Facility A can perform poorly when tested on data from Facility B. Ultimately this slows the radiology community in developing clinically useful models which minimize bias and health inequity.

One way to get around this issue is to acquire training data in a central location from a diverse array of source facilities. Unfortunately, due to the sensitive nature of medical imaging data, this is often not a viable option. That is where federated learning (FL) can help. It offers a solution where multiple facilities participate in model training, while data remain local to each facility and secure behind their firewalls.

Tackling the data sharing problem with federated learning
FL is a method of collaborative AI model training that does not require data sharing. During FL, AI models, not data, are shared among participating facilities and a coordinating server. Facilities train models locally before passing them to a coordinating server, which then aggregates multiple facility-trained local models into a single AI model. In this way, FL allows model training across a distributed network of facilities.

AI models created through FL are generally more robust and able to be generalized across facilities than single-facility-trained models because training data derives from more diverse sources and patient populations. The performance of final AI models produced with FL often reach levels of performance similar to models trained on data collected in a central location.

Recently, ACR participated in a FL initiative that produced a breast-density classification model which performed better on a test dataset consisting of data from all participating facilities than any of the single facility-trained local models produced in the study.

The AI-LAB app for radiologists
ACR Connect® is free ACR-developed software, which can be installed on premises. It offers various medical imaging analysis tools and makes them readily accessible through a series of apps. ACR Connect apps make it possible to participate in cross-site research initiatives and create curated local datasets, without the need for data to leave a site’s protective firewalls.

The AI-LAB™ app in ACR Connect provides even non-technical users with the ability to create or evaluate AI models through an easily navigated graphical interface. Since AI-LAB’s focus is both educating the radiology community about AI and providing a tool to help radiologists develop robust AI models, AI-LAB offers a simple interface and various background material to bring radiologists up to speed about AI. This includes videos explaining important AI concepts and help content throughout the application. As a result, AI-LAB users without any technical expertise can participate in AI development and validation and the radiology community can build expertise in developing clinically useful models.

How AI-LAB federated learning works
AI-LAB is introducing new FL capabilities. At the conceptual level, the process of FL is fairly straightforward with AI-LAB via ACR Connect. To begin, an ACR central server passes a base model, which was developed previously to perform the same or similar task but found to have non-ideal performance, to the ACR Connect instance at each participating facility. At each facility, the base model is then trained on a local dataset prepared by the site, which is used to create a new site-specific model.

New models from each site will be passed back to the ACR central server for aggregation. The process of aggregation will create a single AI model that will be used as the base model in the next round of training. When repeated over many rounds, this process will produce a final AI model that performs well at all participating facilities. For a more detailed look at how ACR Connect clinical integration services work, visit the website.

On the ACR side, a custom AI-LAB implementation will leverage NVIDIA FLARE, which enables multi-party orchestration, training, model sharing, and model aggregation.

Get involved now to prepare for an AI-enabled future
By early 2022, AI-LAB users will be able to get started using the technology to benefit from FL in developing AI models. Registration is now open for practices that would like to participate in FL via AI-LAB. Facilities that sign up to participate will then install ACR Connect (or access a secure instance in the cloud) to participate in the first FL session. Prior to participation, facilities will be required to create a relevant dataset using ACR Connect Data Manager.

Register your site today by visiting the AI-LAB home page, and clicking Learn More in the “Federated Learning” section.

Kendall Schmidt, PhD | Data Scientist | American College of Radiology