Evaluating Diversity in an Artificial Intelligence (AI) Model for Breast Cancer
If your job involves interpreting mammograms to screen for breast cancer, you know how time consuming and challenging that task can be. Developing artificial intelligence (AI) and machine learning to improve this imaging-analysis step can be equally difficult.
It’s especially challenging to find enough data to make your algorithms applicable to every patient, in every situation. Yet you need a wide spectrum of data to ensure your model applies to real-world situations—and, even then, performance may vary across patient groups.
A recent study, supported in part by NCI, confirms the value of AI diversity.
Read the full report, “Patient Characteristics Impact Performance of AI Algorithm in Interpreting Negative Screening Digital Breast Tomosynthesis Studies,” in Radiology.
When evaluating the performance of an AI breast cancer model in 4,855 patients, the researchers found differences in performance depending on various patient characteristics (such as race/ethnicity, age, and breast density).
Corresponding author Dr. Derek Nguyen, from the Department of Radiology at Duke University, said, “Mammograms are incredibly complex images, and radiologists are better at interpreting some studies rather than others. Today’s AI algorithms can help compensate for a radiologist’s weak spots; but, like humans, AI algorithms could benefit from clinical feedback to improve their performance.”
He added, “The developer of the model we tested has access to large and diverse screening mammography data sets, which are in use worldwide. Moving forward, we can leverage these data to better understand those performance differences among patient populations.”
Dr. Lars Grimm, senior author on the study, added, “Our study underscores the value for AI vendors to be transparent in how they train their algorithms and their algorithm’s performance in diverse clinical settings. This will allow radiologists to understand how they should incorporate AI findings into their own clinical judgments.”
The study team is collaborating with the model’s developer to further understand the nuances of AI’s performance in different settings and to refine AI algorithms overall.