Imaging Pearls ❯ Deep Learning ❯ Deep Learning and Breast Apps
|
-- OR -- |
|
- Background: Retrospective studies have suggested that using artificial intelligence (AI) may decrease the workload of radiologists while preserving mammography screening performance.
Purpose: To compare workload and screening performance for two cohorts of women who underwent screening before and afterAI system implementation.
Materials and Methods: This retrospective study included 50–69-year-old women who underwent biennial mammography screening in the Capital Region of Denmark. Before AI system implementation (October 1, 2020, to November 17, 2021), all screenings involved double reading. For screenings conducted after AI system implementation (November 18, 2021, to October 17, 2022), likely normal screenings (AI examination score ≤5 before May 3, 2022, or ≤7 on or after May 3, 2022) were single read by one of 19 senior fulltime breast radiologists. The remaining screenings were read by two radiologists with AI-assisted decision support. Biopsy and surgical outcomes were retrieved between October 1, 2020, and April 15, 2023, ensuring at least 180 days of follow-up. Screening metrics were compared using the χ2 test. Reading workload reduction was measured as saved screening reads.
Early Indicators of the Impact of Using AI in Mammography Screening for Breast Cancer
Andreas D. Lauritzen, PhD • Martin Lillholm, PhD • Elsebeth Lynge
Radiology 2024; 311(3):e232479 - Results: In total, 60 751 and 58 246 women were screened before and after AI system implementation, respectively (median age, 58 years [IQR, 54–64 years] for both cohorts), with a median screening interval before AI of 845 days (IQR, 820–878 days) and with AI of 993 days (IQR, 968–1013 days; P < .001). After AI system implementation, the recall rate decreased by 20.5% (3.09% before AI [1875 of 60 751] vs 2.46% with AI [1430 of 58 246]; P < .001), the cancer detection rate increased (0.70% [423 of 60 751] vs 0.82% [480 of 58 246]; P = .01), the false-positive rate decreased (2.39% [1452 of 60 751] vs 1.63% [950 of 58 246]; P < .001), the positive predictive value increased (22.6% [423 of 1875] vs 33.6% [480 of 1430]; P < .001), the rate of small cancers (≤1 cm) increased (36.6% [127 of 347] vs 44.9% [164 of 365]; P = .02), the rate of node-negative cancers was unchanged (76.7% [253 of 330] vs 77.8%[273 of 351]; P = .73), and the rate of invasive cancers decreased (84.9% [359 of 423] vs 79.6% [382 of 480]; P = .04). The reading workload was reduced by 33.5% (38 977 of 116 492 reads).
Conclusion: In a population-based mammography screening program, using AI reduced the overall workload of breast radiologists while improving screening performance.
Early Indicators of the Impact of Using AI in Mammography Screening for Breast Cancer
Andreas D. Lauritzen, PhD • Martin Lillholm, PhD • Elsebeth Lynge
Radiology 2024; 311(3):e232479 - Key Results
■ In a retrospective study comparing 60 751 and 58 246 women screened for breast cancer before and after artificial intelligence (AI) system implementation, respectively, 66.9% (38 977 of 58 246) of the screenings after AI system implementation were single read, and 33.1% (19 269 of 58 246) were double read with AI assistance.
■ AI system implementation improved the cancer detection rate (0.70% before AI vs 0.82% with AI; P = .01), false-positive rate (2.39% vs 1.63%; P < .001), positive predictive value (22.6% vs 33.6%; P < .001), and rate of small cancers out of all diagnosed invasive cancers (36.6% vs 44.9%; P = .02).
■ Screening with AI decreased radiologist reading workload by 33.5% (38 977 of 116 492 reads) and the recall rate by 20.5% (3.09% before AI vs 2.46% with AI
Early Indicators of the Impact of Using AI in Mammography Screening for Breast Cancer
Andreas D. Lauritzen, PhD • Martin Lillholm, PhD • Elsebeth Lynge
Radiology 2024; 311(3):e232479 - “In conclusion, our findings indicate that using an artificial intelligence (AI) system in population-based mammography screening improved screening performance and reduced workload. Future work should evaluate screening sensitivity, specificity, interval cancer rate, and the impact of higher ductal carcinoma in situ detection. Around November 2024, we will have full 2-year follow-up data for the cohort of women screened with AI. Further, in future work, we aim to quantify the effects of AI stratification, AI decision support, and radiologist access to prior screenings separately.”
Early Indicators of the Impact of Using AI in Mammography Screening for Breast Cancer
Andreas D. Lauritzen, PhD • Martin Lillholm, PhD • Elsebeth Lynge
Radiology 2024; 311(3):e232479
- Background:The Personal Performance in Mammographic Screening (PERFORMS) scheme is used to assess reader performance. Whether this scheme can assess the performance of artificial intelligence (AI) algorithms is unknown.
Purpose:To compare the performance of human readers and a commercially available AI algorithm interpreting PERFORMS test sets.
Materials and Methods:In this retrospective study, two PERFORMS test sets, each consisting of 60 challenging cases, were evaluated by human readers between May 2018 and March 2021 and were evaluated by an AI algorithm in 2022. AI considered each breast separately, assigning a suspicion of malignancy score to features detected. Performance was assessed using the highest score per breast. Performance metrics, including sensitivity, specificity, and area under the receiver operating characteristic curve (AUC), were calculated for AI and humans. The study was powered to detect a medium-sized effect (odds ratio, 3.5 or 0.29) for sensitivity - Results:A total of 552 human readers interpreted both PERFORMS test sets, consisting of 161 normal breasts, 70 malignant breasts, and nine benign breasts. No difference was observed at the breast level between the AUC for AI and the AUC for human readers (0.93% and 0.88%, respectively; P = .15). When using the developer’s suggested recall score threshold, no difference was observed for AI versus human reader sensitivity (84% and 90%, respectively; P = .34), but the specificity of AI was higher (89%) than that of the human readers (76%, P = .003). However, it was not possible to demonstrate equivalence due to the size of the test sets. When using recall thresholds to match mean human reader performance (90% sensitivity, 76% specificity), AI showed no differences inperformance, with a sensitivity of 91% (P =. 73) and a specificity of 77% (P = .85).
Conclusion:Diagnostic performance of AI was comparable with that of the average human reader when evaluating cases from two enriched test sets from the PERFORMS scheme. - Conclusion: Diagnostic performance of AI was comparable with that of the average human reader when evaluating cases from two enriched test sets from the PERFORMS scheme.
Performance of a Breast Cancer Detection AI Algorithm Using the Personal Performance in Mammographic Screening Scheme.
Chen Y, Taib AG, Darker IT, James JJ.
Radiology. 2023 Sep;308(3):e223299. doi: 10.1148/radiol.223299. PMID: 37668522. - Key Results
■No difference in performance was observed between artificial intelligence (AI) and 552 human readers in the detection of breast cancer in 120 examinations from two Personal Performance in Mammographic Screening test sets (area under the receiver operating characteristic curve, 0.93 and 0.88, respectively; P = .15).
■When using AI score recall thresholds that matched mean human reader performance (90% sensitivity, 76% specificity), AI showed no difference in sensitivity (91%, P = .73) or specificity (77%, P = .85) compared with human readers.Figure 1:Flow diagram shows human reader inclusion and exclusion criteria. NHSBSP = National Health Service Breast Screening Programme, PERFORMS =
Personal Performance in Mammographic Screening Performance of a Breast Cancer Detection AI Algorithm Using the Personal Performance in Mammographic Screening Scheme.
Chen Y, Taib AG, Darker IT, James JJ.
Radiology. 2023 Sep;308(3):e223299. doi: 10.1148/radiol.223299. PMID: 37668522. - “The AI algorithm used was a commercially available product (Lunit INSIGHT MMG, version 1.1.7.1; Lunit). All images were analyzed using Lunit software installed on the author’s (Y.C.) local server at the University of Nottingham. Lunit had no access to the cases before, during, or after the study. AI acted as an independent reader of cases. The Lunit AI algorithm provided scores that rated suspicion of malignancy against each feature detected on a scale of 0 (low) to 100 (high). The highest rating given to a feature detected within each breast was taken as the overall score for that breast and compared with the ground truth. As each breast was scored separately, an abnormality requiring re-call had to be localized to the correct breast by AI (Fig 2). When no features of interest were detected, a breast-level score of zero was assigned “
Personal Performance in Mammographic Screening Performance of a Breast Cancer Detection AI Algorithm Using the Personal Performance in Mammographic Screening Scheme.
Chen Y, Taib AG, Darker IT, James JJ.
Radiology. 2023 Sep;308(3):e223299. doi: 10.1148/radiol.223299. PMID: 37668522. - “Diagnostic performance of a commercially available arti-ficial intelligence (AI) algorithm was comparable with that of human readers when evaluating cases from two enriched test sets of the Personal Performance in Mammographic Screening (PERFORMS) scheme. The use of external quality assessment schemes like PERFORMS may provide a model for regularly assessing the performance of AI in a way similar to the monitor-ing of human readers, but further work is needed to ensure this assessment model could work for other AI algorithms, screening populations, and readers.”
Personal Performance in Mammographic Screening Performance of a Breast Cancer Detection AI Algorithm Using the Personal Performance in Mammographic Screening Scheme.
Chen Y, Taib AG, Darker IT, James JJ.
Radiology. 2023 Sep;308(3):e223299. doi: 10.1148/radiol.223299. PMID: 37668522. - “Double reading is recommended by European guide-lines to optimize mammographic sensitivity and is com-mon practice in the United Kingdom (5). Shortages in qualified readers worldwide make double reading an un-sustainable burden, for which AI is a logical solution. The results of this study suggest that AI could confidently act as a second reader to decrease workloads. While double read-ing has generally not been used in the United States, many U.S. radiologists interpreting mammograms are nonspecialized and do not read high volumes of mammograms. Thus, the AI system evaluated by Chen et al could be used as a supplemental tool to aid the performance of readers in the United States or in other countries where screening programs use a single reading.”
The Days of Double Reading Are Numbered: AI Matches Human Performance for Mammography Screening.
Philpotts L.
Radiology. 2023 Sep;308(3):e232034. doi: 10.1148/radiol.232034. PMID: 37668520. - “Mammogram interpretation remains one of the most areas of radiology and is as much an art as it is a science. The adaptation of AI is also proving to be a finely crafted art. The development and incorporation of AI into breast imaging practice has and will continue to be challenging. The precise role of AI is still to be determined; however, studies such as the one conducted by Chen et al will help move the field in a positive direction. As a second reader, it appears AI has a definite role that should ease the demanding job of reading large volumes of screening mammograms.”
The Days of Double Reading Are Numbered: AI Matches Human Performance for Mammography Screening.
Philpotts L.
Radiology. 2023 Sep;308(3):e232034. doi: 10.1148/radiol.232034. PMID: 37668520.
- ”AI-supported mammography screening resulted in a similar cancer detection rate compared with standard double reading, with a substantially lower screen-reading workload, indicating that the use of AI in mammography screening is safe. The trial was thus not halted and the primary endpoint of interval cancer rate will be assessed in 100 000 enrolled participants after 2-years of follow up.”
Artificial intelligence-supported screen reading versus standard double reading in the Mammography Screening with Artificial Intelligence trial (MASAI): a clinical safety analysis of a randomised, controlled, non-inferiority, single-blinded, screening accuracy study
Kristina Lång et al.
Lancet Oncol 2023; 24: 936–44 - “The results from this randomised trial support the findings of earlier retrospective studies, indicating a general potential of AI to improve screening efficacy and reduce workload. The clinical safety analysis concludes that the AI-supported screen-reading procedure can be considered safe. Implementation of AI in clinical practice to reduce the screen-reading workload could therefore be considered to help address workforce shortages. The assessment of the primary endpoint of interval cancer rate, together with a characterisation of detected cancers in the entire study population, will provide further insight into the efficacy of screening, possible side-effects such as overdiagnosis, and the prognostic implications of using AI in mammography screening, taking cost-effectiveness into account.”
Artificial intelligence-supported screen reading versus standard double reading in the Mammography Screening with Artificial Intelligence trial (MASAI): a clinical safety analysis of a randomised, controlled, non-inferiority, single-blinded, screening accuracy study
Kristina Lång et al.
Lancet Oncol 2023; 24: 936–44 - To our knowledge, this is the first randomised controlled trial investigating the use of AI in mammography screening. In this first report, the objective was to assess the safety of an AI-supported screen-reading procedure, involving triage and detection support. AI-supported screening resulted in 20% more cancers being detected and exceeded the lowest acceptable limit for safety compared with standard double reading without AI, without affecting the false positive rate. The AI supported screen-reading procedure enabled a 44·3% reduction in the screen-reading workload. The results indicate that the proposed screening strategy is safe.
Artificial intelligence-supported screen reading versus standard double reading in the Mammography Screening with Artificial Intelligence trial (MASAI): a clinical safety analysis of a randomised, controlled, non-inferiority, single-blinded, screening accuracy study
Kristina Lång et al.
Lancet Oncol 2023; 24: 936–44 - “In summary, this clinical safety analysis of the MASAI trial, in which an AI system was used to triage screening examinations to single or double reading and as detection support, showed that AI-supported mammography screening can be considered safe, since it resulted in a similar rate of screen-detected cancer—exceeding the lowest acceptable limit for safety—without increasing rates of recalls, false positives, or consensus meetings, and while substantially reducing the screen-reading workload compared with screening by means of standard double reading.”
Artificial intelligence-supported screen reading versus standard double reading in the Mammography Screening with Artificial Intelligence trial (MASAI): a clinical safety analysis of a randomised, controlled, non-inferiority, single-blinded, screening accuracy study
Kristina Lång et al.
Lancet Oncol 2023; 24: 936–44
- Purpose: To compare selected existing mammography artificial intelligence (AI) algorithms and the Breast Cancer Surveillance Consortium (BCSC) risk model for prediction of 5-year risk.
Materials and Methods: This retrospective case-cohort study included data in women with a negative screening mammographic examination (no visible evidence of cancer) in 2016, who were followed until 2021 at Kaiser Permanente Northern California. Women with prior breast cancer or a highly penetrant gene mutation were excluded. Of the 324 009 eligible women, a random subcohort was selected, regardless of cancer status, to which all additional patients with breast cancer were added. The index screening mammographic examination was used as input for five AI algorithms to generate continuous scores that were compared with the BCSC clinical risk score. Risk estimates for incident breast cancer 0 to 5 years after the initial mammographic examination were calculated using a time-dependent area under the receiver operating characteristic curve (AUC).
Comparison of Mammography AI Algorithms with a Clinical Risk Model for 5-year Breast Cancer Risk Prediction: An Observational Study
Vignesh A. Arasu et al.
Radiology 2023; 307(5):e222733 - Results: The subcohort included 13 628 patients, of whom 193 had incident cancer. Incident cancers in eligible patients (additional 4391 of 324 009) were also included. For incident cancers at 0 to 5 years, the time-dependent AUC for BCSC was 0.61 (95% CI: 0.60, 0.62). AI algorithms had higher time-dependent AUCs than did BCSC, ranging from 0.63 to 0.67 (Bonferroni-adjusted P < .0016). Time-dependent AUCs for combined BCSC and AI models were slightly higher than AI alone (AI with BCSC time-dependent AUC range, 0.66–0.68; Bonferroni-adjusted P < .0016).
Conclusion: When using a negative screening examination, AI algorithms performed better than the BCSC risk model for predicting breast cancer risk at 0 to 5 years. Combined AI and BCSC models further improved prediction.
Comparison of Mammography AI Algorithms with a Clinical Risk Model for 5-year Breast Cancer Risk Prediction: An Observational Study
Vignesh A. Arasu et al.
Radiology 2023; 307(5):e222733 - Summary
“Negative screening mammographic examinations were analyzed with five artificial intelligence (AI) algorithms; all predicted breast cancer risk to 5 years better than the Breast Cancer Surveillance Consortium (BCSC) clinical risk model, and combining AI and BCSC models further improved prediction.”
Comparison of Mammography AI Algorithms with a Clinical Risk Model for 5-year Breast Cancer Risk Prediction: An Observational Study
Vignesh A. Arasu et al.
Radiology 2023; 307(5):e222733 - Key Results
■ Five artificial intelligence (AI) algorithms were used to generate continuous risk scores from retrospectively acquired screening mammographic examinations negative for cancer in 18 019 women.
■ AI predicted incident cancers at 0 to 5 years better than the Breast Cancer Surveillance Consortium (BCSC) clinical risk model (AI time-dependent area under the receiver operating characteristic curve [AUC] range, 0.63–0.67; BCSC time-dependent AUC, 0.61; Bonferroni-adjusted P < .0016).
■ Combining AI algorithms with BCSC slightly improved the time dependent AUC versus AI alone (AI with BCSC time-dependent AUC range, 0.66–0.68; Bonferroni-adjusted P < .0016).
Comparison of Mammography AI Algorithms with a Clinical Risk Model for 5-year Breast Cancer Risk Prediction: An Observational Study
Vignesh A. Arasu et al.
Radiology 2023; 307(5):e222733 - “Mammography AI algorithms provide an approach for improving breast cancer risk prediction beyond clinical variables such as age, family history, or the traditional imaging risk biomarker of breast density. The absolute increase in the AUC for the best mammography AI relative to BCSC was 0.09 for interval cancer risk and 0.06 for overall 5-year risk, a substantial and clinically meaningful improvement. The overall performance improvement remained when restricting the analysis to invasive cancer only. In order for an AI model to achieve an AUC of approximately 0.7, the model must have predictors that are two to three times more informative than clinical models such as the BCSC with an AUC of approximately 0.6 (1)”
Comparison of Mammography AI Algorithms with a Clinical Risk Model for 5-year Breast Cancer Risk Prediction: An Observational Study
Vignesh A. Arasu et al.
Radiology 2023; 307(5):e222733 - “In conclusion, mammography artificial intelligence (AI) algorithms provided prediction of breast cancer risk to 5 years that was better than the Breast Cancer Surveillance Consortium (BCSC) clinical risk model, and the combination of AI and BCSC models further improved prediction. Our results imply that mammography AI algorithms alone may provide a clinically meaningful improvement compared with current clinical risk models at early time horizons (ie, time during which risk is assessed), with further improvements in prediction when AI and clinical risk models are combined. Although AI algorithm performance declines with longer time horizons, most of the algorithms evaluated have not yet been trained to predict longer-term outcomes, suggesting a rich opportunity for further improvement.”
Comparison of Mammography AI Algorithms with a Clinical Risk Model for 5-year Breast Cancer Risk Prediction: An Observational Study
Vignesh A. Arasu et al.
Radiology 2023; 307(5):e222733
- “Breast cancer poses a global public health threat, with an estimated 2.3 million cases in 2020 and predictions that there will be more than 3 million cases by 2040 (1). Although breast cancer mortality has decreased due to advances in early detection and diagnosis, approximately 685 000 women still annually die of the disease. A greater proportion of these deaths affect women living in low- and middle-income countries. Although such lower-resource environments face many challenges, one of the greatest problems is the lack of integrated health infrastructure that provides timely access to preventative screening and subsequent surgical and oncologic care. Although there have been many technological advances over the past few decades, mammographic screening is not commonplace in these countries, and, therefore, unlike in the United States, most women with breast cancer present with a palpable mass and often locally advanced disease. This presentation, coupled with a relative lack of access to state-of-the-art oncologic care, translates into higher morbidity and mortality.” from the disease.
The Promise of AI in Advancing Global Radiology
Priscilla J. Slanetz
Radiology 2023; 00:e230895 - “The results show that inexperienced, moderately experienced, and very experienced radiologists reading mammograms are prone to automation bias when being supported by an AI-based system. This and other effects of human and machine interaction must be considered to ensure safe deployment and accurate diagnostic performance when combining human readers and AI.”
Automation Bias in Mammography: The Impact of Artificial Intelligence BI-RADS Suggestions on Reader Performance
Thomas Dratsch et al.
Radiology 2023; 000:1–9 - Background: Automation bias (the propensity for humans to favor suggestions from automated decision-making systems) is a known source of error in human-machine interactions, but its implications regarding artificial intelligence (AI)–aided mammography reading are unknown.
Conclusion: The results show that inexperienced, moderately experienced, and very experienced radiologists reading mammograms are prone to automation bias when being supported by an AI-based system. This and other effects of human and machine interaction must be considered to ensure safe deployment and accurate diagnostic performance when combining human readers and AI.
Automation Bias in Mammography: The Impact of Artificial Intelligence BI-RADS Suggestions on Reader Performance
Thomas Dratsch et al.
Radiology 2023; 000:1–9 - “ The percentage of correctly rated mammograms by inexperienced, moderately experienced , and very experienced radiologists was significantly impacted by the correctness of the AI prediction of BI-RADS category. Inexperienced radiologists were significantly more likely to follow the suggestions of the purported AI when it incorrectly suggested a higher BI-RADS category than the actual ground truth compared with both moderately and very experienced readers.”
Automation Bias in Mammography: The Impact of Artificial Intelligence BI-RADS Suggestions on Reader Performance
Thomas Dratsch et al.
Radiology 2023; 000:1–9 - “Several studies have shown a synergistic combination of radiologists and AI is possible. However, there is also the danger that radiologists may stop critically engaging with the AI results and start mindlessly following them. This overreliance on a decision support system, known as automation bias, has been observed in a wide range of fields, such as aviation, engineering, and medicine, and a long line of research has focused on the conditions that cause automation bias to arise.”
Automation Bias in Mammography: The Impact of Artificial Intelligence BI-RADS Suggestions on Reader Performance
Thomas Dratsch et al.
Radiology 2023; 000:1–9 - Summary
Incorrect advice by a purported artificial intelligence–based decision support system impaired the performance of radiologists with varying levels of expertise, ranging from inexperienced to very experienced, when reading mammograms.
Key Results
■ In this prospective experiment, 27 radiologists who interpreted 50 mammograms with the assistance of a purported artificial intelligence (AI)–based system were significantly affected by incorrect suggestions from the system.
■ Inexperienced radiologists were more likely to follow the suggestions of the AI system when it incorrectly suggested a higher Breast Imaging Reporting and Data System category compared with moderately (mean degree of bias, 4.0 +/- 1.8 vs 2.4 +/- 1.5; P = .044; r = 0.46) and very (mean degree of bias, 4.0 +/- 1.8 vs 1.2 +/- 0.8; P = .009; r = 0.65) experienced readers.
Automation Bias in Mammography: The Impact of Artificial Intelligence BI-RADS Suggestions on Reader Performance
Thomas Dratsch et al.
Radiology 2023; 000:1–9 - “Overall, our results show that automation bias can affect the performance of radiologists regardless of their experience. Therefore, specific strategies should be considered to mitigate automation bias. Previous research has shown that presenting users with the confidence levels of the decision support system can help reduce automation bias by keeping users more critically engaged with the output of the system. In the case of an AI-based system, this could be implemented by displaying the probability of each output. Another strategy to mitigate automation bias involves educating users about the reasoning process of the decision support system .”
Automation Bias in Mammography: The Impact of Artificial Intelligence BI-RADS Suggestions on Reader Performance
Thomas Dratsch et al.
Radiology 2023; 000:1–9 - AI and User Experience
- AI has the potential to make everyone “an expert”
- AI has the potential to increase error if the program is not accurate as less experienced users are more apt to trust AI - AI and Early Cancer Detection
- Breast cancer
- Pancreas cancer
- Colon cancer
- Lung cancer
- Liver cancer - AI and Its Impact
- Radiology
- Pathology
- Dermatology
- Ophthalmology
- Internal Medicine
- Surgery
- Objectives: To evaluate and compare the diagnostic performances of a commercialized artificial intelligence (AI) algorithm for diagnosing pulmonary embolism (PE) on CT pulmonary angiogram (CTPA) with those of emergency radiologists in routine clinical practice.
Methods: This was an IRB-approved retrospective multicentric study including patients with suspected PE from September to December 2019 (i.e., during a preliminary evaluation period of an approved AI algorithm). CTPA quality and conclusions by emergency radiologists were retrieved from radiological reports. The gold standard was a retrospective review of CTPA, radiological and clinical reports, AI outputs, and patient outcomes. Diagnostic performance metrics for AI and radiologists were assessed in the entire cohort and depending on CTPA quality.
How artificial intelligence improves radiological interpretation in suspected pulmonary embolism
Alexandre Ben Cheikh et al.
European Radiology 2022https://doi.org/10.1007/s00330-022-08645-2 - Results: Overall, 1202 patients were included (median age: 66.2 years). PE prevalence was 15.8% (190/1202). The AI algorithm detected 219 suspicious PEs, of which 176 were true PEs, including 19 true PEs missed by radiologists. In the cohort, the highest sensitivity and negative predictive values (NPVs) were obtained with AI (92.6% versus 90% and 98.6% versus 98.1%, respectively), while the highest specificity and positive predictive value (PPV) were found with radiologists (99.1% versus 95.8% and 95% versus 80.4%, respectively). Accuracy, specificity, and PPV were significantly higher for radiologists except in subcohorts with poor-to-average injection quality. Radiologists positively evaluated the AI algorithm to improve their diagnostic comfort (55/79 [69.6%]).
Conclusion: Instead of replacing radiologists, AI for PE detection appears to be a safety net in emergency radiology practice due to high sensitivity and NPV, thereby increasing the self-confidence of radiologists.
How artificial intelligence improves radiological interpretation in suspected pulmonary embolism
Alexandre Ben Cheikh et al.
European Radiology 2022 https://doi.org/10.1007/s00330-022-08645-2 - Key Points
• Both the AI algorithm and emergency radiologists showed excellent performance in diagnosing PE on CTPA (sensitivity and specificity ≥ 90%; accuracy ≥ 95%).
• The AI algorithm for PE detection can help increase the sensitivity and NPV of emergency radiologists in clinical practice, especially in cases of poor-to-moderate injection quality.
• Emergency radiologists recommended the use of AI for PE detection in satisfaction surveys to increase their confidence and comfort in their final diagnosis.
How artificial intelligence improves radiological interpretation in suspected pulmonary embolism
Alexandre Ben Cheikh et al.
European Radiology 2022https://doi.org/10.1007/s00330-022-08645-2 - “Indeed, in the entire cohort-2019, AIDOC captured 19 PEs that were not diagnosed by radiologists in 19 distinct patients. In other words, the AI algorithm could correct a misdiagnosed PE approximately every 63 CTPAs (≈1202/19). This estimation must be considered in parallel with the high number of CTPAs required by emergency physicians (≈18,000 CTPAs in 2020 in our group—so approximately 285 [≈18000/1202 × 19] true PEs detected by AI but initially misdiagnosed by radiologists in 2020) and with human and financial consequences of missed PEs [32]. Indeed, mortality and recurrence rates for untreated or missed PE range between 5 and 30%.”
How artificial intelligence improves radiological interpretation in suspected pulmonary embolism
Alexandre Ben Cheikh et al.
European Radiology 2022https://doi.org/10.1007/s00330-022-08645-2 - “In conclusion, this study confirms the high diagnostic performances of AI algorithms relying on DCNN to diagnose PE on CTPA in a large multicentric retrospective emergency series. It also underscores where and how AI algorithms could better support (or “augment”) radiologists, i.e., for poor- quality examinations and by increasing their diagnostic con- fidence through the high sensitivity and high NPV of AI. Thus, our work provides more scientific ground for the concept of “AI-augmented” radiologists instead of supporting the theory of radiologists’ replacement by AI.”
How artificial intelligence improves radiological interpretation in suspected pulmonary embolism
Alexandre Ben Cheikh et al.
European Radiology 2022https://doi.org/10.1007/s00330-022-08645-2
- “In an independent study of six radiologists, the AI system outperformed all of the human readers: the area under the receiver operating characteristic curve (AUC-ROC) for the AI system was greater than the AUC-ROC for the average radiologist by an absolute margin of 11.5%. We ran a simulation in which the AI system participated in the double-reading process that is used in the UK, and found that the AI system maintained non-inferior performance and reduced the workload of the second reader by 88%. This robust assessment of the AI system paves the way for clinical trials to improve the accuracy and efficiency of breast cancer screening.”
International evaluation of an AI system for breast cancer screening
Scott Mayer McKinney et al
Nature | Vol577 | 2January2020 - Objectives: Digital breast tomosynthesis (DBT) increases sensitivity of mammography and is increasingly implemented in breast cancer screening. However, the large volume of images increases the risk of reading errors and reading time. This study aims to investigate whether the accuracy of breast radiologists reading wide-angle DBT increases with the aid of an artificial intelligence (AI) support system. Also, the impact on reading time was assessed and the stand-alone performance of the AI system in the detection of malignancies was compared to the average radiologist.
Conclusions: Radiologists improved their cancer detection and reduced reading time when evaluating DBT examinations using an AI reading support system.
Impact of artificial intelligence support on accuracy and reading time in breast tomosynthesis image interpretation: a multi-reader multi-case study.
van Winkel SL et al
Eur Radiol. 2021 Nov;31(11):8682-8691. - • Radiologists improved their cancer detection accuracy in digital breast tomosynthesis (DBT) when using an AI system for support, while simultaneously reducing reading time.
• The stand-alone breast cancer detection performance of an AI system is non-inferior to the average performance of radiologists for reading digital breast tomosynthesis exams.
• The use of an AI support system could make advanced and more reliable imaging techniques more accessible and could allow for more cost-effective breast screening programs with DBT.
Impact of artificial intelligence support on accuracy and reading time in breast tomosynthesis image interpretation: a multi-reader multi-case study.
van Winkel SL et al
Eur Radiol. 2021 Nov;31(11):8682-8691.
- “Screening mammography aims to identify breast cancer at earlier stages of the disease, when treatment can be more successful . Despite the existence of screening programmes worldwide, the interpretation of mammograms is affected by high rates of false positives and false negatives .Here we present an artificial intelligence(AI) system that is capable of surpassing human experts in breast cancer prediction. To assess its performance in the clinical setting, we curated a large representative dataset from the UK and a large enriched dataset from the USA. We show an absolute reduction of 5.7% and 1.2% (USA and UK) in false positives and 9.4% and 2.7% in false negatives.”
International evaluation of an AI system for breast cancer screening
Scott Mayer McKinney et al
Nature | Vol577 | 2January2020 - “In an independent study of six radiologists, the AI system outperformed all of the human readers: the area under the receiver operating characteristic curve (AUC-ROC) for the AI system was greater than the AUC-ROC for the average radiologist by an absolute margin of 11.5%. We ran a simulation in which the AI system participated in the double-reading process that is used in the UK, and found that the AI system maintained non-inferior performance and reduced the workload of the second reader by 88%. This robust assessment of the AI system paves the way for clinical trials to improve the accuracy and efficiency of breast cancer screening.”
International evaluation of an AI system for breast cancer screening
Scott Mayer McKinney et al
Nature | Vol577 | 2January2020 - "The optimal use of the AI system within clinical workflows remains to be determined. The specificity advantage exhibited by the system suggests that it could help to reduce recall rates and unnecessary biopsies. The improvement in sensitivity exhibited in the US data shows that the AI system may be capable of detecting cancers earlier than the standard of care. An analysis of the localization performance of the AI system suggests it holds early promise for flagging suspicious regions for review by experts. Notably, the additional cancers identified by the AI system tended to be invasive rather than in situ disease.”
International evaluation of an AI system for breast cancer screening
Scott Mayer McKinney et al
Nature | Vol577 | 2January2020 - "Beyond improving reader performance, the technology described here may have a number of other clinical applications. Through simulation, we suggest how the system could obviate the need for double reading in 88% of UK screening cases, while maintaining a similar level of accuracy to the standard protocol. We also explore how high-confidence operating points can be used to triage high-risk cases and dismiss low-risk cases. These analyses highlight the potential of this technology to deliver screening results in a sustainable manner despite workforce shortages in countries such as the UK . Prospective clinical studies will be required to understand the full extent to which this technology can benefit patient care.”
International evaluation of an AI system for breast cancer screening
Scott Mayer McKinney et al
Nature | Vol577 | 2January2020
- "We present a deep convolutional neural network for breast cancer screening exam classification, trained and evaluated on over 200,000 exams (over 1,000,000 images). Our network achieves an AUC of 0.895 in predicting the presence of cancer in the breast, when tested on the screening population. We attribute the high accuracy to a few technical advances.”
Deep Neural Networks Improve Radiologists’ Performance in Breast Cancer Screening
Nan Wu et al.
IEEE - “We attribute the high accuracy to a few technical advances. (i) Our network’s novel two-stage architecture and training procedure, which allows us to use a high-capacity patch-level network to learn from pixel-level labels alongside a network learning from macroscopic breast-level labels. (ii) A custom ResNet-based network used as a building block of our model, whose balance of depth and width is optimized for high-resolution medical images. (iii) Pretraining the network on screening BI-RADS classification, a related task with more noisy labels. (iv) Combining multiple input views in an optimal way among a number of possible choices.”
Deep Neural Networks Improve Radiologists’ Performance in Breast Cancer Screening
Nan Wu et al.
IEEE 
- “To validate our model, we conducted a reader study with 14 readers, each reading 720 screening mammogram exams, and show that our model is as accurate as experienced radiologists when presented with the same data. We also show that a hybrid model, averaging the probability of malignancy predicted by a radiologist with a prediction of our neural network, is more accurate than either of the two separately.”
Deep Neural Networks Improve Radiologists’ Performance in Breast Cancer Screening
Nan Wu et al.
IEEE - "Furthermore, the task we considered in this work, predicting whether the patient had a visible cancer at the time of the screening mammography exam, is the simplest possible among many tasks of interest. In addition to testing the utility of this model in real-time reading of screening mammograms, a clear next step would be predicting the development of breast cancer in the future–before it is even visible to a trained human eye.”
Deep Neural Networks Improve Radiologists’ Performance in Breast Cancer Screening
Nan Wu et al.
IEEE
- Summary
A deep learning algorithm was used to assess mammographic breast density at the level of an experienced mammographer during routine clinical practice.
Implications for Patient Care:
* A deep learning algorithm was used to reliably and accurately assess mammographic breast density in a large clinical practice.
* Given the high level of agreement between the deep learning algorithm and experienced mammographers, this algorithm has the potential to standardize and automate routine breast density assessment.
Mammographic Breast Density Assessment Using Deep Learning: Clinical Implementation
Lehman CD et al.
Radiology 2019; 290:52–58 - “Inconsistency in density assessment of mammograms has been widely recognized for the potential to cause patient anxiety and result in unnecessary procedures. To address this issue, we developed a DL model to assess mammographic breast density that was trained by using the assessments of experienced breast imagers. Our DL model was deployed in the mammography clinic to assess performance and acceptance in a large academic breast imaging practice. In this setting, the DL model density assessment was accepted as the final reading in 90% of mammograms by an experienced breast imager.”
Mammographic Breast Density Assessment Using Deep Learning: Clinical Implementation
Lehman CD et al.
Radiology 2019; 290:52–58 - “Also, this model was trained on mammograms at one academic center that used mammography units from one vendor (Hologic), and further testing on diverse mammograms acquired with machines from multiple vendors and from different institutions is needed. Finally, during the clinical implementation of our project, acceptance of the DL density assessment was measured in an unblinded manner."
Mammographic Breast Density Assessment Using Deep Learning: Clinical Implementation
Lehman CD et al.
Radiology 2019; 290:52–58 - In summary, we present an analysis of clinical implementation of a DL model used to assess breast density in women undergoing screening digital mammography. Our DL model provides efficient and reliable density assessments, both at the patient level and at the population level, and it is designed to be widely available, simple to use, and cost effective. It can be used to measure breast density in a diverse set of patients, without limitations based on prior surgery or other breast interventions. Our tool can potentially address concerns for current breast density legislation, and it can help providers supply more accurate information to patients and help health systems optimize the use of supplemental screening resources. To this end, we have made our tool publicly available for research use at http://learningtocure. csail.mit.edu.
Mammographic Breast Density Assessment Using Deep Learning: Clinical Implementation
Lehman CD et al.
Radiology 2019; 290:52–58 - “Our tool can potentially address concerns for current breast density legislation, and it can help providers supply more accurate information to patients and help health systems optimize the use of supplemental screening resources. To this end, we have made our tool publicly available for research use at http://learningtocure. csail.mit.edu.
Mammographic Breast Density Assessment Using Deep Learning: Clinical Implementation
Lehman CD et al.
Radiology 2019; 290:52–58