Kiran Batra, Yin Xi, Khaled M Al-Hreish, Fernando Kay, Travis Browning, Chris Baker, Ronald M Peshock
AJR Am J Roentgenol . 2022 Jul 13. doi: 10.2214/AJR.22.27895. Online ahead of print.
Background: Artificial intelligence (AI) algorithms have shown strong performance for detection of pulmonary embolus (PE) on CT examinations performed using a dedicated protocol for PE detection. AI performance is less well studied for detecting PE on examinations ordered for reasons other than suspected PE [i.e., incidental PE (iPE)].
Objective: To assess the diagnostic performance of an AI algorithm for detection of iPE on conventional contrast-enhanced chest CT examinations.
Methods: This retrospective study included 2555 patients (mean age, 53.6±14.3 years; 1340 women, 1215 men) who underwent 3003 conventional contrast-enhanced chest CT examinations (i.e., not using CT pulmonary angiography protocols) between September 2019 and February 2020. A commercial AI algorithm was applied to the images to detect acute iPE. A vendor-supplied natural language processing algorithm was applied to the clinical reports to identify examinations interpreted as positive for iPE. For all examinations positive by the AI-based image review or by NLP-based report review, a multireader adjudication process was implemented to establish a reference standard for iPE. Images were also reviewed to identify explanations of AI misclassifications.
Results: Based on the adjudication process, the frequency of iPE was 1.3% (40/3003). AI detected 4 iPEs missed by clinical reports, and clinical reports detected 7 iPEs missed by AI. AI, compared with clinical reports, exhibited significantly lower specificity (92.7% vs 99.8%, p=.045) and PPV (86.8% vs 97.3%, p=.03), but no significant difference in sensitivity (82.5% vs 90.0%, p=.37) or NPV (99.8% vs 99.9%, p=.36). For AI, neither sensitivity nor specificity varied significantly in association with age, sex, examination location, or cancer-related clinical scenario (all p>.05). Explanations of false positives by AI included metastatic lymph nodes and pulmonary venous filling defect, and of false negatives by AI included surgically altered anatomy and small-caliber subsegmental vessel.
Conclusion: AI had high NPV and moderate PPV for iPE detection, detecting some iPEs missed by radiologists.
Clinical Impact: Potential applications of the AI tool include serving as a second reader to help detect additional iPEs or as a worklist triage tool to allow earlier iPE detection and intervention. Various explanations of AI misclassifications may provide targets for model improvement.
Read Full Article Here: https://doi.org/10.2214/ajr.22.27895