Deep Learning Exhibit

Artificial Intelligence for Early Detection of Pancreatic Cancer: Preliminary Observations and Challenges

Satomi Kawamoto, M.D.

The Russell H. Morgan Department of Radiology and Radiological Science, Department of Computer Science, Department of Pathology, and the Department of Cancer Research
Johns Hopkins University

Disclosure

Research supported by the Lustgarten Foundation

Satomi Kawamoto
Seyoun Park
Linda Chu
Alejandra Blanco
Shahab Shayesteh
Saeed Ghandili
Daniel Fadaei Fouladi
Alan Yuille
Ralph Hruban
Kenneth Kinzler
Bert Vogelstein
Elliot K. Fishman

Learning Objectives

To learn the current status of artificial intelligence for early detection of pancreatic cancer
To discuss the limitations and challenges that need to overcome to improve AI performance
To learn future directions of artificial intelligence for early detection of pancreatic cancer

Introduction

Pancreatic cancer remains the major health problem.
For pancreatic adenocarcinoma (PDAC), less than 20% of patients are eligible for surgery at the time of diagnosis.
Early detection is key for successful treatment of PDAC to increase the number of eligible patients for surgical attempt at cure.
Pancreatic neuroendocrine tumors (PNET) are less common than PDAC, but diagnosis of PNET is increasing due to increasing use of cross-sectional imaging.
We describe the preliminary observations and challenges of our project with the goal of developing deep learning algorithms to assist in interpretation of CT for early detection of pancreatic cancer.

Deep Learning Applications in Cancer Imaging

Artificial intelligence (AI) offers the opportunity to transform image interpretation from a purely qualitative and subjective task to effortless quantifiable and reproducible task.
Deep learning, a subset of AI, uses training data and multiple layers of equations to develop a mathematical model that fits the data.
Convolutional neural network (CNN), a typical representative of deep learning, has bee validated as an effective approach for radiological image classification in various studies.
Currently potential clinical applications of deep learning algorithms in automated disease detection have been focused on the detection of pulmonary nodules and on mammographic screening.

Table of Contents

Deep learning approach using CT: our preliminary experience
- Deep network prediction of normal pancreas
- Deep network prediction of pancreatic cancer
- Data collection process of normal and abnormal pancreas
- Difficult cases: false positives and false negatives
Challenges in detection of early pancreatic cancer
- Collecting training data and segmentation: normal pancreas and pancreatic cancer
- Improving AI algorithm to increase sensitivity and specificity
- Generalize applicability of our AI algorithm to more variable data
Future vision
- Integrate AI system into radiology workflow as a “second reader”

Deep network prediction of normal pancreas

Deep learning has largely relied on supervised learning approaches, i.e., large amounts of training data that composed of input images and ground truth annotations.

We collected abdominal CT data from 575 normal subjects (potential renal donors) that are used as the input for deep CNN for development of deep learning algorithms for automatic recognition of a normal pancreas.

Deep network prediction of normal pancreas

Park S, et al. Diagnostic and Interventional Imaging 2019 [Epub ahead of print]

Deep network prediction of normal pancreas

In our series using 236 normal CT data, deep network prediction for the pancreas showed high fidelity with mean Dice similarity coefficients of 87.8±3.1% and mean surface distances of 1.05 ± 0.65 mm.

Deep network prediction of normal pancreas

Wang Y, et al. http://arxiv.org/abs/1804.08414
Park S, et al. Diagnostic and Interventional Imaging 2019 [Epub ahead of print]

Deep network prediction of normal pancreas and other organs

Average DICE-Sørensen similarity coefficient (DSC, %) and average surface distances of major abdominal organs for 236 normal CT cases.
Pancreas is more difficult compared to other abdominal organs including liver, spleen, kidneys and gallbladder.
It may be related to poor boundary of pancreas from adjacent organs (e.g., duodenum, vessels) and large variation of shape compared to other organs.

Wang Y, et al. http://arxiv.org/abs/1804.08414
Park S, et al. Diagnostic and Interventional Imaging 2019 [Epub ahead of print]

Deep network prediction of pancreatic cancer

To achieve best results, we decided to manually segment pancreatic cancer for supervised learning with high-quality input data.
We collect abdominal CT data from patients with 750 PDAC and PNET, and manually segmented the tumor and other organs that are used as the input for CNN for development of deep learning algorithms.

Deep network prediction of pancreatic cancer

The proposed framework of these studies offers high sensitivity (94.1%) and specificity (98.5%), which demonstrates the potential to make a clinical impact.

Our previous studies:
- 439 CT scans (136 PDAC and 303 normal cases)
- Sensitivity of 94.1% at specificity of 98.5% in detection of PDAC cases of varying size. (Zhu Z, Fishman EK, Yuille AL, et al. Multi-scale coarse-to-fine segmentation for screening pancreatic ductal adenocarcinoma) https://arxiv.org/pdf/1807.02941.
- 456 CT scans (156 PDAC and 300 normal cases)
- Sensitivity of 80.2% at specificity of 90.2% in detecting PDAC cases of varying size. (Liu F, Fishman EK, Yuille AL, et al. Joint Shape Representation and Classification for Detecting PDAC in abdominal CT scans) https://arxiv.org/pdf/1804.10684.

RSNA News https://www.rsna.org/news/2018/april/machine-learning-benefits-radiology

Deep network prediction of pancreatic cancer: Difficult cases

Currently, tumors which are more likely difficult to detect by deep learning algorithms are
- Small tumors
- Tumors with poor contrast resolution
- Exophytic tumors
False positive cases
- Subtle attenuation difference without true tumor such as focal fat deposition
- Structures outside of the pancreas could be predicted as a pancreatic cancer
Reviewing cases of false positive and false negative can help fine tuning and improvement of algorithms.

Example of FALSE NEGATIVE case of deep network prediction of pancreatic cancer (PNET)

Small PNET in the body of the pancreas seen as a subtle enhancing lesion in arterial phase, and nearly iso-atteunating to the normal pancreas in venous phase, but visible by precontrast T1-weighted MRI.

Example of FALSE NEGATIVE case of deep network prediction of pancreatic cancer (PNET)

Example of FALSE NEGATIVE case of deep network prediction of pancreatic cancer (PNET)

Small PNET in the body of the pancreas seen as a subtle enhancing lesion in arterial phase, and not visible in venous phase.

Example of FALSE NEGATIVE case of deep network prediction of pancreatic cancer (PNET)

Example of FALSE NEGATIVE case of deep network prediction of pancreatic cancer (PNET)

Small PNET in the body of the pancreas seen as a subtle enhancing mass (yellow arrow) in arterial phase with upstream pancreatic duct dilatation (blue arrow).

Example of FALSE NEGATIVE case of deep network prediction of pancreatic cancer (PNET)

Example of FALSE POSITIVE case of deep network prediction of pancreatic cancer (PDAC)

False positive PDAC prediction in the head of the pancreas

Uneven focal fatty infiltration of the head of the pancreas might be related to false positive prediction

Example of FALSE POSITIVE case of deep network prediction of pancreatic cancer (PDAC)

Wang Y, Zhou Y, Shen W, Park S, Fishman EK, Yuille AL. Abdominal multi-organ segmentation with organ-attention networks and statistical fusion. https://arxiv.org/abs/1804.08414

Example of FALSE POSITIVE case of deep network prediction of pancreatic cancer (PDAC)

Wang Y, Zhou Y, Shen W, Park S, Fishman EK, Yuille AL. Abdominal multi-organ segmentation with organ-attention networks and statistical fusion. https://arxiv.org/abs/1804.08414

Challenges in detection of pancreatic cancer

Tumors with poor contrast resolution (isoattenuating tumors) are challenging for human radiologists but also difficult for AI.
AI can be trained important secondary signs (e.g., dilatation of pancreatic duct with abrupt termination) as an additional parameter for classification.
Deep network can be exquisitely sensitive to subtle attenuation differences. To reduce false positives, we will need to add training cases with the entire spectrum of fatty infiltration to train AI to recognize fat as a benign entity.

Challenges in obtaining large training data

We chose an initial goal of segmenting CT studies from 500 subjects without known pancreatic cancer as a controls and 500 patients with PDAC, because most deep learning algorithms need at least a few hundred cases to achieve reasonable results.
We selected 500 control subjects from our renal donor database as these subjects are presumably healthy, and had clinical follow-up to make sure that they did not have early pancreas cancers or other malignancy.

Challenges in increasing training data

Our preliminary experience was based on a single CT vender with a standardized pancreatic protocol with arterial and venous phases.
We need to include training data from different protocols, venders, and institutions, and refine the algorithm to ensure its general applicability.
To improve the performance of AI, we need to enrich the dataset with small and otherwise clinically challenging cases.

Challenges in data segmentation

There are currently no standards for image segmentation for AI applications.
To achieve best results, we decided to manually segment the pancreas in normal controls and pancreatic cancer cases as well as other abdominal structures for supervised learning with high-quality input data.
However, it is a labor intensive and time-consuming process require the expertise of radiologists and researchers with understanding cross-sectional anatomy.

Park S, et al. Diagnostic and Interventional Imaging 2019 [Epub ahead of print]
Chu L, et al. Application of deep learning to pancreatic cancer detection lessons learned from our initial experience JACR 2019;9:1338

Challenges in segmentation of pancreatic cancer

PDAC are typically hypoattenuating on postcontrast CT, and PNET are typically hyperattenuating on early postcontrast CT.
However, certain number of PDAC and PENT are isoattenuating, and difficult to segment.
Boundary of PDAC is often not clearly defined.
Careful segmentation of pancreatic cancer in correlating with both arterial and venous phase CT, as well as other imaging findings and pathology results is needed for accurate segmentation.
More than one radiologist can be engaged to review the images to have a better ground truth.

Example of challenging PNET case for segmentation

Small PNET in the body of the pancreas seen as a subtle enhancing mass seen only in arterial phase (yellow arrow), only possible with correlation with EUS finding (7 mm hypoechoic mass was detected in the anterior portion of the tail by EUS). This case was FALSE NEGATIVE by deep network prediction.

Example of challenging PNET case for segmentation

Potential advantages to segment other abdominal organs

We decided to annotate not only pancreas but other abdominal organs because

Pancreatic cancer can be associated with other organ abnormalities (e.g., pancreatic duct, biliary duct, peripancreatic vessels, duodenum or liver)
Al algorithm may help to prune out false positive predictions that overlap with other organs.
The dataset can be used as the normal dataset for a matched case-control study to detect abnormalities of other organs.

Example of advantage to segment other abdominal organs

Chu L, et al. Application of deep learning to pancreatic cancer detection lessons learned from our initial experience JACR 2019;9:1338

Future Vision

In the future, we envision that AI system for automatic pancreatic cancer detection will be integrated into the radiology workflow seamlessly as a “second reader”.
The AI system will directly receive the CT data from the PACS, automatically segment the abdominal organs, and annotate any suspicious pancreatic pathology.
These annotated cases will be sent back to the PACS for the radiologist to review.
It will increase the diagnostic confidence of the radiologist, and potentially increase the chance to identify subtle pancreatic cancer which could be otherwise missed in busy clinical practice.

Chu L, et al. Application of deep learning to pancreatic cancer detection lessons learned from our initial experience JACR 2019;9:1338

Conclusion (1)

There are currently multiple challenges for early detection of pancreatic cancer by deep learning algorithm, including collecting training data, segmentation and improving sensitivity/specificity.
However, our current deep learning algorithm offers high sensitivity (94.1%) and specificity (98.5%) based on CT data from a single CT vendor with a standardized protocol.
Training data need to be expanded to include CT data from different protocols, venders, and institutions, and AI algorithm needs to be refined to ensure general applicability of our AI algorithm.

Conclusion (2)

Our goal is to integrate our AI algorithm into the radiology workflow as a “second reader”, that can improve diagnostic confidence of radiologists and potentially increase detection of subtle early pancreatic cancer which can be otherwise missed by a radiologist in busy clinical practice.
AI integrated workflow has potential to enhance efficiency and reduce errors for radiologists, and to improve patient outcome.

References

Chu L, et al. Application of deep learning to pancreatic cancer detection lessons learned from our initial experience JACR 2019;9:1338
Park S, et al. Annotated normal CT data of the abdomen for deep learning. Diagnostic and Interventional Imaging 2019 [Epub ahead of print]
Wang Y, Zhou Y, Shen W, Park S, Fishman EK, Yuille AL. Abdominal multi-organ segmentation with organ-attention networks and statistical fusion. http://arxiv.org/abs/1804.08414
Zhu Z, Fishman EK, Yuille AL, et al. Multi-scale coarse-to-fine segmentation for screening pancreatic ductal adenocarcinoma. https://arxiv.org/pdf/1807.02941.
Liu F, Fishman EK, Yuille AL, et al. Joint Shape Representation and Classification for Detecting PDAC in abdominal CT scans. https://arxiv.org/pdf/1804.10684.

Acknowledgements

Satomi Kawamoto, M.D.
Seyoun Park, Ph.D.
Linda Chu, MD
Alejandra Blanco, M.D.
Shahab Shayesteh, M.D.
Saeed Ghandili, M.D.
Daniel Fadaei Fouladi, M.D.
Alan Yuille, Ph.D.
Ralph Hruban, M.D.
Kenneth Kinzler, Ph.D.
Bert Vogelstein, M.D.
Elliot K. Fishman, M.D.