Artificial Intelligence Predictive Analytics in the Management of Outpatient MRI Appointment No-Shows
AJR Am J Roentgenol . 2020 Nov;215(5):1155-1162. doi: 10.2214/AJR.19.22594. Epub 2020 Sep 9.
Le Roy Chong, Koh Tzan Tsai, Lee Lian Lee, Seck Guan Foo, Piek Chim Chang
OBJECTIVE. Outpatient appointment no-shows are a common problem. Artificial intelligence predictive analytics can potentially facilitate targeted interventions to improve efficiency. We describe a quality improvement project that uses machine learning techniques to predict and reduce outpatient MRI appointment no-shows.
MATERIALS AND METHODS. Anonymized records from 32,957 outpatient MRI appointments between 2016 and 2018 were acquired for model training and validation along with a holdout test set of 1080 records from January 2019. The overall no-show rate was 17.4%. A predictive model developed with XGBoost, a decision tree-based ensemble machine learning algorithm that uses a gradient boosting framework, was deployed after various machine learning algorithms were evaluated. The simple intervention measure of using telephone call reminders for patients with the top 25% highest risk of an appointment no-show as predicted by the model was implemented over 6 months.
RESULTS. The ROC AUC for the predictive model was 0.746 with an optimized F1 score of 0.708; at this threshold, the precision and recall were 0.606 and 0.852, respectively. The AUC for the holdout test set was 0.738 with an optimized F1 score of 0.721; at this threshold, the precision and recall were 0.605 and 0.893, respectively. The no-show rate 6 months after deployment of the predictive model was 15.9% compared with 19.3% in the preceding 12-month preintervention period, corresponding to a 17.2% improvement from the baseline no-show rate (p < 0.0001). The no-show rates of contactable and noncontactable patients in the group at high risk of appointment no-shows as predicted by the model were 17.5% and 40.3%, respectively (p < 0.0001).
CONCLUSION. Machine learning predictive analytics perform moderately well in predicting complex problems involving human behavior using a modest amount of data with basic feature engineering, and they can be incorporated into routine workflow to improve health care delivery.
Read Full Article Here: https://doi.org/10.2214/ajr.19.22594