• Logo
  • HamaraJournals

Applications of Artificial Intelligence and Machine Learning in Diagnosis and Prognosis of COVID-19 infection: A systematic review

Mahdieh Montazeri, Ali Afraz, Mitra Montazeri, Sadegh Nejatzadeh, Fatemeh Rahimi, Mohsen Taherian, Mohadeseh Montazeri, Leila Ahmadian



Introduction: Our aim in this study was to summarize information on the use of intelligent models for predicting and diagnosing the Coronavirus disease 2019 (COVID-19) to help early and timely diagnosis of the disease.

Material and Methods: A systematic literature search included articles published until 20 April 2020 in PubMed, Web of Science, IEEE, ProQuest, Scopus, bioRxiv, and medRxiv databases. The search strategy consisted of two groups of keywords: A) Novel coronavirus, B) Machine learning. Two reviewers independently assessed original papers to determine eligibility for inclusion in this review. Studies were critically reviewed for risk of bias using prediction model risk of bias assessment tool.

Results: We gathered 1650 articles through database searches. After the full-text assessment 31 articles were included. Neural networks and deep neural network variants were the most popular machine learning type. Of the five models that authors claimed were externally validated, we considered external validation only for four of them. Area under the curve (AUC) in internal validation of prognostic models varied from .94 to .97. AUC in diagnostic models varied from 0.84 to 0.99, and AUC in external validation of diagnostic models varied from 0.73 to 0.94. Our analysis finds all but two studies have a high risk of bias due to various reasons like a low number of participants and lack of external validation.

Conclusion: Diagnostic and prognostic models for COVID-19 show good to excellent discriminative performance. However, these models are at high risk of bias because of various reasons like a low number of participants and lack of external validation. Future studies should address these concerns. Sharing data and experiences for the development, validation, and updating of COVID-19 related prediction models is needed.



World Health Organization. Coronavirus disease (COVID-2019) situation reports 2020 [Internet]. 2020 [cited: 27 Jul 2021]. Available from: https://www.who.int/emergencies/diseases/novel-coronavirus-2019

Sohrabi C, Alsafi Z, O'Neill N, Khan M, Kerwan A, Al-Jabir A, et al. World Health Organization declares global emergency: A review of the 2019 novel coronavirus (COVID-19). Int J Surg. 2020; 76: 71-6. PMID: 32112977 DOI: 10.1016/j.ijsu.2020.02.034

World Health Organization. Coronavirus disease (COVID-19) pandemic 2020 [Internet]. 2020 [cited: 27 Jul 2021]. Available from: https://www.who.int/emergencies/diseases/novel-coronavirus-2019/events-as-they-happen

Arabi YM, Murthy S, Webb S. COVID-19: A novel coronavirus and a novel challenge for critical care. Intensive Care Med. 2020; 46(5): 833-6. PMID: 32125458 DOI: 10.1007/s00134-020-05955-1

Grasselli G, Pesenti A, Cecconi M. Critical care utilization for the COVID-19 outbreak in Lombardy, Italy: Early experience and forecast during an emergency response. JAMA. 2020; 323(16): 1545-6. PMID: 32167538 DOI: 10.1001/jama.2020.4031

Xie J, Tong Z, Guan X, Du B, Qiu H, Slutsky AS. Critical care crisis and some recommendations during the COVID-19 epidemic in China. Intensive Care Med. 2020; 46(5): 837-40. PMID: 32123994 DOI: 10.1007/s00134-020-05979-7

Chung M, Bernheim A, Mei X, Zhang N, Huang M, Zeng X, et al. CT imaging features of 2019 novel coronavirus (2019-nCoV). Radiology. 2020; 295(1): 202-7. PMID: 32017661 DOI: 10.1148/radiol.2020200230

Yang X, Yu Y, Xu J, Shu H, Liu H, Wu Y, et al. Clinical course and outcomes of critically ill patients with SARS-CoV-2 pneumonia in Wuhan, China: A single-centered, retrospective, observational study. Lancet Respir Med. 2020; 8(5): 475-81. PMID: 32105632 DOI: 10.1016/S2213-2600(20)30079-5

Xia C, Li X, Wang X, Kong B, Chen Y, Yin Y, et al. A multi-modality network for cardiomyopathy death risk prediction with CMR images and clinical information. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2019.

Kong B, Wang X, Bai J, Lu Y, Gao F, Cao K, et al. Learning tree-structured representation for 3D coronary artery segmentation. Comput Med Imaging Graph. 2020; 80: 101688. PMID: 31926366 DOI: 10.1016/j.compmedimag.2019.101688

Zaharchuk G, Gong E, Wintermark M, Rubin D, Langlotz C. Deep learning in neuroradiology. AJNR Am J Neuroradiol. 2018; 39(10): 1776-84. PMID: 29419402 DOI: 10.3174/ajnr.A5543

Yuan J, Liao H, Luo R, Luo J. Automatic radiology report generation based on multi-view image fusion and medical concept enrichment. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2019.

Kermany DS, Goldbaum M, Cai W, Valentim CC, Liang H, Baxter SL, et al. Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell. 2018; 172(5): 1122-31. PMID: 29474911 DOI: 10.1016/j.cell.2018.02.010

Rajaraman S, Candemir S, Kim I, Thoma G, Antani S. Visualization and interpretation of convolutional neural network predictions in detecting pneumonia in pediatric chest radiographs. Appl Sci (Basel). 2018; 8(10): 1715. PMID: 32457819 DOI: 10.3390/app8101715

Depeursinge A, Chin AS, Leung AN, Terrone D, Bristow M, Rosen G, et al. Automated classification of usual interstitial pneumonia using regional volumetric texture analysis in high-resolution CT. Invest Radiol. 2015; 50(4): 261-7. PMID: 25551822 DOI: 10.1097/RLI.0000000000000127

Anthimopoulos M, Christodoulidis S, Ebner L, Christe A, Mougiakakou S. Lung pattern classification for interstitial lung diseases using a deep convolutional neural network. IEEE Trans Med Imaging. 2016; 35(5): 1207-16. PMID: 26955021 DOI: 10.1109/TMI.2016.2535865

Pal R, Sekh AA, Kar S, Prasad DK. Neural network based country wise risk prediction of COVID-19. Applied Sciences. 2020; 10(18): 6448.

Carr D. Sharing research data and findings relevant to the novel coronavirus (COVID-19) outbreak [Internet]. 2020 [cited: 17 Jul 2021]. Available from: https://wellcome.org/press-release/sharing-research-data-and-findings-relevant-novel-coronavirus-ncov-outbreak

Stewart LA, Clarke M, Rovers M, Riley RD, Simmonds M, Stewart G, et al. Preferred reporting items for a systematic review and meta-analysis of individual participant data: The PRISMA-IPD statement. JAMA. 2015; 313(16): 1657-65. PMID: 25919529 DOI: 10.1001/jama.2015.3656

Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan: A web and mobile app for systematic reviews. Syst Rev. 2016; 5(1): 210. PMID: 27919275 DOI: 10.1186/s13643-016-0384-4

Moons KG, de Groot JA, Bouwmeester W, Vergouwe Y, Mallett S, Altman DG, et al. Critical appraisal and data extraction for systematic reviews of prediction modelling studies: The CHARMS checklist. PLoS Med. 2014; 11(10): e1001744. PMID: 25314315 DOI: 10.1371/journal.pmed.1001744

Moons KG, Wolff RF, Riley RD, Whiting PF, Westwood M, Collins GS, et al. PROBAST: A tool to assess risk of bias and applicability of prediction model studies: Explanation and elaboration. Annals of Internal Medicine. 2019; 170: W1-33.

Wynants L, van Calster B, Bonten MM, Collins GS, Debray TP, de Vos M, et al. Prediction models for diagnosis and prognosis of covid-19 infection: Systematic review and critical appraisal. BMJ. 2020; 369: m1328. PMID: 32265220 DOI: 10.1136/bmj.m1328

Diaz-Quijano FA, da Silva JMN, Ganem F, Oliveira S, Vesga-Varela AL, Croda J. A model to predict SARS-CoV-2 infection based on the first three-month surveillance data in Brazil. Trop Med Int Health. 2020; 25(11): 1385-94. PMID: 32790891 DOI: 10.1111/tmi.13476

de Moraes Batista AF, Miraglia JL, Donato THR, Chiavegatto Filho ADP. COVID-19 diagnosis prediction in emergency care patients: A machine learning approach [Internet]. 2020 [cited: 14 Apr 2020]. Available from: http://medrxiv.org/lookup/doi/10.1101/2020.04.04.20052092

Ozturk S, Ozkaya U, Barstugan M. Classification of coronavirus images using Shrunken features. Int J Imaging Syst Technol. 2020; Online ahead of print. PMID: 32904960 DOI: 10.1002/ima.22469

Mahdy LN, Hassanien AE, Ezzat KA, Elmousalami HH, Aboul Ella H. Automatic X-ray COVID-19 lung image classification system based on multi-level thresholding and support vector machine [Internet]. 2020 [cited: 14 Apr 2020]. Available from: https://www.medrxiv.org/content/10.1101/2020.03.30.20047787v1

Bukhari SUK, Bukhari SSK, Syed A, Shah SSH. The diagnostic evaluation of convolutional neural network (CNN) for the assessment of chest X-ray of patients infected with COVID-19 [Internet]. 2020 [cited: 14 Apr 2020]. Available from: http://medrxiv.org/lookup/doi/10.1101/2020.03.26.20044610

Fu M, Yi S-L, Zeng Y, Ye F, Li Y, Dong X, et al. Deep learning-based recognizing COVID-19 and other common infectious diseases of the lung by chest CT scan images [Internet]. 2020 [cited: 14 Apr 2020]. Available from: http://medrxiv.org/lookup/doi/10.1101/2020.03.28.20046045

Zhou M, Chen Y, Wang D, Xu Y, Yao W, Huang J, et al. Improved deep learning model for differentiating novel coronavirus pneumonia and influenza pneumonia. Ann Transl Med. 2021; 9(2): 111. PMID: 33569413 DOI: 10.21037/atm-20-5328

Castiglioni I, Ippolito D, Interlenghi M, Monti CB, Salvatore C, Schiaffino S, et al. Artificial intelligence applied on chest X-ray can aid in the diagnosis of COVID-19 infection: A first experience from Lombardy, Italy. Eur Radiol Exp. 2021; 5(1): 7. PMID: 33527198 DOI: 10.1186/s41747-020-00203-z

Wang S, Kang B, Ma J, Zeng X, Xiao M, Guo J, et al. A deep learning algorithm using CT images to screen for corona virus disease (COVID-19). Eur Radiol. 2021; 31(8): 6096-104. PMID: 33629156 DOI: 10.1007/s00330-021-07715-1

Song Y, Zheng S, Li L, Zhang X, Zhang X, Huang Z, et al. Deep learning enables accurate diagnosis of novel coronavirus (COVID-19) with CT images. IEEE/ACM Trans Comput Biol Bioinform. 2021; Online ahead of print. PMID: 33705321 DOI: 10.1109/TCBB.2021.3065361

Zheng C, Deng X, Fu Q, Zhou Q, Feng J, Ma H, et al. A weakly-supervised framework for COVID-19 classification and lesion localization from chest CT. IEEE Transactions on Medical Imaging. 2020; 39(8): 2615-25.

Chen J, Wu L, Zhang J, Zhang L, Gong D, Zhao Y, et al. Deep learning-based model for detecting 2019 novel coronavirus pneumonia on high-resolution computed tomography: A prospective study. Sci Rep. 2020; 10(1): 19196. PMID: 33154542 DOI: 10.1038/s41598-020-76282-0

Meng Z, Wang M, Song H, Guo S, Zhou Y, Li W, et al. Development and utilization of an intelligent application for aiding COVID-19 diagnosis [Internet]. 2020 [cited: 14 Apr 2020]. Available from: http://medrxiv.org/lookup/doi/10.1101/2020.03.18.20035816

Feng C, Huang Z, Wang L, Chen X, Zhai Y, Zhu F, et al. A novel triage tool of artificial intelligence assisted diagnosis aid system for suspected COVID-19 pneumonia in fever clinics. Ann Transl Med. 2021; 9(3): 201. PMID: 33708828 DOI: 10.21037/atm-20-3073

Jin S, Wang B, Xu H, Luo C, Wei L, Zhao W, et al. AI-assisted CT imaging analysis for COVID-19 screening: Building and deploying a medical AI system in four weeks. Appl Soft Comput. 2021; 98: 106897. PMID: 33199977 DOI: 10.1016/j.asoc.2020.106897

Li L, Qin L, Xu Z, Yin Y, Wang X, Kong B, et al. Artificial intelligence distinguishes COVID-19 from community acquired pneumonia on chest CT. Radiology. 2020; 200905. PMID: 32191588 DOI: 10.1148/radiol.2020200905

Maghdid HS, Asaad AT, Ghafoor KZ, Sadiq AS, Khan MK. Diagnosing COVID-19 pneumonia from X-ray and CT images using deep learning and transfer learning algorithms [Internet]. 2020 [cited: 15 Apr 2020]. Available from: http://arxiv.org/abs/2004.00038

Farooq M, Hafeez A. COVID-ResNet: A deep learning framework for screening of COVID19 from radiographs [Internet]. 2020 [cited: 15 Apr 2020]. Available from: http://arxiv.org/abs/2003.14395

Wang L, Wong A. COVID-Net: A tailored deep convolutional neural network design for detection of COVID-19 cases from chest radiography images. Scientific Reports. 2020; 10: 19549.

Zhang J, Xie Y, Li Y, Shen C, Xia Y. COVID-19 screening on chest X-ray images using deep learning based anomaly detection [Internet]. 2020 [cited: 15 Apr 2020]. Available from: http://arxiv.org/abs/2003.12338

Chowdhury MEH, Rahman T, Khandakar A, Mazhar R, Kadir MA, Mahbub ZB, et al. Can AI help in screening viral and COVID-19 pneumonia? IEEE Access. 2020; 8: 132665-76.

Apostolopoulos ID, Bessiana T. Covid-19: Automatic detection from X-Ray images utilizing transfer learning with convolutional neural networks. Phys Eng Sci Med. 2020; 43(2): 635-40. PMID: 32524445 DOI: 10.1007/s13246-020-00865-4

Hemdan EE-D, Shouman MA, Karar ME. COVIDX-Net: A framework of deep learning classifiers to diagnose COVID-19 in X-ray images [Internet]. 2020 [cited: 15 Apr 2020]. Available from: http://arxiv.org/abs/2003.11055

Narin A, Kaya C, Pamuk Z. Automatic detection of coronavirus disease (COVID-19) using X-ray images and deep convolutional neural networks. Pattern Anal Appl. 2021; Online ahead of print. PMID: 33994847 DOI: 10.1007/s10044-021-00984-y

Barstugan M, Ozkaya U, Ozturk S. Coronavirus (COVID-19) classification using CT images by machine learning methods [Internet]. 2020 [cited: 15 Apr 2020]. Available from: http://arxiv.org/abs/2003.09424

Abbas A, Abdelsamea M, Gaber M. Classification of COVID-19 in chest X-ray images using DeTraC deep convolutional neural network. Appl Intell. 2021; 51: 854–64.

Shi F, Xia L, Shan F, Wu D, Wei Y, Yuan H, et al. Large-scale screening of COVID-19 from community acquired pneumonia using infection size-aware classification. Phys Med Biol. 2021; Online ahead of print. PMID: 33607630 DOI: 10.1088/1361-6560/abe838

Xu X, Jiang X, Ma C, Du P, Li X, Lv S, et al. A deep learning system to screen coronavirus disease 2019 pneumonia. Engineering (Beijing). 2020; 6(10): 1122-9. PMID: 32837749 DOI: 10.1016/j.eng.2020.04.010

Pourhomayoun M, Shakibi M. Predicting mortality risk in patients with COVID-19 using artificial intelligence to help medical decision-making. Smart Health (Amst). 2021; 20: 100178. PMID: 33521226 DOI: 10.1016/j.smhl.2020.100178

Sarkar J, Chakrabarti P. A machine learning model reveals older age and delayed hospitalization as predictors of mortality in patients with COVID-19 [Internet]. 2020 [cited: 14 Apr 2020]. Available from: http://medrxiv.org/lookup/doi/10.1101/2020.03.25.20043331

Fang C, Bai X, Zhou Y, Bai S, Liu Z, Chen Q, et al. Deep learning for predicting COVID-19 malignant progression. Medical Image Analysis. 2021; 72: 102096.

Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): The TRIPOD statement. BMJ. 2015; 350: g7594. PMID: 25569120 DOI: 10.1136/bmj.g7594

van Calster B, McLernon D, van Smeden M, Wynants L, Steyerberg E. Calibration: The achilles heel of predictive analytics. BMC Med. 2019; 17(1): 230. PMID: 31842878 DOI: 10.1186/s12916-019-1466-7

Steyerberg EW. Clinical prediction models: Practical approach to development, validation, and updating. SpringerLink; 2019.

Steyerberg EW. Evaluation of clinical usefulness. In: Steyerberg EW. Clinical rediction models: Practical approach to development, validation, and updating. SpringerLink; 2019.

Riley RD, Ensor J, Snell KI, Debray TP, Altman DG, Moons KG, et al. External validation of clinical prediction models using big datasets from e-health records or IPD meta-analysis: Opportunities and challenges. BMJ. 2016; 353: i3140. PMID: 27334381 DOI: 10.1136/bmj.i3140

Wynants L, Kent D, Timmerman D, Lundquist C, van Calster B. Untapped potential of multicenter studies: a review of cardiovascular risk prediction models revealed inappropriate analyses and wide variation in reporting. Diagn Progn Res. 2019; 3: 6. PMID: 31093576 DOI: 10.1186/s41512-019-0046-9

Cui Z, Gong G. The effect of machine learning regression algorithms and sample size on individualized behavioral prediction with functional connectivity features. Neuroimage. 2018; 178: 622-37. PMID: 29870817 DOI: 10.1016/j.neuroimage.2018.06.001

Chu C, Hsu A-L, Chou K-H, Bandettini P, Lin C, Alzheimer's Disease Neuroimaging Initiative. Does feature selection improve classification accuracy? Impact of sample size and feature selection on classification using anatomical magnetic resonance images. Neuroimage. 2012; 60(1): 59-70. PMID: 22166797 DOI: 10.1016/j.neuroimage.2011.11.066

Arbabshirani MR, Plis S, Sui J, Calhoun VD. Single subject prediction of brain disorders in neuroimaging: Promises and pitfalls. Neuroimage. 2017; 145: 137-65. PMID: 27012503 DOI: 10.1016/j.neuroimage.2016.02.079

Riley RD, Ensor J, Snell KI, Harrell FE, Martin GP, Reitsma JB, et al. Calculating the sample size required for developing a clinical prediction model. BMJ. 2020; 368: m441. PMID: 32188600 DOI: 10.1136/bmj.m441

Huang J, Ling CX. Using AUC and accuracy in evaluating learning algorithms. IEEE Transactions on Knowledge and Data Engineering. 2005;17(3): 299-310.

Jeni LA, Cohn JF, de la Torre F. Facing imbalanced data: Recommendations for the use of performance metrics. Humaine Association Conference on Affective Computing and Intelligent Interaction. IEEE; 2013.

DOI: http://dx.doi.org/10.30699/fhi.v10i1.321


  • There are currently no refbacks.