COVID-19 Classification Challenge¶

Welcome to the COVID-19 CT classification challenge. We invite users of grand-challenge.org to submit algorithms to analyze a CT scan of a COVID-19 suspect and assign a CO-RADS score to the scan.

Early in the COVID-19 pandemic, chest CT imaging was found useful for diagnosis and follow-up of COVID-19 patients. Although a definite diagnosis of COVID-19 requires a positive RT-PCR test, there was and still is a shortage of such tests in many parts of the world, and it takes a lot of time, up to days, before the test results are available. When sick patients go to the emergency ward of a hospital during a pandemic, clinicians need to make an immediate decision about whether or not to admit the patient to the hospital and send the patient to a COVID-19 and non-COVID-19 ward. CT has proven to be a good tool to help in this decision process. 

CT reporting systems have been proposed to translate radiological findings to standardized scores. This has been shown to improve communication between radiologists and other healthcare providers. In particular, the Dutch Radiological Society proposed the CO-RADS scoring system. Using this guideline, defined for non-contrast low-dose chest CT exams and intended to be used for patients with suspected COVID-19 infection in a moderate to high prevalence setting, every scan receives a score from very low or CO-RADS 1 up to very high or CO-RADS 5. Scans with insufficient quality, for example, because of severe breathing artifacts, receive a CO-RADS 0 score. There is also a CO-RADS 6 score that can be assigned to CT scans if a definite positive diagnosis from RT-PCR testing is available. An illustrated guide to the CO-RADS reporting system is available here. 

CO-RADS has been implemented in many hospitals in The Netherlands and other European countries. The guideline was published in Radiology and a validation study in which eight chest radiologists scored 105 scans demonstrated that the average area under the ROC curve (AUC) was 0.91 for predicting RT-PCR outcome and 0.95 relative to a clinical diagnosis (some patients are considered COVID-19 positive even though a positive RT-PCR test is not (yet) available). 

Another study in Radiology presented a deep learning system CO-RADS-AI, available on this site, that automatically assigns a CO-RADS score to a CT scan. This system also achieved an AUC of 0.95 on the same test data set as the human readers used in the previously mentioned CO-RADS guideline publication. On an external test set from a different hospital with a different CT scanner manufacturer and protocol as the data CO-RADS-AI was trained with, performance went slightly down to AUC of 0.88.

In this challenge, we invite teams to submit an algorithm to grand-challenge.org that, like CORADS-AI, processes a chest CT scan and produces a CO-RADS score from 1 to 5, indicating a very low, low, moderate, high, or very high suspicion of COVID-19. We use data from the public iCTCF database to validate each submitted algorithm.