A study led by Louis Lind Plesner, MD comparing the performance of four commercial AI CXR tools was published in the Radiology Journal, with Oxipit ChestEye among the ones tested. The study evaluated AI algorithm performance for detecting airspace disease, pneumothorax, and pleural effusion.
This retrospective study included consecutive adult patients who underwent chest radiography at one of four Danish hospitals in January 2020. Two thoracic radiologists (or three, in cases of disagreement) who had access to all previous and future imaging labeled chest radiographs independently for the reference standard.
The data set comprised 2040 patients (median age, 72 years [IQR, 58–81 years]; 1033 female), of whom 669 (32.8%) had target findings. The AI tools demonstrated areas under the receiver operating characteristic curve ranging 0.83–0.88 for airspace disease, 0.89–0.97 for pneumothorax, and 0.94–0.97 for pleural effusion. Sensitivities ranged 72%–91% for airspace disease, 63%–90% for pneumothorax, and 62%–95% for pleural effusion.
The researchers conclude that current-generation AI tools showed moderate to high sensitivity for detecting airspace disease, pneumothorax, and pleural effusion on chest radiographs.
However, they produced more false-positive findings than radiology reports, and their performance decreased for smaller-sized target findings and when multiple findings were present.
Researchers outline that sensitivity for several AI tools decreased – as with clinical radiologists – when presented with more subtle findings on chest radiographs. However, it should be stated that many mistakes made by AI tools would also be difficult or even impossible for a human reader to detect without access to additional imaging and patient history.
“AI tools were compared with clinical radiology reports that were generated by radiologists who had access to lateral chest radiographs, clinical information, and prior imaging, whereas the AI tools did not, which gives the radiologists an “unfair advantage.” – reads the research.
Oxipit ChestEye was among the AI tools evaluated in the study.
Oxipit ChestEye performed at 0.86 AUC for airspace disease, 0.97 AUC for pneumothorax and 0.94 AUC for pleural effusion.
“What is striking is how similar the performance of all compared algorithms was, with only insignificant statistical variations. The study well represents the limitations of CAD products – especially in difficult cases – where human readers have clinical context and additional clinical data available”, – says Chief Medical Officer at Oxipit Naglis Ramanauskas.
The full study titled Commercially Available Chest Radiograph AI Tools for Detecting Airspace Disease, Pneumothorax, and Pleural Effusion, can be accessed here.