The Great Reality Check Part 3: Urolithiasis
Read the results of our new user studies – up-to-date and transparent!
Purpose:
The aim of the study was to prospectively determine the performance of a common AI assistant in urolithiasis, validated with the first read reports of radiologists specialized in emergency radiology as well as imaging and clinical follow-up.
Patients, Materials and Methods:
In late 2025, 104 patients (age: 18 to 90 years, mean: 44 years, standard deviation: ± 19 years) who had been referred to ERS Emergency Radiology Schueller, a provider of teleradiology services, for abdominal CT scans due to suspected urolithiasis, were randomly and prospectively enrolled in the study over six consecutive weeks. CT studies of these patients were evaluated by a common, commercially available AI assistant (xAID, Dover, DE, USA). Radiologists reported the CT studies without the initial knowledge of the AI results and compared the radiological with the AI findings in a second step. Gold standard were the specialists´ reports as well as imaging and clinical follow-up. In case of discrepancies between the radiologists´ and the AI assistants´ findings, CT studies were second read within 30 minutes at the latest.
Results:
Of 104 patients, 19 AI results could not be retrieved. For 85 patients, radiologists and clinical follow-up diagnosed 46 calculi of the urinary tract, which were considered by consensus to be the cause of the acute symptoms (54.1%). The AI assistant yielded 46 true positive (TP), 1 false positive (FP), 15 false negative (FN), and 23 true negative (TN) results; sensitivity 0.754; specificity 0.958; positive predictive value (PPV) 0.979; negative predictive value (NPV) 0.605. In one case, a bladder catheter was mistaken for a calculus. The calculus size of 22 FN ranged up to 4.4 mm, and in one FN it was 7 mm. The location of the FN along the urinary tract showed no statistical significance.
Discussion:
The AI assistant yielded 15 out of 85 FP (27%), which suggests that the software company, likely aware of the challenging description of calculi equal to or less than 4 mm, accepts FN in favor of specificity. At least phleboliths frequently found in the pelvis were not misinterpreted, and the FP rate is comparable to the results we published on this platform on March 28, 2025 (compare https://www.radailogy.com/detect-kidney-stones-quickly-and-reliably/). In comparison to the manufacturer’s publications (see also: De Perrot T et al., Eur Radiol 2019), the remaining statistical data deviate to the disadvantage of our current study. Our study shows that the AI assistant for urolithiasis can be successfully used for stones from approximately 4.5 mm, also in teleradiology with the continued need for double-checking by experienced emergency radiologists.
Gerd Schueller and the Radailogy Team