Researchers from the DQBM Krauthammer Lab assessed six vision-language models for lung nodule detection using 247 chest radiographs with CT-based reference labels. The models were evaluated without task-specific training, fine-tuning or prompt optimization.
All six models showed poor overall accuracy and very low sensitivity, despite moderate to excellent specificity. Even the most sensitive model detected only 27.3% of nodules overall. The findings underline the need for careful validation before vision-language models are used for specific diagnostic tasks in chest radiography.
Publication: https://doi.org/10.2214/AJR.26.35421