Performance of CAD4TB artificial intelligence technology in TB screening programmes among the adult population in South Africa and Lesotho

Highlights
•CAD4TB version 7 significantly outperformed version 6.
•Both versions showed decreased performance in older individuals and people with a previous history of TB.
•Variability noted across the different X-ray hardware systems used.
•Thresholds differed across the two versions, and the various demographic and characteristic subgroups.•CAD4TB version 7 is closer to meeting the WHO Target Product Profile’s recommendation for triage test.

Abstract
There is growing evidence of the performance accuracy and potential impact of Computer-Aided Diagnosis (CAD) products in TB-burdened settings. It remains unclear, however, which factors of populations and settings can affect CAD performance. We aimed to investigate the parameters affecting the performance accuracy of the two latest versions of CAD4TB in TB screening programmes in South Africa and Lesotho.

We included participants recruited for the Lesotho National Prevalence Survey and the TB TRIAGE + ACCURACY studies, who underwent digital chest radiography and microbiological reference testing for TB. In total, 6,524 chest images were included in the analysis: 288 cases and 6,236 controls. CAD4TB versions 6 and 7 interpreted the X-ray images, and the performance of both versions was investigated. Threshold analyses were performed, as well as subgroup analyses, including age, X-ray hardware and HIV status.

CAD4TB v7 showed overall improved performance accuracy compared to v6 (p < 0.01). The area under the ROC curve was 0.833 (95 % CI 0.808–0.859) for v6 and 0.865 (95 % CI 0.842–0.889) for v7. At 90 % sensitivity, v7 showed a higher specificity of 65 % compared to the 54 % achieved by v6. Both versions showed lower performance in the older age group (≥60 years) and individuals with a previous history of TB. The threshold required to achieve the same sensitivity or specificity varies notably across the two versions.

CAD4TB performed well as a TB screening tool; however, factors such as software version, age, TB history and X-ray hardware should be considered in threshold determination and performance evaluation.