Autonomous artificial intelligence for sorting the results of preventive radiological studies on the example of mammography

The purpose of research. Radiation diagnostics is central to the detection of malignant neoplasms. Recently, the implementation of screening programs has faced a number of obstacles, including staff shortages and limited funding. The introduction of artificial intelligence (AI)-based systems capable of absolutely accurate sorting of research into two categories - "normal" and "not normal", seems to be a promising solution to these problems. However, before they are widely used, it is critically important to verify their ability to guarantee the safety and high quality of the screening process. The aim of the study is to evaluate the possibility of using autonomous sorting of mammographic examination results in real clinical conditions.  

Methods. The study was carried out in 2 stages. At the first stage, 25,892 mammographic studies processed by the AI service were retrospectively analyzed. A ROC analysis of these results was carried out in order to assess the possibility of configuring the AI service for 100% sensitivity. At the prospective stage, the results of 82,372 mammograms were analyzed. All studies were processed by AI services configured for 100% sensitivity. The tasks of the AI services included the sorting of mammography results into the categories "normal" and "not normal". Next, the decisions of AI services and radiologists on categorization were compared. 

Results. According to the results of a retrospective study, when configuring the AI service for 100% sensitivity, the specificity was 39%. In the course of a prospective study, the proportion of defects (false attribution of research results to the "norm" category) was 0.08%, the specific weight of clinically significant defects in AI services was 0.02%, which is significantly lower than that of a radiologist. 

Conclusion. The use of autonomous sorting of mammographic research results in clinical practice is possible in order to optimize the diagnostic process during preventive measures, as well as under the condition of monitoring the quality of artificial intelligence technologies. Keywords: artificial intelligence, mammography, preventive examinations, radiation diagnostics. Conflict of interest: The author declares the absence of obvious and potential conflicts of interest related to the publication of this article. 

1. . Kaprin A.D. (eds.) Malignant neoplasms in Russia in 2022 (morbidity and mortality). Moscow: MNIOI im. P.A. Gercena – filial FGBU «NMIC radiologii» Minzdrava Rossii; 2023. 275 p. (In Russ.)

2. Beljaev A.M. (eds.) Methodological recommendations for implementing a populationbased screening program for malignant neoplasms of the breast among the female population. (In Russ.) Available at: http://onkokms.ru/images/doc/vracham/popsk.pdf?ysclid=m1655l7 nps854446616 (accessed 11.06.2024).

3. Lomakov S.Ju. Volumes of mammographic studies in modern conditions of providing preventive measures. Profilakticheskaya meditsina = Russian Journal of Preventive Medicine. 2020;23(4):41–44. (In Russ.)

4. Golubev N.A., Ogryzko E.V., Tyurina E.M., Shelepova E.A., Shelekhov P.V. Features of the development of the radiation diagnostics service in the Russian Federation for 2014– 2019. Sovremennye problemy zdravookhraneniya i meditsinskoi statistiki = Modern Problems of Healthcare and Medical Statistics. 2021;(2):356–376. (In Russ.) https://doi.org/10.24412/2312-2935-2021-2-356-376

5. Shelekhov P.V. Personnel situation in radiative diagnostics. Sovremennye problemy zdravookhraneniya i meditsinskoi statistiki = Modern Problems of Healthcare and Medical Statistics. 2019;(1):265–275. (In Russ.) https://doi.org/10.24411/2312-2935-2019-10018

6. Michalopoulou E., Clauser P., Gilbert F.J., Pijnappel R.M., Mann R.M., Baltzer P.A.T., Chen Y., Fallenberg E.M. A survey by the European Society of Breast Imaging on radiologists’ preferences regarding quality assurance measures of image interpretation in screening and diagnostic mammography. Eur. Radiol. 2023;33(11):8103–8111. https://doi.org/10.1007/s00330023-09973-7

7. Vasilev Yu.A, Tyrov I.A., Vladzymyrskyy A.V., Arzamasov K.M., Shulkin I.M., Kozhikhina D.D., Pestrenin L.D. Double-reading mammograms using artificial intelligence technologies: A new model of mass preventive examination organization. Digital Diagnostics. 2023;4(2):93−104. (In Russ.) https://doi.org/10.17816/DD321423

8. Vasiliev Y.A., Vladzimirsky A.V., Arzamasov K.M., Shulkin I.M., Aksenova L.E., Pestrenin L.D., Semenov S.S., Bondarchuk D.V., Smirnov I.V. The first 10,000 mammography exams performed as part of the «description and interpretation of mammography data using artificial intelligence» service. Menedzher zdravookhraneniya = Manager Zdravookhraneniya. 2023;(8):54–67. (In Russ.) https://doi.org/10.21045/1811-0185-2023-8-54-67

9. Basic recommendations for the operation of artificial intelligence services for radiation diagnostics: methodological recommendations No 54. Moscow: Nauchno-prakticheskij klinicheskij centr diagnostiki i telemedicinskih tekhnologij Departamenta zdravoohraneniya goroda Moskvy; 2022. 68 p. (In Russ.)

10. Bossuyt P.M., Reitsma J.B., Bruns D.E., Gatsonis C.A., Glasziou P.P., Irwig L., Lijmer J.G., Moher D., Rennie D., de Vet H.C., Kressel H.Y., Rifai N., Golub R.M., Altman D.G., Hooft L., Korevaar D.A., Cohen J.F. For the STARD Group. STARD 2015: An Updated List of Essential Items for Reporting Diagnostic Accuracy Studies. Radiology. 2015;(277):826–832. https://doi.org/10.1148/radiol.2015151516

11. Vladzymyrskyy A.V., Kudryavtsev N.D., Kozhikhina D.D., Shulkin I.M., Morozov S.P., Ledikhova N.V., Klyashtorniy V.G., Goncharova I.V., Novikov A.V., Vnukova O.M. Effectiveness of using artificial intelligence technologies for dual descriptions of the results of preventive lung examinations. Profilakticheskaya Meditsina = Preventative Medicine. 2022;25(7):7–15. (In Russ.) https://doi.org/10.17116/profmed2022250717

12. Bae M.S., Moon W.K., Chang J.M., Koo H.R., Kim W.H., Cho N., Yi A., Yun B.L., Lee S.H., Kim M.Y., Ryu E.B., Seo M. Breast cancer detected with screening US: reasons for nondetection at mammography. Radiology. 2014;270(2):369–377.

13. Hovda T., Larsen M., Romundstad L., Sahlberg K.K., Hofvind S. Breast cancer missed at screening; hindsight or mistakes? Eur. J. Radiol. 2023;(165):110913. https://doi.org/10.1016/j.ejrad.2023.110913

14. Ekpo E.U., Alakhras M., Brennan P. Errors in Mammography Cannot be Solved Through Technology Alone. Asian. Pac J. Cancer. Prev. 2018;19(2):291–301. https://doi.org/10.22034/APJCP.2018.19.2.291

15. Evans K.K., Birdwell R.L., Wolfe J.M. If you don't find it often, you often don't find it: why some cancers are missed in breast cancer screening. PLoS One. 2013;8(5):e64366.

16. Martiniussen M.A., Sagstad S., Larsen M., Larsen A.S.F., Hovda T., Lee C.I., Hofvind S. Screen-detected and interval breast cancer after concordant and discordant interpretations in a population based screening program using independent double reading. Eur. Radiol. 2022;32(9):5974–5985. https://doi.org/10.1007/s00330-022-08711-9

17. Palazzetti V., Guidi F., Ottaviani L., Valeri G., Baldassarre S., Giuseppetti G.M. Analysis of mammographic diagnostic errors in breast clinic. Radiol Med. 2016;121(11):828–833. https://doi.org/10.1007/s11547-016-0655-0