Two-loop validation of neural network-based pneumothorax segmentation under class imbalance

The purpose of the research is to develop a clinically oriented methodology for training, validating, and selecting a neural network model for pneumothorax segmentation on chest radiographs under severe class imbalance and the need for controlled limitation of false positive alerts in clinical decision support systems.

Methods. We propose a two-loop segmentation quality assessment framework comprising a technical loop for monitoring training convergence using the loss function values and raw metrics, and a clinical loop for evaluating binarized masks after a fixed post-processing pipeline. For each training epoch, a binarization threshold is swept; an operating point is deemed acceptable if predefined clinical constraints are satisfied. The final epoch is then selected as the one that maximizes Dice index on positive cases among all acceptable solutions. The segmenter is implemented using a UNet++ architecture and trained via a two-stage scheme with increasing spatial resolution, using a hybrid loss function.

Results. On clinically representative data SIIM-ACR development set, configuration selected by the protocol achieved Dice index on positive cases = 0,753 (95% CI 0,734–0,771) with false positive rate = 0,177 (95% CI 0,158– 0,197) and image-level sensitivity = 0,948, and substantially outperformed the baseline configuration.

Conclusion. The proposed two-loop validation methodology and constraint-based epoch selection within the admissible region provide reproducible control of the trade-off between segmentation quality on pathological images and the false alarm rate on non-pathological images, reducing the risk of selecting a suboptimal model when relying solely on minimum loss. The results support the use of the developed protocol in designing segmentation modules for decision support systems in radiological imaging tasks.

1. Tárnoki D.L., Karlinger K., Ridge C.A., Kiss F.J., Györke T., Grabczak E.M., Tárnoki Á.D. Lung imaging methods: indications, strengths and limitations. Breathe (Sheff). 2024;20(3):230127. https://doi.org/10.1183/20734735.0127-2023.

2. Dal I., Kaya H.B. Multidisciplinary Evaluation of an AI-Based Pneumothorax Detection Model: Clinical Comparison with Physicians in Edge and Cloud Environments. J. Multidiscip. Healthc. 2025;18:4099-4111. https://doi.org/10.2147/JMDH.S535405.

3. De Biase A., Sijtsema N.M., Janssen T., Hurkmans C., Brouwer C., van Ooijen P. Clinical adoption of deep learning target auto-segmentation for radiation therapy: challenges, clinical risks, and mitigation strategies. BJR Artificial Intelligence. 2024;1(1):ubae015. https://doi.org/10.1093/bjrai/ubae015.

4. Maier-Hein L., Reinke A., Godau P., et al. Metrics reloaded: recommendations for image analysis validation. Nat. Methods. 2024;21(2):195-212. https://doi.org/10.1038/s41592-023-02151-z.

5. Reinke A., Tizabi M.D., Baumgartner M., et al. Understanding metric-related pitfalls in image analysis validation. Nat. Methods. 2024;21(2):182-194. https://doi.org/10.1038/s41592-023-02150-0.

6. Mohammadi M., Mollazade K., Behroozi-Khazaei N. Under- and over-segmentation: new metrics for image segmentation accuracy measurement. Array. 2025;28:100624. https://doi.org/10.1016/j.array.2025.100624.

7. Miglioretti D.L., Abraham L., Sprague B.L., et al. Association Between False-Positive Results and Return to Screening Mammography in the Breast Cancer Surveillance Consortium Cohort. Ann. Intern. Med. 2024;177(10):1297-1307. https://doi.org/10.7326/M24-0123.

8. Büttner M., Schneider L., Krasowski A., et al. Conquering class imbalances in deep learning-based segmentation of dental radiographs with different loss functions. J. Dent. 2024;148:105063. https://doi.org/10.1016/j.jdent.2024.105063.

9. Cabezas M., Diez Y. An Analysis of Loss Functions for Heavily Imbalanced Lesion Segmentation. Sensors. 2024;24(6):1981. https://doi.org/10.3390/s24061981.

10. Reinke A., Tizabi M. D., Maier-Hein L. Advancing standards in biomedical image analysis validation: A perspective on Metrics Reloaded. Clin. Transl. Med. 2025;15(9):e70237. https://doi.org/10.1002/ctm2.70237.

11. Malyutina I.A., Kuz’min A.A., Shatalova O.V. Methods and algorithms for analyzing chest radiographs using local windows in pathology detection tasks. Prikaspiiskii zhurnal: upravlenie i vysokie tekhnologii = Caspian Journal: Management and High Technologies. 2017;(3):131-138. (In Russ.)

12. Kudryavtsev P.S., Kuz’min A.A., Savinov D.Yu., Filist S.A., Shatalova O.V. Modeling of morphological formations on chest radiographs in intelligent medical diagnostic systems. Prikaspiiskii zhurnal: upravlenie i vysokie tekhnologii = Caspian Journal: Management and High Technologies. 2017;(3):109-120. (In Russ.)

13. Dyudin M.V., Kudryavtsev P.S., Podmaster’ev K.V., Filist S.A., Shatalova O.V. Mathematical models for intelligent systems of chest radiograph classification. Izvestiya Yugo-Zapadnogo gosudarstvennogo universiteta. Serija: Upravlenie, vychislitel'naja tekhnika, informatika. Meditsinskoe priborostroenie = Proceedings of the Southwest State University. Series: Control, Computer Engineering, Information Science. Medical Instruments Engineer- ing. 2016;2(19):94-107. (In Russ.)

14. Filist S.A., Shashkova E.A., Shatalova O.V., Tomakova R.A. Analysis of biomedical images by various segmentation methods based on gradient operators. Perspektivy razvitiya informatsionnykh tekhnologii = Prospects for the Development of Information Technology. 2011;(3-1):146-150. (In Russ.)

15. Dabagov A.R., Gorbunov V.A., Filist S.A., et al. An Automated System for Classification of Radiographs of the Breast. Biomedical Engineering. 2020;53(6):425-428. https://doi.org/10.1007/s10527-020-09957-7.

16. Khatatneh K., Filist S., Shatalova O., et al. Hybrid neural networks with virtual flows in medical risk classifiers. Journal of Intelligent and Fuzzy Systems. 2022;43(1):1621– 1632. https://doi.org/10.3233/JIFS-212617.

17. Korovin E.N., Shatalova O.V., Zhilin V.V. Application of a hybrid neural network with macrolayers for the classification of cardiovascular diseases. Biomeditsinskaya radioelektronika = Biomedical Radioelectronics. 2014;(9):32-37. (In Russ.)

18. Filist S., Al-Kasasbeh R.T., Shatalova O.V., et al. Biotechnical neural network system for predicting cardiovascular health state using processing of bio-signals. International Journal of Medical Engineering and Informatics. 2024;16(4):324-349. https://doi.org/10.1504/IJMEI.2024.139884.

19. Krupchatnikov R.A., Mednikov D.A., Protasova Z.U., et al. Convolutional neural network for medical risk classifier models with synergetic channels. Izvestiya Yugo- Zapadnogo gosudarstvennogo universiteta. Serija: Upravlenie, vychislitel'naja tekhnika, informatika. Meditsinskoe priborostroenie = Proceedings of the Southwest State University. Series: Control, Computer Engineering, Information Science. Medical Instruments Engineering. 2021;11(2):25–50. (In Russ.)

20. Filist S.A., Myasnyankin M.B., Safronov R.I., Kuzmin A.A. Multimodal neural network classifier of the functional state of the respiratory system. Journal of Physics: Confer- ence Series. 2021;2094:032064. https://doi.org/10.1088/1742-6596/2094/3/032064.

21. Radhachandran A., Kinzel A., Chen J., Sant V., Patel M., Masamed R., Arnold C.W., Speier W. A multitask approach for automated detection and segmentation of thyroid nodules in ultrasound images. Comput. Biol. Med. 2024;170:107974. https://doi.org/10.1016/j.compbiomed.2024.107974.

22. Zhou Z., Siddiquee M.M.R., Tajbakhsh N., Liang J. UNet++: A Nested U-Net Architecture for Medical Image Segmentation. Deep Learn Med Image Anal Multimodal Learn Clin Decis Support. 2018;11045:3–11. https://doi.org/10.1007/978-3-030-00889-5_1.

23. Lee C.Y., Xie S., Gallagher P., Zhang Z., Tu Z. Deeply-Supervised Nets. Available at: https://arxiv.org/abs/1409.5185 (accessed 16.03.2026).

24. Haque M.I.U., Dubey A.K., Danciu I., Justice A.C., Ovchinnikova O.S., Hinkle J.D. Effect of image resolution on automated classification of chest X-rays. J. Med. Imaging (Bel- lingham). 2023;10(4):044503. https://doi.org/10.1117/1.JMI.10.4.044503.

25. Yoo S.W., Yang S., Kim J.E., et al. CACSNet for automatic robust classification and segmentation of carotid artery calcification on panoramic radiographs using a cascaded deep learning network. Sci. Rep. 2024;14:13894. https://doi.org/10.1038/s41598-024-64265-4.

26. Sim J.Z.T., Lin J., Fong Q.W., Soon A.Y.Q., Khin L.W., Balakrishnan S., Lin T., Wong S., Tan C.H. How threshold customisation affects the performance of a multiclass X-ray AI model for primary care triage: a retrospective study. BMJ Open. 2026;16(2):e111127. https://doi.org/10.1136/bmjopen-2025-111127.

27. Tabassum M., Di Ieva A., Liu S. Meta transfer learning for brain tumor segmentation using nnUNet in meningioma and metastasis cases. Sci. Rep. 2025;15:37599. https://doi.org/10.1038/s41598-025-20957-z.