Intelligent Image Processing System Obtained from Unmanned Aerial Vehicles

The purpose of research. Timely detection of the fire source at the stage of its development allows to reduce both material and human losses. Therefore, the purpose of the research was to develop models, methods and algorithms for controlling the fire and medical and environmental safety monitoring system, ensuring an increase in its effectiveness through the analysis of video data from unmanned aerial vehicles.

Methods. The method of classifying aerial photographs of a video sequence when monitoring a fire situation involves their decomposition into rectangular segments of a given size and assigning them to one of three classes: smoke, flame, indifferent class. "Strong" and "weak" classifiers are used to classify segments. The Walsh-Hadamard transform was used to generate descriptors for "weak" classifiers. Descriptors are calculated for three "weak" classifiers. First, the Walsh-Hadamard transform is calculated for the window of the entire segment and its spectral coefficients are used for the first "weak" classifier. Then descriptors are calculated for two windows whose sizes are two and four times smaller than the size of the original window.

Results. The classifier consists of three independently trained neural networks - "weak" classifiers. A simple ensemble averaging unit is used to combine the output data of neural networks. The software for classification of aerial photographs has been developed, which allows to form a database of segments of the "smoke" and "flame" classes, to determine the two-dimensional Walsh-Hadamard spectrum of aerial photograph segments, to train fully connected neural networks and to conduct research analysis to study the relevance of two-dimensional spectral coefficients.

Conclusion. Experimental studies on the classification of video data containing flame and smoke showed an average smoke detection accuracy of 86%, and flame detection of 89,5%. Type II errors averaged 13% for smoke detection and 4,5% for flame detection. To set up and check the classifiers, we used real data from surveillance cameras in open spaces.

1. Chunyu Yu, Zhibin Mei, Xi Zhang. A Real-time Video Fire Flame and Smoke Detection Algorithm. Procedia Engineering, 2013, vol. 62, pp. 891-898.

2. Gubbi J., Marusic S., Palaniswami M. Smoke detection in video using wavelets and support vector machines. Fire Safety Journal, 2009, vol. 44, pp. 1-6.

3. Rakkammal S., Surendar R. Forest Fire Detection and Protection Based on Convolutional Neural Network Using Deep learning Models. Available at: https://ijcrt.org/papers/IJCRT_195726.pdf (accessed 02.08.2022)

4. Zhang Li, DiDi Wu. Forest fire recognition based on dynamic feature similarity of multiview images. Available at: https://www.preprints.org/manuscript/202104.0736/v2/download. (accessed 11.08.2022)

5. Pu Li, Wangda Zhao. Image fire detection algorithms based on convolutional neural networks. Case Studies in Thermal Engineering, 2020, no. 19, pp. 1-11.

6. Khan Muhammad, Jamil Ahmad, Irfan Mehmood, Seungmin Rho, Sung Wook Baik. Convolutional Neural Networks based Fire Detection in Surveillance Videos. IEEE Access, 2018, vol. 6, pp. 18174-18183.

7. Chongyuan Tao, Jian Zhang, Pan Wang. Smoke Detection Based on Deep Convolutional Neural Networks. International Conference on Industrial Informatics - Computing Technology, Intelligent Technology, Industrial Information Integration. China, 2016.

8. Qeşmeci D., Gullu M. K. Phase-correlation-based hyperspectral image classification using multiple class representatives obtained with k-means clustering. International Journal of Remote Sensing, 2009, no. 30 (14), pp. 3827-3834.

9. Borges J. S., Bioucas-Dias J. M., Marcal A. R. S. Bayesian Hyperspectral Image Segmentation With Discriminative Class Learning. IEEE Transactions on Geoscience and Remote Sensing, 2011, vol. 49, no. 6, pp. 2151-2164. https://doi.org/10.1109/TGRS.2010.2097268

10. Prabhpreet Kaur, Gurvinder Singh, Parminder Kaur. Intellectual detection and validation of automated mammogram breast cancer images by multi-class SVM using deep learning classification. Informatics in Medicine Unlocked, 2019, vol. 16, p. 100151.

11. Filist S., Alkasasbe R. T., Shatalova O., Aikeeva A., Korenevskii N., Shakadan A., eds. Razrabotka neirosetevoi modeli dlya prognozirovaniya sostoyaniya serdtsa i serdechno-sosudistoi sistemy s ispol'zovaniem bioelektricheskoi obrabotki signalov [Development of a neural network model for predicting the state of the heart and cardiovascular system using bioelectric signal processing]. Vychislitel'nye metody Biomech Biomed Engin = Computational Methods of Biomech Biomed Engine, 2021, pp. 1-14.

12. Shatalova O., Filist S., Korenevskaya N., Al'kasasbekh R. T., Shakadan A., Protasova Z., eds. Primenenie nechetkoi neirosetevoi modeli i analiza toka-napryazheniya biologicheski aktivnykh tochek dlya prognozirovaniya posleoperatsionnykh riskov [Application of fuzzy neural network model and current-voltage analysis of biologically active points to predict postoperative risks]. Vychislitel'nye metody Biomech Biomed Engin = Computational Methods of Biomech Biomed Engine, 2021, no. 24 (13), pp. 1504-1516.

13. Dabagov A. R., Gorbunov V. A., Filist S. A., Malyutina I. A., Kondrashov D. S. Avtomatizirovannaya sistema klassifikatsii rentgenogramm molochnoi zhelezy [Automated classification system of breast radiographs]. Biomeditsinskaya inzheneriya = Biomedical Engineering, 2020, vol. 53, no. 6, pp. 425-428. https://doi.org/10.1007%2Fs10527-020-09957-7

14. Bilkent database. Available at: http://signal.ee.bilkent.edu.tr/VisiFire/Demo/Fire-Clips. (accessed 10.08.2022)

15. Bakhshiev A. V., Gundelakh F. V. Application the Spiking Neuron Model with Structural Adaptation to Describe Neuromorphic Systems. Procedia Computer Science, 2017, vol. 103, pp. 190-197.

16. Tomakova R. A., Filist S. A., Durakov I. V. Programmnoe obespechenie dlya avtomaticheskoi klassifikatsii rentgenogramm grudnoi kletki na osnove gibridnykh klassifikatorov [Software for automatic classification of chest radiographs based on hybrid classifiers]. Ekologiya cheloveka = Human Ecology, 2018, no. 6, pp. 59-64.

17. Mici L., Parisi G. I., Wermter S. A self-organizing neural network architecture for learning human-object interactions. Neurocomputing, 2018, vol. 307, pp. 14-24.

18. Filist S. A., Tomakova R. A., Degtyarev S. V., Rybochkin A. F. Gibridnye intellektual'nye modeli dlya segmentatsii rentgenovskikh izobrazhenii grudnoi kletki [Hybrid intelligent models for segmentation of chest X-ray images]. Biomeditsinskaya inzheneriya = Biomedical Engineering, 2018, vol. 51, no. 5, pp. 358-363. https://doi.org/article/10.1007/ s10527-018-9748-5

19. Graphics Recognition Techniques Handbook of Document Image Processing and Recognition, 2014, Josep Llados, Marçal Rusinol.

20. Erdi Qalli, Ecem Sogancioglu, Bram van Ginneken, Kicky G. van Leeuwen, Keelin Murphy. Deep learning for chest X-ray analysis: A survey. Medical Image Analysis, 2021, vol. 72, p. 102125.

21. Filist S. A., Ali Kassim K. D., Kuz'min A. A., Shatalova O. V., Alyab'ev E. A. Formirovanie prostranstva priznakov dlya zadach klassifikatsii slozhnykh strukturirovannykh izobrazhenii na osnove spektral'nykh okon i struktur neironnykh setei [Formation of a feature space for classification problems of complex structured images based on spectral windows and neural network structures]. Izvestiya Yugo-Zapadnogo gosudarstvennogo universiteta = Proceedings of the Southwest State University, 2016, no. 4 (67), pp. 56-68.

22. Jeremy M. Webb, Shaheeda A. Adusei, Yinong Wang, Naziya Samreen, Kalie Adler, Duane D. Meixner, Robert T. Fazzio, Mostafa Fatemi, Azra Alizad. Comparing deep learning-based automatic segmentation of breast masses to expert interobserver variability in ultrasound imaging. Computers in Biology and Medicine, 2021, vol. 139, p. 104966.

23. Kudryavtsev P. S., Kuz'min A. A.,. Savinov D. Yu, Filist S. A., Shatalova O. V. Mod- elirovanie morfologicheskikh obrazovanii na rentgenogrammakh grudnoi kletki v intellektual'nykh diagnosticheskikh sistemakh meditsinskogo naznacheniya [Modeling of morphological formations on radiographs of a breast cell in intelligent diagnostic systems for medical purposes]. Kaspiiskii zhurnal: Menedzhment i vysokie tekhnologii = Caspian Journal: Management and High Technologies, 2017, no. 3 (39), pp. 109-120.