Methods and algorithms for identifying special points in images obtained from an onboard photo video recorder of an unmanned aerial vehicle

The purpose of the research is to develop methods for controlling unmanned aerial vehicles based on the analysis of data coming from a video stream.
Methods. An unmanned aerial vehicle can lose contact with the satellite navigation system, so the task of ensuring its orientation using an onboard photo video recorder with onboard data processing becomes relevant. For this purpose, special points on the terrain are used, the identification of which in the picture allows restoring the orientation of the aircraft. To search for special points of the blob type in the picture, a method is proposed for transforming the original image into a criterion image, after threshold processing of which the coordinates of the blobs are obtained. A method has been developed for transforming the original image into a criterion image, which consists in determining correlation images. For each correlation image, a scalar blob identification criterion is determined, which allows determining the coordinates of a special point in pictures obtained from an onboard photo video recorder.
Results. To improve the accuracy of determining the coordinates of blobs in pictures, an aggregated blob of three special points was used. An algorithm for two-stage identification of coordinates of an aggregated blob is investigated. At the first stage, the coordinates of special points closest to the coordinates of the vertices of the aggregated blob are determined, and at the second stage, the coordinates of the vertices of the triangle whose center of gravity is closest to the center of gravity of the aggregated blob are determined. The algorithms for searching for special points have shown their efficiency at a high level of interference modeled in the image by means of Gaussian noise, as well as interference associated with the deviation of the aircraft from the specified course.
Conclusion. The formation of an aggregated blob with subsequent multi-stage identification allows for increasing the accuracy of determining its coordinates, and also makes it possible to record the deviation from the course of the aircraft in the area of two adjacent images and introduce appropriate corrections into the navigation system.

1. Lunev E.M. Convergence study of a new algorithm for determining navigation parameters of an unmanned aerial vehicle based on a photographic image. Trudy MAI = Proceedings of MAI. 2011;(45):46. (In Russ.)

2. Dyudin M.V., Zuev I.V., Filist S.A., Chudinov S.M. Automatic classifiers of complex structured images based on multimethod technologies of multicriteria selection. Voprosy radioelektroniki = Issues of Radio Electronics. 2015;(1):130‒140. (In Russ.)

3. Filist S.M., Riad Taha Al-Kasasbeh, Tomakova R.A., Osama M. Al-Habahbeh, A'kif Al-Fugara, Shatalova O., Korenevsky N.A., Gorbachev I.N., Ashraf Shaqadan, Ilyash M. Automated system for classifying images of video streams for timely detection of fires. International Journal of Remote Sensing. 2024;45:8157‒8180. https://doi.org/10.1080/01431161.2024.2398818

4. Khan A., Gupta S., Gupta S.K. Multi-hazard disaster studies: Monitoring, detection, recovery, and management, based on emerging technologies and optimal techniques. International Journal of Disaster Risk Reduction. 2020;(47):101642.

5. Filist S.A., Tomakova R.A., Brezhneva A.N., Malyutina I.A., Alekseev V.A. Cellular processors in multichannel image classifiers. Radiopromyshlennost' = Radio Industry. 2019; 29(1):45‒52. (In Russ.)

6. Goritov A.N., Bodrukhin A.A. Comparison of methods for selecting special points of objects in images of the working stage of a robot manipulator. Doklady TUSUR = Reports of TUSUR. 2019;22(3):61‒66. (In Russ.)

7. Andrianov N.A., Dementiev V.E., Tashlinsky A.G. Detection of objects in the image: from criteria Bayes and Neiman – Pearson on detectors based on EfficientDet neural. Komp'yuternaya optika = Computer Optics. 2022;46(1):139‒159. (In Russ.) https://doi.org/10.18287/2412-6179-CO-922

8. Filist S.A., Dabagov A.R., Tomakova R.A., Malyutina I.A., Kondrashov D.S. Cascade segmentation method of breast radiographs. Izvestiya Yugo-Zapadnogo gosudarstvennogo universiteta. Seriya: Upravlenie, vychislitel'naya tekhnika, informatika. Meditsinskoe priborostroenie = Proceedings of the Southwest State University. Series: Control, Computer Engineering, Information Science. Medical Instruments Engineering. 2019;9(1):49‒61. (In Russ.)

9. Zubov I.G. Method of automatic detection of key points of an object in an image. Izvestiya vuzov. Elektronika = Proceedings of Universities. Electronics. 2020; 23(6):6‒16. (In Russ.) https://doi.org/10.32603/1993-8985-2020-23-6-6-16

10. Kumar A. SURF feature descriptor for image analysis. Imaging and Radiation Research. 2023; 6:5643. (In Russ.) https://doi.org/10.24294/irr.v6i1.5643

11. David G. Lowe. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision. 2004;(60):91‒110.

12. Filist S.A., Tomakova R.A., Shatalova O.V., Kuzmin A.A., Kassim K.D.A. Method of classification of complex structured images based on self-organizing neural network structures. Radiopromyshlennost' = Radio Industry. 2016;(4):57‒65. (In Russ.)

13. Hui Kong, Hatice Cinar Akakin, Sanjay E. Sarma. A Generalized Laplacian of Gaussian Filter for Blob Detection and Its Applications. IEEE Transactions on Cybernetics. 2013:43:1719‒ 1733.

14. Filist S.A., Dabagov A.R., Tomakova R.A., Malyutina I.A., Kondrashov D.S. Multilayer morphological operators for segmentation of complexly structured raster halftone images. Izvestiya Yugo-Zapadnogo gosudarstvennogo universiteta. Seriya: Upravlenie, vychislitel'naya tekhnika, informatika. Meditsinskoe priborostroenie = Proceedings of the Southwest State University. Series: Control, Computer Engineering, Information Science. Medical Instruments Engineering. 2019; 9(3):44‒63. (In Russ.)

15. Lindeberg T., Garding J. Shape-adapted smoothing in estimation of 3-D shape cues from affine deformations of local 2-D brightness structure. Image Vis. Comput. 1997;15(6):415‒434.

16. Denisov A.A., Novikov A.I. Analysis of methods for detecting, describing and comparing key image points. Vestnik RGRTU = Bulletin of the Russian State Technical University. 2024;(89):104‒116. (In Russ.) https://doi.org/10.21667/1995-4565-2024-89-104-116

17. D2-Net: A Trainable CNN for Joint Description and Detection of Local Features / M. Dusmanu, I. Rocco, T. Pajdla, M. Pollefeys, J. Sivic, A. Torii, T. Sattler. URL: https://arXiv:1905.03561v1 (accessed 17.12.2024).

18. Filist S., Al-Kasasbeh R.T., Tomakova R.A., A'kif Al-Fugara, Al-Habahbeh O.M., Shatalova O., Korenevsky N.A., Gorbachev I.N., Shaqadan A., Ilyash M. An unmanned aerial vehicle autonomous flight trajectory planning method and algorithm for the early detection of the ignition source during fire monitoring, International. Journal of Remote Sensing. 2024;(45):4178‒4197. https://doi.org/10.1080/01431161.2024.2358451

19. Krasnobaev E.A., Chistobaev D.V., Malyshev A.L. Comparison of binary descriptors of singular image points under distortion conditions. Computer Optics. 2019;43(3):434‒445. (In Russ.) https://doi.org/10.18287/2412-6179-2019-43-3-434-445

20. Mikhailov A.P., Chibunichev A.G. Photogrammetry. Moscow: Izdatel'stvo MIIGaik; 2016. 294 p. (In Russ.) 21. Filist S.A., Shevtsov M.V., Belozerov V.A., Kondrashov D.S., Gorbachev I.N., Korsunsky N.A. An automated system for classifying images of video streams. Izvestiya Yugo-Zapadnogo gosudarstvennogo universiteta. Seriya: Upravlenie, vychislitel'naya tekhnika, informatika. Meditsinskoe priborostroenie = Proceedings of the Southwest State University. Series: Control, Computer Engineering, Information Science. Medical Instruments Engineering. 2021;11(4):85‒105. (In Russ.)