Analysis of Game Space Strategies Using Neural Networks

The purpose of the research is to develop methods to improve the efficiency of neural networks for building artificial intelligence systems in the analysis of the game space.

Methods. As the main method used in the developed software solution, the method of deep learning of neural networks with reinforcement, based on the use of a model of proximal strategy optimization, was used. We used a special ML- Agents plugin for the Unity game engine. On its basis, ready-made environments for training agents were used, and new environments for training agents that adaptively change during the implementation of the game were developed. The diagram of the interaction of the learning environment with the Python API system is presented, the components that make up the plugin are shown. A reinforcement learning cycle has been formed, which allows you to form the order of various states, the player's possible actions in relevant situations and the potential rewards he receives in the learning process. The goal is to maximize the expected payoff that a player can get during the entire learning cycle. The proximal strategy optimization algorithm in the ML-Agents plugin is implemented through the Tensor Flow machine learning software library and is executed in a separate Python API process that interacts with the launched Unity scene through an external communicator.

Results. It is shown that an increase in the efficiency of neural networks for the subsequent training of artificial intelligence is achieved, firstly, through the use of ultra-precise neural networks, and secondly, due to the expansion of functional capabilities, by choosing the pivot point of the formula.

Conclusion. Consideration of the actions of players will help to develop a software product for analyzing a game strategy using neural networks, which automatically determines their behavior, aimed at studying the individual characteristics of the game space.

1. Rutkovskaya D., Rutkovsky L., Pilinsky M. Neironnye seti, geneticheskie algoritmy i nechetkie sistemy [Neural networks, genetic algorithms and fuzzy systems]. Moscow, Hotline-Telecom Publ., 2006. 452 c.

2. Metody klassifikatsii i prognozirovaniya. Neironnye seti [Classification and forecasting methods. Neural networks]. Available at: https://www.intuit.ru/studies/professional_skill_im-provements/1210/courses/6/lecture/178. (accessed 10.02.2021)

3. Perseptrony [Perceptrons]. Available at: https://neuralnet.info/chapter/nepcernpoHbi/. (accessed 10.02.2021)

4. Ryszard T., Barbara B. Vvedenie v mashinnoe obuchenie s pomoshch'yu Python. Rukovodstvo dlya spetsialistov po rabote s dannymi [Introduction to Machine learning using Python. Guide for specialists in working with data]. Moscow, Hotline-Telecom Publ., 2017. 546 р.

5. Donskikh A. O., Sirota A. A. Metod iskusstvennogo razmnozheniya dannykh v zadachakh mashinnogo obucheniya s ispol'zovaniem neparametricheskikh yadernykh otsenok plotnosti raspredeleniya veroyatnostei [The method of artificial reproduction of data in machine learning problems using nonparametric nuclear estimates of the probability distribution density]. Vestnik Voronezhskogo gosudarstvennogo universiteta. Seriya: Sistemnyi analiz i informatsionnye tekhnologii = Bulletin of the Voronezh State University. Series: System Analysis and Information Technologies, 2017, no. 3. pp. 142-155.

6. Filist S. A., Dabagov A. R., Tomakova R. A., Kondrashov D. S. Metod iskusstvennogo razmnozheniya dannykh v zadachakh mashinnogo obucheniya s ispol'zovaniem neparametricheskikh yadernykh otsenok plotnosti raspredeleniya veroyatnostei [Multilayered morphological operators for segmentation of complex-structured raster semitone images]. Izvestiya Yugo-Zapadnogo gosudarstvennogo universiteta. Seriya: Upravlenie, vychislitel'naya tekhnika, informatika. Meditsinskoe priborostroenie = Proceedins of the State University. Series: Management, Computer Engineering, Computer Science. Medical Instrumentation, 2019, vol. 9, no. 3(32), pp. 44-63.

7. Tomakova R. A., Naser A. A., Shatalova O. V. Matematicheskoe obespechenie raspoz- navaniya i klassifikatsii slozhnostrukturiruemykh biologicheskikh ob"ektov [Mathematical support for recognition and classification of complexly structured biological objects]. Mezhdunarodnyi zhurnal prikladnykh i fundamental'nykh issledovanii = International Journal of Applied and Functional Research, 2012, no. 4, pp. 48-49.

8. Filist S. A., Tomakova R. A., Brezhneva A. N., Malyutina I. A., Alekseev V. A. Kletochnye protsessory v klassifikatorakh mnogokanal'nykh izobrazhenii [Cellular processors in classifiers of multichannel images]. Radiopromyshlennost' = Radio Industry, 2019, no. 1, pp. 45-52.

9. Tomakova R. A., Naser A. A., Shatalova O. V., Rudakova E. V. Universal'nye setevye struktury v zadachakh klassifikatsii mnogomernykh dannykh [Universal network structures in problems of classification of multidimensional data]. Sovremennye naukoemkie tekhnologii = Modern High-tech Technologies, 2012, no. 8, pp. 48-49. 10. Rashid T. Sozdaem neironnuyu set' [Creating a neural network]. Moscow, Publishing House "Williams", 2018. 435 р.

10. Kan K. A. Neironnye seti. Evolyutsiya [Neural networks. Evolution]. Moscow, LitRes Publ., 2019. 428 р.

11. Plug-in ml-agent for unity. Available at: https://habr.com/ru/post/416297/. (accessed 15.02.2021)

12. Unity ML-Agents Toolkit. Available at: https://github.com/Unity-Technologies/ml-agents. (accessed 15.02.2021)

13. AI based on Unity ML Agents. Available at: https://api-2d3d-cad.com/unity_ml_agents_quickstart/. (accessed 15.02.2021)

14. Primery sozdaniya okruzheniya dlya obucheniya v ML-Agents [Examples of creating an environment for training in ML-Agents]. Available at: https://github.com/Unity-Technolo-gies/ml-agents/blob/release_2_docs/docs/Learning-Environment-Examples.md. (accessed 17.02.2021)

15. Sozdanie Agentov v ML-Agents [Creating Agents in ML-Agents]. Available at: https://github.com/Unity-Technologies/ml-agents/blob/release_2_docs/docs/Learning-Environment-Design-Agents.md. (accessed 17.02.2021)

16. Primery optimizatsii agentov v ML-Agents [Examples of agent optimization in ML- Agents]. Available at: https://blogs.unity3d.com/ru/2019/11/11/training-your-agents-7-times-faster-with-ml-agents. (accessed 20.02.2021)

17. Sozdanie okruzheniya v ML-Agents [The creation of an environment in the ML- Agents]. Available at: https://github.com/Unity-Technologies/ml-agents/blob/release_2_docs/docs/Learning-Environment-Create-New.md. (accessed 20.02.2021)

18. Unity ML-Agents 1.0. Available at: [https://www.youtube.com/watch?v=_9aPZH6pyA8&ab_channel=SebastianSchuchmann. (accessed 21.02.2021)