Autonomous Agent Learning for Navigation in continuous  Environments: Deep Reinforcement Learning Approach

Ina'am rabah; Maha Adham Al-Bayati; Farah qais Al khalidi

doi:10.24237/ASJ.03.03.870A

Authors

Ina'am rabah Al-Sharia Department , College of Islamic Sciences, Diyala University https://orcid.org/0009-0005-6896-1654
Maha Adham Al-Bayati Computer Science Department, College of science, Al-Mustansiriya University, Baghdad, Iraq https://orcid.org/0000-0003-0975-432X
Farah qais Al khalidi Computer Science Department, College of science, Al-Mustansiriya University, Baghdad, Iraq https://orcid.org/0000-0002-8604-2484

DOI:

https://doi.org/10.24237/ASJ.03.03.870A

Keywords:

Reinforcement learning, Simulated environment, DQN, Autonomous Learning

Abstract

Recently, there has been an increasing interest for applying techniques of machine learning in autonomous agent learning. The autonomous ability of agents to detect and adapt to their environment enhances their adaptability and efficacy in completing numerous activities. A learning framework known as Reinforcement learning (RL) learns an agent how to act in a way that maximizes the value of a reward signal. Within the domains of agentics, agents trained to perform various tasks through trial and error, by employing reinforcement learning. In this paper, by utilizing one of the reinforcement learning algorithms, Deep Q-learning Network (DQN). DQN, used for autonomous agent learning, the agent can learn and determine the optimum policy that will guide it to its destination from its interactions with the environment. The agent's decision-making ability allows it to modify its policy as a response to observable circumstances. Experiments performed in a simulated continuous environment, the revealed results that presented the ability of an agent to react in a response to changing conditions and demonstrated that the agent was effectively adapting its behavior in accordance with the learned policy.

Downloads

Download data is not yet available.

References

T. Ribeiro, F. Gonçalves, I. Garcia,G. Lopes, A.F. Ribeiro,” Q-learning for autonomous autonomous robot obstacle avoidance,” In2019 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC) ,IEEE, pp. 1-7,2019.

R. S. Sutton and A. G. Barto,” Reinforcement learning: An introduction,” MIT press, 2018.‏

F. Gul , W. Rahiman and S. S. N. Alhady,” A comprehensive study for agent navigation techniques”, Cogent Engineering, vol. 6, no. 1,pp. 1-25,2019.

W. Wang, Z. Wu, H. Luo and B. Zhang,” Path planning method of autonomous robot using improved deep reinforcement learning,”Journal of Electrical and Computer Engineering, vol. 2022 , pp.1-7, Apr 2022.

W.D. Smart, L.P. Kaelbling, "Practical reinforcement learning in continuous spaces," in Proceedings of the seventeenth international conference on machine learning, pp. 903-910, 2000.

W. D. Smart, L. P. Kaelbling,”Effective reinforcement learning for autonomous robots,” In Proceedings 2002 IEEE International Conference on Robotics and Automation, vol. 4, pp. 3404-3410, 2002.

Z. Liu, Q.Wang and B. Yang,” Reinforcement Learning-Based Path Planning Algorithm for Autonomous Robots,” Wireless Communications and Autonomous Computing, vol. 2022, May 2022.

H. Boem and H. Cho, "A sensor-based navigation for a autonomous agent using fuzzy logic and reinforcement learning," IEEE Transaction on System, Man, and Cybernetics, vol. 25, pp. 464-477, 1995.

N. Yung and C. Ye, "Self-learning fuzzy navigation of autonomous vehicle," In Proceedings of Third International Conference on Signal Processing ,vol. 2, pp. 1465-1468,1996.

G. Yang, E. Chen and C. An, "Autonomous agent navigation using neural Q learning," in IEEE proceedings of the third international conference on machine learning and cybernetics, 2004.

K. Macek, I. Petrovic and N. Peric, "A reinforcement learning approach to obstacle avoidance of autonomous agents," in 7th international workshop on advanced motion control, 2002.

J. Gao, W. Ye, J. Guo, Z. Li,” Deep reinforcement learning for indoor autonomous robot path planning,” Sensors. Vol. 20, no. 19, p. 5493, 2020.

Z. H. Zhengwan, Z. H. Chunjiong, L. I. Hongbing, and X. I. Tao, “Multipath transmission selection algorithm based on immune connectivity model,” Journal of Computer Applications, vol. 40, no. 12, p. 3571, 2020.

D. Barnes, W. Maddern, and I. Posner, “Find your own way: Weakly-supervised segmentation of path proposals for urban autonomy,” In 2017 IEEE International Conference on Robotics and Automation (ICRA) ,pp. 203-210,2017.

R. Hadsell, P. Sermanet, J. Ben, A. Erkan, M. Scoffier, K. Kavukcuoglu, U. Muller and Y. LeCun, “Learning long-range vision for autonomous off-road driving,” Journal of Field Agentics, vol. 26, no. 2, pp. 120–144, 2009.

D. A. Pomerleau, “Alvinn: An autonomous land vehicle in a neural network,” Advances in neural information processing systems, vol. 1, pp. 305–313, 1989.

M. Bojarski, D. Del Testa, D. Dworakowski, B. Firner, B. Flepp, P. Goyal, L. D. Jackel, M. Monfort, U. Muller and J. Zhang , “End to end learning for self-driving cars,” arXiv preprint arXiv:1604.07316. 2016 .

F. Al-Mahamid and K. Grolinger ,” Reinforcement Learning Algorithms: An Overview and Classification”, In 2021 IEEE Canadian Conference on Electrical and Computer Engineering (CCECE) ,pp. 1-7,2021.

A. S. Shaker, S. R. Ahmed,” Information Retrieval for Cancer Cell Detection Based on Advanced Machine Learning Techniques,” Al-Mustansiriyah Journal of Science, vol. 33, Issue 3, pp. 20-26, 2022.

G.Butler, A. Gantchev, P. Grogono,” Object-oriented design of the subsumption architecture”, Software: Practice and Experience, vol. 31,no. 9, pp. 911-923, 2001.

K. Karur, N. Sharma , C. Dharmatti and J. E. Siegel,” A Survey of Path Planning Algorithms for Autonomous Agents,” Vehicles , vol. 3,no. 3, pp. 448-468,2021.

J. Cui, G. Nie,” Motion Route Planning and Obstacle Avoidance Method for Autonomous Robot Based on Deep Learning,” Journal of Electrical and Computer Engineering. vol. 2022 ,Article ID 5739765, pp. 1-11, 2022.

H.Y. Ryu , J.S. Kwon, J. H. Lim, A.H. Kim, S.J. Baek and J. W. Kim ,” Development of an Autonomous Driving Smart Wheelchair for the Physically Weak”, Applied Sciences, vol. 12, no. 1, pp. 1-18, 2021.

J. Raajan, P.V. Srihari, J.P. Satya, B. Bhikkaji and R. Pasumarthy,” Real time path planning of agent using deep reinforcement learning,” IFAC-PaperOnLine, vol. 53, no. 2,pp. 15602-15607, 2020.

V. A. Bakale, V.S. YK, V. C. Roodagi, Y. N. Kulkarni, M. S. Patil, S. Chickerur,” Indoor Navigation with Deep Reinforcement Learning”, In 2020 International Conference on Inventive Computation Technologies (ICICT), pp. 660-665, 2020.

R. Singh, J. Ren and X. Lin,” A Review of Deep Reinforcement Learning Algorithms for Autonomous Agent Path Planning,” Vehicles, vol. 5, no. 4, pp. 1423-1451,2023.

H. H. Ali, J. R. Naif and W. R. Humood, " A New Smart Home Intruder Detection System Based on Deep Learning, " Al-Mustansiriyah Journal of Science, vol. 34, Issue: 2, pp. 60-69, 2023.

X. Lei, Z. Zhang and P. Dong,” Dynamic path planning of unknown environment based on deep reinforcement learning,” Journal of Robotics, vol. 2018, no. 12,pp. 1-10, 2018.

V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, M. Riedmiller “Playing atari with deep reinforcement learning,” arXiv:1312.5602v1 [cs.LG], 2013.

N. M. Khassaf and S. H. Shaker,” Image Retrieval based Convolutional Neural Network,” Al-Mustansiriyah Journal of Science, vol. 31, Issue: 4, pp. 43- 54, 2020.

K. Yeom,” Deep Reinforcement Learning Based Autonomous Driving with Collision Free for Autonomous Agents,” International Journal of Mechanical Engineering and Robotics Research, vol. 11, no. 5, pp. 338-44, 2022.