Author(s) :

Wenjin Zhao, Sara Pohland.

Volume/Issue :

Abstract :

Reinforcement Learning (RL) has proven to be a powerful and versatile framework for solving complex problems. It has demonstrated success in areas ranging from game theory to robotic control to autonomous navigation and path planning. However, the rapid development of numerous RL algorithms has outpaced the field’s ability to provide clear, standardized comparisons, leaving practitioners with limited guidance for selecting the most appropriate algorithm for a given task. This work addresses this gap by conducting a systematic empirical study comparing five prominent RL algorithms–Vanilla Policy Gradient (VPG), Deep Deterministic Policy Gradient (DDPG), Proximal Policy Optimization (PPO), Twin Delayed Deep Deterministic Policy Gradient (TD3), and Soft Actor-Critic (SAC)–across three distinct task categories: locomotion (Humanoid), continuous control (LunarLander), and navigation (FrozenLake). To ensure a fair comparison, we first performed a hyperparameter sweep for each algorithm in each environment. The final evaluation, which is based on average return, sample efficiency, and training stability, reveals that no single algorithm dominates in all domains. The key findings are that SAC is the superior choice for complex, continuous control, achieving the highest average episode return in both Humanoid and LunarLander; VPG performs surprisingly well in discrete, sparse-reward settings, achieving the highest average episode return in FrozenLake; and a critical trade-off exists between peak performance and training stability. Our findings aid practitioners in understanding the trade-offs to be considered in RL algorithm selection for different types of robotics tasks.