## 4 Experiments and Results
### Implement in Real Environment
The processor performs the point cloud processing stage and subsequently transfers the processed information to the deep reinforcement learning algorithm. The computed action commands are then transmitted via VESC to control the motor and execute agent movements. Throughout this process, communication between the agent and station is maintained through a WiFi antenna. The station transmits commands (such as initiation or termination of experiments) to the agent, while the agent returns experimental data and results back to the station. Figure 8: The experimental agent. Figure 9: The system architecture of the real-world experiment. The real-world experiments were conducted in a controlled laboratory environment. The algorithms and models used in the experiments were kept the same as in the simulator. Since this study focuses on testing the effectiveness of the algorithms applied in real-world environments and there is a limitation of the battery capacity in the hardware conditions, the task objective of the experiment was simplified to basic obstacle avoidance and driving behavior selection in a defined environment. The effective implementation of the basic task is the basis for future in-depth research on difficult tasks in complex environments. The map used for the experiment is shown in Figure 10. The closest obstacle is in front of the right side of the experimental vehicle, and the open area is on the left side. The experimental vehicle freely explores the map and selects action outputs to form its own driving strategy. The experimental process consisted of different stages, where the system's ability to self-learn and adapt was observed through multiple trials. In the initial stage, the agent failed to avoid the first obstacle and collided with a chair (Figure 11a). This highlights the need for additional training to refine its decision-making capabilities. In the subsequent trials, after more than 200 steps, the vehicle was able to bypass the first obstacle but encountered difficulties with the second chair, colliding with it (Figure 11b). After additional training iterations, around 600 steps, the vehicle successfully avoided both obstacles. Instead of continuing toward the obstacle on the right, the vehicle adjusted its trajectory to move toward the open space on the left (Figure 11c). This showed the system's ability to learn from previous experiences and improve its strategy through reinforcement learning. Figure 12 shows the reward curve of the experimental vehicle following DDPG training. It illustrates the reward dynamics of the agent during the training process. The reward obtained by the agent is increases as the training progresses. During the initial training phase, the reward values remained relatively low. When the steps reached about 300, the reward value increased significantly, but it was not stable enough and fluctuated a lot. Beyond 600 steps, the reward values stabilized and consistently maintained a high level. This trend demonstrates the effectiveness of the DDPG training approach for the agent. This research has successfully transitioned from implementing autonomous driving tasks through reinforcement learning in virtual environments to effectively executing algorithms and completing tasks in real-world environments. This provides a foundation and starting point for applying reinforcement learning to achieve complex autonomous driving in real-world scenarios. This experiment is intended to demonstrate the practical  
Figure 10: Map of the experiment. feasibility of the algorithm, so basic experimental tasks were set and implemented in a simple environment. Future applications and implementations could enhance the capacity of the battery to perform more complex tasks and set up more sophisticated reward mechanisms based on diverse road conditions and environmental scenarios. By establishing more complex reward mechanisms and parameter adjustments, the agent can potentially achieve superior performance and more intelligent behavior in real-world driving scenarios.