###### Abstract  
Autonomous driving technology faces significant challenges in processing complex environmental data and making real-time decisions. Traditional supervised learning approaches heavily rely on extensive data labeling, which incurs substantial costs. This study presents a complete implementation framework combining Deep Deterministic Policy Gradient (DDPG) reinforcement learning with 3D-LiDAR perception techniques for practical application in autonomous driving. DDPG meets the continuous action space requirements of driving, and the point cloud processing module uses a traditional algorithm combined with attention mechanisms to provide high awareness of the environment. The solution is first validated in a simulation environment and then successfully migrated to a real environment based on a 1/10-scale F1 tenth experimental vehicle. The experimental results show that the method proposed in this study is able to complete the autonomous driving task in the real environment, providing a feasible technical path for the engineering application of advanced sensor technology combined with complex learning algorithms in the field of autonomous driving. autonomous driving; deep reinforcement learning; Deep Deterministic Policy Gradient (DDPG); 3D-LiDAR; point cloud; attention mechanism +
Footnote †: journal: Applied sciences  
0000-0001-999-4000-0000-0000-0000-0000-0000-0000-0000-0000-0000-0000-0000-00000-  
facing diverse and unpredictable road scenarios, struggling to handle uncertainties that were not present in the training data. Moreover, supervised learning's dependency on extensive labeled data to improve model performance demands substantial human resources and time [4]. Deep reinforcement learning addresses these challenges, drawing inspiration from human learning patterns. Agents learn and make decisions through environmental interaction and feedback without explicit human supervision for data labeling [5]. This unsupervised learning approach can accomplish autonomous driving tasks while reducing technology costs. This study implements autonomous driving tasks using reinforcement learning with 3D-LiDAR sensors in both simulated and real environments. While Deep Q-Network (DQN) serves as a foundational reinforcement learning algorithm with proven success in many applications, it exhibits inherent limitations when handling continuous action spaces, which are crucial for fine vehicle control in real-world scenarios [6]. Vehicles typically require precise, continuous adjustments in acceleration, braking, and steering to ensure both safety and passenger comfort. DQN is designed for discrete action selection, making it difficult for it to perform well in such situations. To address these issues, this study applies the Deep Deterministic Policy Gradient (DDPG) algorithm, which directly maps sensory inputs to continuous action outputs, making it particularly well suited for continuous control problems [7]. DDPG effectively combines deep reinforcement learning benefits with an actor-critic architecture, learning optimal driving policies through systematic trial and error. The actor network determines optimal actions based on current states, while the critic network evaluates action quality and provides detailed feedback to refine the actor's behavior. This combination enhances learning efficiency, ensuring vehicles can adapt to various driving scenarios while maintaining consistent performance. Compared to traditional RGB camera sensors that rely primarily on vision, 3D-LiDAR technology provides significantly more precise depth measurements, clearly detecting object contours and positions regardless of challenging lighting conditions, including night-time operations, strong direct sunlight, or heavily shadowed areas. It constructs detailed 3D environmental maps by emitting laser pulses and measuring their return times, generating high-precision distance information with millimeter-level accuracy.