## 1 Introduction
However, their algorithms only used the information about the velocity and angular value of the joints for training, they did not take advantage of the graph topology and the binding relationship of the humanoid robot, as in our method. Our method is compared with algorithms [20; 21; 22] in Section 4, which shows the high efficiency of this method. The authors were inspired by the fact that the robot's joints have geometric relations, for example, the knee joint is related to the thigh joint or the foot joint is related to the knee joint, so that this paper uses GCN to represent the constraints of robot joints above and integrate GCN into the reinforcement learning algorithm. In practice, Graph Neural Network (GNN) often applies to problems in the field of chemistry [26; 27] that require chemical bonding between the molecules, or GNN is often used to recommend web-scale [28]. The relationship between the joints of a bipedal robot has a similarity to the relationship between elements in chemistry. In several problems related to human joints, GCN achieved high efficiency when applied to predict actions in a video after extracting points on the human body into the form of a skeleton [29; 30; 31; 32]. In the field of reinforcement learning, GCN has been applied to the multi-agent problem where it finds out the relationship between agents for cooperative tasks [33]. There has been no research using graph neural networks for the robot walking problem before. The authors found that taking advantage of the features obtained from GCN before putting these features into the reinforcement learning algorithm gave better results than previous reinforcement learning algorithms. This article contributes a method dealing with the robot locomotion problems for the bipedal robot. Besides combining GCN with the TD3 algorithm to supplement the linking information of the joints, this article also builds a suitable reward function to achieve high efficiency. A two-legged robot model with six independent joint coordinates was also built by the authors for the implementation process. The process of training the model as well as testing the results is implemented through the Robot Operating System (ROS), which is suitable and has many utilities for robot control [34; 35; 36]. Simulation is performed in the Gazebo simulator [37; 38] that can be easily combined with ROS. Simulation results obtained after training are recorded in this link: [[https://youtu.be/t1](https://youtu.be/t1) MVmvCIWr0]([https://youtu.be/t1](https://youtu.be/t1) MVmvCIWr0) (accessed on 30 January 2022) (Supplementary Material). The content of this paper includes five sections: introduction, network architecture, training process, evaluation and conclusion. The introduction section gives ideas and proposes the GCTD3 method; the network architecture section aims to present in detail the theoretical basis of GCTD3 algorithm; the training process section depicts how to implement the algorithm in ROS and Gazebo as well as our experience in training process; the evaluation section is to compare our proposed method with other RL methods through evaluation metrics and result graphs; and the last section presents the conclusions drawn from this research.