Patent Publication Number: US-11022983-B2

Title: Gait control method, device, and terminal device for biped robot

Description:
CROSS REFERENCE TO RELATED APPLICATIONS 
     This application claims priority to Chinese Patent Application No. 201711471845.9, filed Dec. 29, 2017, which is hereby incorporated by reference herein as if set forth in its entirety. 
     BACKGROUND 
     1. Technical Field 
     The present disclosure relates to robot technology, and particularly to a gait control method, device, and terminal device for a biped robot. 
     2. Description of Related Art 
     A biped robot has two walking modes: static walking and dynamic walking. In the dynamic walking mode, since the projection point of the centroid of the biped robot can be outside the supporting polygon of the robot, it will be relatively complicated to control the biped robot to perform dynamic walking, which makes dynamic walking a hot research topic in robot technology. 
     At present, although many biped robots have successfully realized that going up and down the stairs by means of dynamic walking, in the gait control method of the existing biped robots, in order to reduce the difficulty in gait planning and control algorithms, it is usually necessary to introduce the premise that the plantar of the robot is always parallel to the ground. That is to say, whenever the robot raises and falls the legs, the plantar has to keep parallel to the ground at all times. Therefore, this gait is quite different from the gait of humans, and has a low simulation level. Moreover, in this gait, it is difficult for the robot to lift the leg to a large extent while the plantar is always parallel to the ground, the speed of the biped robot to go up and down the stairs is limited, and the monovalent efficiency of the biped robot is therefore reduced. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       To describe the technical schemes in the embodiments of the present disclosure more clearly, the following briefly introduces the drawings required for describing the embodiments or the prior art. Apparently, the drawings in the following description merely show some examples of the present disclosure. For those skilled in the art, other drawings can be obtained according to the drawings without creative efforts. 
         FIG. 1  is a flow chart of a biped robot gait control method according to an embodiment of the present disclosure. 
         FIG. 2  is a schematic block diagram of a leg model of a biped robot according to an embodiment of the present disclosure. 
         FIG. 3  is a flow chart of an embodiment of step S 103  of the biped robot gait control method according to the present disclosure. 
         FIG. 4  is a schematic block diagram of a walking gait of a biped robot on a flat land according to an embodiment of the present disclosure. 
         FIG. 5  is a schematic block diagram of a model of a biped robot according to an embodiment of the present disclosure. 
         FIG. 6  is a schematic diagram of a biped robot gait control device according to an embodiment of the present disclosure. 
         FIG. 7  is a schematic diagram of a terminal device according to an embodiment of the present disclosure. 
     
    
    
     DETAILED DESCRIPTION 
     In the following descriptions, for purposes of explanation instead of limitation, specific details such as particular system architecture and technique are set forth in order to provide a thorough understanding of embodiments of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure may be implemented in other embodiments that are less specific of these details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present disclosure with unnecessary detail. 
     In order to explain the technical solutions described in the present disclosure, the following provides a description through specific embodiments. 
       FIG. 1  is a flow chart of a biped robot gait control method according to an embodiment of the present disclosure. In this embodiment, the method is a computer-implemented method executable for a processor. The biped robot gait control method of this embodiment can be applied to a servo of a robot, in the robot may be but not limited to a biped robot which can be equipped with sensors such as infrared sensors, ultrasound sensors, or laser sensors, and the servo may be used as a joint of the robot so as to realize the movement of a limb of the robot which connected to the joint, where the movement of the servo specifically means the movement of the output shaft of the servo. The robot includes a head, a body, a left arm, and a right arm, a left leg, and a right leg. As shown in  FIG. 1 , the method includes the following steps. 
     S 101 : planning an initial position of an ankle joint of the biped robot and a rotation angle of a sole of the biped robot to rotate around a toe or a heel of the biped robot. 
     S 102 : planning a body pose of the biped robot, where the body pose includes a body position and a body posture. 
     The initial position of the ankle joint and the rotation angle of the sole may be planned by, for example, manually or automatically defining and/or inputting the initial position and the rotation angle according to actual needs. In this embodiment, the robot specifically refers to a humanoid robot having two feet and being capable of walking. The robot can perform various motions or postures corresponding to a control command by receiving the control command of an external device or receiving the control command directly input by, the user. 
     The pose of the robot itself or a part of the robot (e.g., the both or the foot) refers to the position and the posture of the robot or the part in a space. In which, the position is a specific space coordinate, which can be represented by a position matrix; the posture refers to the final motion form that the robot or the part needs to present, which can be represented by a posture matrix composed of the cosine values of the included angles between each two of three axes in a coordinate system. 
     In this embodiment, a foot pose includes a foot position (e.g., the position of the ankle joint) and a foot posture (e.g., the rotation angle of the sole) which are respectively the position and the posture of each movable rod of the foot of the robot in a space. In which, the foot refers to as part of the robot which is composed of the ankle joint and the movable rod(s) below the ankle joint. 
     In this embodiment, in order to control the biped robot to go up and down the stairs by means of dynamic walking, various initialization parameters are planned through inputting by the user. The above-mentioned initialization parameters include, but are not limited to, the step height, the step width, and the angle between the foot of the robot and the horizontal ground. The initialization para asters are identified based on a preset algorithm to determine various control parameters of the biped robot at the current time. For example, the height of the leg of the biped robot at the current time, the length of the stride, the gait cycle, and the magnitude of the centroid shift are determined. Based on the determined control parameters, a centroid motion trajectory of the robot and a foot motion trajectory are obtained. In which, the foot motion trajectory includes the above-mentioned foot position and the foot posture. 
     The body pose of the biped robot may be planned by, for example, manually or automatically defining and/or inputting the body pose according to actual needs. The body pose of the robot is the pose of the trunk of the robot, which is used to indicate the position and posture of the centroid of the trunk at various times. The body pose of the robot is also obtained based on the above-mentioned various control parameters of the biped robot. 
     S 103  calculating a target position of the ankle joint based on the initial position of the ankle joint and the rotation angle of the sole. 
     S 104 : obtaining a joint angle of each of a plurality of joints of the biped robot by performing an operation on the body pose and the target position of the ankle joint utilizing an inverse kinematics algorithm. 
     The inverse kinematics is used to calculate all joint variables of the corresponding position of the robot based on the position and posture of a known end of the robot. 
     In this embodiment, the preset inverse kinematics algorithm is used to calculate the body pose and the foot posture including the target position of the ankle joint of the robot, so as to obtain the joint angle of each of the joints of the left and right legs of the robot. The above-mentioned inverse kinematics algorithm includes, but is not limited to, an analytical method and a numerical method. The joints of the robot include but are not limited to ankle joints, knee joints, and hip joints. Each joint is equipped with one servo for controlling the joint to rotate. In which, in the angle obtained through the above-mentioned inverse kinematics algorithm, the joint angle of each joint includes a roll angle, a pitch angle, and a swine angle of the joint. 
       FIG. 2  is a schematic block diagram of a leg model of a biped robot according to an embodiment of the present disclosure. As shown in  FIG. 2 , if the body pose and the sole pose of the biped robot are (p 1 , R 1 ) and (p 7 , R 7 ), respectively (a pose includes a position p and a rotation angle R), the distance from the origin of the coordinate system of the trunk to the hip joint is D, the length of the thigh is A, and the length of the calf is B, the position of the hip joint can be calculated by the following formula: 
     
       
         
           
             
               
                 
                   
                     
                       p 
                       2 
                     
                     = 
                     
                       
                         p 
                         1 
                       
                       + 
                       
                         
                           R 
                           1 
                         
                         ⁡ 
                         
                           [ 
                           
                             
                               
                                 0 
                               
                             
                             
                               
                                 D 
                               
                             
                             
                               
                                 0 
                               
                             
                           
                           ] 
                         
                       
                     
                   
                   ; 
                 
               
               
                 
                   ( 
                   1 
                   ) 
                 
               
             
           
         
       
     
     in the coordinate system of the ankle joint, the position vector of the hip joint is:
 
 r=R   7   T ( p   2   −p   7 )=[ r   x    r   y    r   z ] T   (2)
 
     then, the relative distance C between the ankle joint and the hip joint is:
 
 C =√{square root over ( r   x   2   +r   y   2   +r   z   2 )}  (3)
 
     according to the cosine theorem, in the biped robot, the angle q 5  of the knee joint is: 
     
       
         
           
             
               
                 
                   
                     
                       q 
                       5 
                     
                     = 
                     
                       
                         - 
                         
                           arccos 
                           ⁡ 
                           
                             ( 
                             
                               
                                 
                                   A 
                                   2 
                                 
                                 + 
                                 
                                   B 
                                   2 
                                 
                                 - 
                                 
                                   C 
                                   2 
                                 
                               
                               
                                 2 
                                 ⁢ 
                                 AB 
                               
                             
                             ) 
                           
                         
                       
                       + 
                       π 
                     
                   
                   ; 
                 
               
               
                 
                   ( 
                   4 
                   ) 
                 
               
             
           
         
       
     
     in the coordinate system of the ankle joint, based on the vector r, the roll angle q 7  and the pitch angle q 6  of the ankle joint are respectively:
 
 q   7 =atan 2( r   y   ,r   z ); and  (5)
 
 q   6 =−atan 2( r   x ,sign( r   z )√{square root over ( r   y   2   +r   z   2 )}−α;  (6)
 
     for the swing angle q 2 , the roll angle q 3 , and the pitch angle q 4  of the hip joint, the relationship between the postures of each connection rod of the hip joint is:
 
 R   7   =R   1   R   z ( q   2 ) R   x ( q   3 ) R   y ( q   4 ) R   y ( q   5   +q   6 ) R   x ( q   7 );  (7)
 
     therefore, based on the above-mentioned formula (7):
 
 q   2 =atan 2(− R   12   ,R   22 );  (8)
 
 q   2 =atan 2(− R   32   ,−R   12  sin  q   2   =R   22  cos  q   2 ); and  (9)
 
 q   4 atan 2(− R   31   ,R   33 )  (10)
 
     In this embodiment, a change amount of the position of the ankle joint of the biped robot in calculated based on the size of the sole (a pre-designed parameter of the biped robot) and the planned rotation angle of the sole, and then the target position of the ankle joint is obtained by adding the planned initial position of the ankle joint and the change amount of the position of the ankle joint (see  FIG. 5 ). 
     S 105 : adjusting a gait of the biped robot based on the joint angle of each of the joints. 
     Based on the joint angle of each joint in the biped robot obtained by the above-mentioned calculation, each joint of the biped robot is controlled to rotate, so that the joint reaches its corresponding swing angle, roll angle and pitch angle in each direction. Therefore, through the joint action of the joints, the adjustment of the gait of the biped robot during walking is realized, so that it can have a preset body posture and foot posture at each moment. 
     Furthermore, after adjusting the gait of the biped robot based on the calculated joint angles, it returns to the above-mentioned S 101  to recalculate the joint angle of each joint of the biped robot at this moment based on the corresponding foot pose (including the initial position of the ankle joint and the rotation angle of the sole) and body pose of the biped robot at the latest moment. 
     In this embodiment, planning and control to the foot pose of the robot is added in the process of adjusting the walking gait of the biped robot, which calculates the joint angle of each joint of the robot by integrating the body pose and the foot pose of the robot, so that the biped robot can realize walking the manner of humanoid gait after adjusting the gait of the robot based on the joint angle of each joint. Consequently, the simulation level of the robot is enhanced, so that the walking gait of the robot is smoother, the speed of the biped robot to go up and down the stairs is therefore increased, and the movement ability and movement efficiency of the robot are enhanced. 
       FIG. 3  is a flow chart of an embodiment of step S 103  of the biped robot gait control method according to the present disclosure. As shown in  FIG. 3 , the method includes the following steps. 
     S 103 : controlling the sole of the biped robot to rotate around the toe or the heel. 
     During the process of the robot to swing the legs, the sole of the robot is controlled to rotate around the toe or the heel at the moment of lifting and falling the legs, so that the plantar of the foot of the robot and the ground are no longer keeping horizontal, and a certain included angle is kept between the plantar and the ground. 
       FIG. 4  is a schematic block diagram of a walking gait of a biped robot on a flat land according to an embodiment of the present disclosure. As shown in  FIG. 4 , specifically, for one foot of the biped robot, when the change amount of the angle of the ankle joint in accordance with the planning is a positive value (the change amount is obtained by the calculation in step S 1032 ), a contact point A of the toe of the biped robot and a supporting surface is taken as an end point to control the sole of the biped robot to rotate around the contact point A; when the change amount of the angle of the ankle joint in accordance with the planning is a negative value, a contact point B of the heel of the biped robot and the supporting surface is taken as an end point to control the sole of the biped robot to rotate around the contact point B. Through the method provided by this embodiment, the biped robot is able to walk in a humanoid gait manner that advance with the toes first and then grounding with the heck. 
     S 1032 : calculating a change amount of the position of the ankle joint of the biped robot based on the initial position of the ankle joint and the rotation angle of the sole, so as to calculate the target position of the ankle joint. 
     In the process of the robot going up and down the stairs, the coordinates of the ankle joint of the robot at different moments are different. In this embodiment, at each moment, the target position of the and joint that is about to reach at the next moment is calculated. 
       FIG. 5  is a schematic block diagram of a model of a biped robot according to an embodiment of the present disclosure. As shown in  FIG. 5 , the solid line is the position of the leg for supporting before the rotation of the sole of the biped robot, and the broken line is the position where the leg for supporting is located after the sole of the biped robot is rotated for an angle θ. It is defined that the forward direction of the robot is the X-axis of the coordinate system, the vertical direction is the Z-axis of the coordinate system, the Y-axis of the coordinate system is determined by the right-handed system, and the origin of the coordinate system is located at the E point of the toe of the biped robot. In which, the point A represents the hip joint of the biped robot, the points B and C represent the initial position of the ankle joint and the knee joint of the biped robot and EF is the length of one foot of the biped robot. The B and C respectively represent the target positions of the knee joint and the ankle joint after the sole is rotated for the angle θ. 
     When the foot posture of the leg for supporting of the biped robot changes, the position of the coordinate of the ankle joint of the robot will change to indirectly affect the position and posture of the body of the robot. Therefore, in this embodiment, the change amount of the position of the ankle joint is calculated based on the rotation angle of the sole and the initial position of the ankle joint. 
     Specifically, in the sole EF of the biped robot in  FIG. 5 , the coordinate system is established with the toe E as the origin. Before the sole EF of the robot is controlled to rotate around the toe E, the initial position C of the ankle joint in the coordinate system is: 
     
       
         
           
             
               
                 
                   
                     [ 
                     
                       
                         
                           
                             C 
                             x 
                           
                         
                       
                       
                         
                           
                             C 
                             z 
                           
                         
                       
                     
                     ] 
                   
                   = 
                   
                     [ 
                     
                       
                         
                           
                             - 
                             DE 
                           
                         
                       
                       
                         
                           CD 
                         
                       
                     
                     ] 
                   
                 
               
               
                 
                   ( 
                   11 
                   ) 
                 
               
             
           
         
       
     
     After controlling the sole FE of the robot to rotate around the toe F for the angle θ, the ankle joint is rotated from the initial position C of the to the target position C. It can be known by the geometric relationship that, the position of the target position C in the coordinate system is: 
     
       
         
           
             
               
                 
                   
                     [ 
                     
                       
                         
                           
                             C 
                             x 
                             ′ 
                           
                         
                       
                       
                         
                           
                             C 
                             z 
                             ′ 
                           
                         
                       
                     
                     ] 
                   
                   = 
                   
                     [ 
                     
                       
                         
                           
                             
                               - 
                               CE 
                             
                             * 
                             
                               cos 
                               ⁡ 
                               
                                 ( 
                                 
                                   θ 
                                   + 
                                   
                                     θ 
                                     0 
                                   
                                 
                                 ) 
                               
                             
                           
                         
                       
                       
                         
                           
                             CE 
                             * 
                             
                               sin 
                               ⁡ 
                               
                                 ( 
                                 
                                   θ 
                                   + 
                                   
                                     θ 
                                     0 
                                   
                                 
                                 ) 
                               
                             
                           
                         
                       
                     
                     ] 
                   
                 
               
               
                 
                   ( 
                   12 
                   ) 
                 
               
             
           
         
       
     
     where θ o  is an initial included angle between the plane where the instep CE of the biped robot locates and the horizontal ground. 
     Based on the above formulas (11) and (12), it can be found that after the foot EF of the biped robot rotates the angle θ around the toe E, the change amount of the position of the ankle joint is: 
     
       
         
           
             
               
                 
                   
                     [ 
                     
                       
                         
                           
                             δ 
                             ankle_x 
                           
                         
                       
                       
                         
                           
                             δ 
                             ankle_z 
                           
                         
                       
                     
                     ] 
                   
                   = 
                   
                     
                       
                         [ 
                         
                           
                             
                               
                                 C 
                                 x 
                                 ′ 
                               
                             
                           
                           
                             
                               
                                 C 
                                 z 
                                 ′ 
                               
                             
                           
                         
                         ] 
                       
                       - 
                       
                         [ 
                         
                           
                             
                               
                                 C 
                                 x 
                               
                             
                           
                           
                             
                               
                                 C 
                                 z 
                               
                             
                           
                         
                         ] 
                       
                     
                     = 
                     
                       [ 
                       
                         
                           
                             
                               
                                 
                                   - 
                                   CE 
                                 
                                 * 
                                 
                                   cos 
                                   ⁡ 
                                   
                                     ( 
                                     
                                       θ 
                                       + 
                                       
                                         θ 
                                         0 
                                       
                                     
                                     ) 
                                   
                                 
                               
                               + 
                               DE 
                             
                           
                         
                         
                           
                             
                               
                                 CE 
                                 * 
                                 
                                   sin 
                                   ⁡ 
                                   
                                     ( 
                                     
                                       θ 
                                       + 
                                       
                                         θ 
                                         0 
                                       
                                     
                                     ) 
                                   
                                 
                               
                               - 
                               CD 
                             
                           
                         
                       
                       ] 
                     
                   
                 
               
               
                 
                   ( 
                   13 
                   ) 
                 
               
             
           
         
       
     
     The target position of the ankle joint is calculated based on the positional change of the ankle joint by adding the positional change of the ankle joint and the initial position of the ankle joint. Based on the body pose and the target position of the ankle joint corresponding to the biped robot at the current time, the joint angle of each joint of the biped robot is calculated by inverse kinematics. 
     In this embodiment, by controlling the sole of the robot to rotate, the thigh of the robot can be swung in advance based on the change of the position of the ankle joint before the body of the robot moves, thereby reducing the influence of the thigh on the posture of the body of the robot during the swing of the body. In this embodiment, since the sole of the robot is no longer kept parallel with the ground at every moment, and walks in a manner that advance with the toes first and then grounding with the heels. As a result, the biped robot realizes a manner of going up and down the stairs in a humanoid gait. 
     It should be understood that, the sequence of the serial number of the steps in the above-mentioned embodiments does not mean the execution order while the execution order of each process should be deter pined by its function and internal logic, which should not be taken as any limitation to the implementation process of the embodiments. 
       FIG. 6  is a schematic diagram of a biped robot gait control device according to an embodiment of the present disclosure. The biped robot, gait control device corresponds to the biped robot gait control method described in the above-mentioned embodiment. For the convenience of description, only parts related to this embodiment are shown. 
     As shown in  FIG. 6 , the device is: 
     a foot posture planning unit  71  configured to plan an initial position of an ankle joint cat the biped robot and a rotation angle of a sole of the biped robot to rotate around a toe or a heel of the biped robot; 
     a body pose planning unit  72  configured to plan a body pose of the biped robot, where the body pose includes a body position and a body posture; 
     a target position calculating unit  73  configured to calculate a target position of the ankle joint based on the initial position of the ankle joint and the rotation angle of the sole; 
     a joint angle obtaining unit  74  configured to obtain a joint angle of each of a plurality of joints of the biped robot by performing an operation on the body pose and the target position of the ankle joint utilizing an inverse kinematics algorithm; and 
     a gait adjusting unit  75  configured to adjust a gait of the biped robot based on the joint angle of each of the joints. 
     Optionally, the gait adjusting unit  75  includes: 
     a rotation subunit configured to control the sole of the biped robot to rotate around the toe or the heel; and 
     a first calculating subunit configured to calculate the target position of the ankle joint by calculating a change amount of the position of the ankle joint of the biped robot based on the initial position of the ankle joint and the rotation angle of the sole. 
     Optionally, the rotation subunit is specifically configured to: 
     take a contact point of the toe of the biped robot and a supporting surface as a first end point to control the sole of the biped robot to rotate around the first end point; and 
     take a contact point of the heel of the biped robot and the supporting surface as a second end point to control the sole of the biped robot to rotate around the second end point. 
     Optionally, the biped robot gait control device further includes: 
     A returning unit  76  configured to return to the step (S 101 ) of planning the initial position of the ankle joint of the biped robot and the rotation angle of the sole of the biped robot to rotate around the toe or the heel of the biped robot. 
     In this embodiment, planning and control to the foot pose of the robot is added in the process of adjusting the walking gait of the biped robot, which calculates the joint angle of each joint of the robot by integrating the buds pose and the foot pose of the robot, so that the biped robot can realize walking in the manner of humanoid gait after adjusting the gait of the robot based on the joint angle of each joint. Consequently, the simulation level of the robot is enhanced, so that the walking gait of the robot is smoother, the speed of the biped robot to go up and down the stairs is therefore increased, and the movement ability and movement efficiency of the robot are enhanced. 
       FIG. 7  is a schematic diagram of a terminal device according to an embodiment of the present disclosure. The terminal device of this embodiment can be used to control a servo of a robot. As shown in  FIG. 7 , the terminal device  8  of this embodiment includes a processor  80 , a memory  81 , and a computer program  82  stored in the memory  81  and executable on the processor  80 , for example, a biped robot gait control program. When executing (instructions in) the computer program  82 , the processor  80  implements the steps in the above-mentioned embodiments of the biped robot gait control method, for example, steps S 101 -S 104  shown in  FIG. 1 . Alternatively, when the processor  80  executes (instructions in) the computer program  82 , the functions of each module/unit in the above-mentioned device embodiments, for example, the functions of the modules  71 - 75  shown in  FIG. 5  are implemented. 
     Exemplary, the computer program  82  may be divided into one or more modules/units, and the one or more modules/units are stored in the storage  81  and executed by the processor  80  to realize the present disclosure. The one or more modules/units may be a series of computer program instruction sections capable of performing a specific function, and the instruction sections are for describing the execution process of the computer program  82  in the terminal device  8 . 
     The terminal device  8  may be a computing device such as a desktop computer, a notebook computer, a tablet computer, and a cloud server. The terminal device  8  may include, but is not limited to, a processor  80  and a storage  81 . It can be understood by those skilled in the art that  FIG. 7  is merely an example of the terminal device  8  and does not constitute a limitation on the terminal device  8 , and may include more or fewer components than those shown in the figure, or a combination of some components or different components. For example, the terminal device  8  may further include an input/output device, a network access device, a bus, and the like. 
     The processor  80  may be a central processing unit (CPU), or be other general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or be other programmable logic device, a discrete gate, a transistor logic device, and a discrete hardware component. The general purpose processor may be a microprocessor, or the processor may also be any conventional processor. 
     The storage  81  may be an internal storage unit of the terminal device  8 , for example, a hard disk or a memory of the terminal device  8 . The storage  81  may also be an external storage device of the terminal device  8 , for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, flash card, and the like, which is equipped on the terminal device  8 . Furthermore, the storage  81  may further include both an internal storage unit and an external storage device, of the terminal device  8 . The storage  81  is configured to store the computer program and other programs and data required by the terminal device  8 . The storage  81  may also be used to temporarily store data that has been or will be output. 
     Those, skilled in the art may clearly understand that, for the convenience and simplicity of description, the division of the above-mentioned motional units and modules is merely an example for illustration. In actual applications, the above-mentioned functions may be allocated to be performed by different functional units according to requirements, that is, the internal structure of the device may be divided into different functional units or modules to complete all or part of the above-mentioned functions. The functional units and modules in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional unit. In addition, the specific name of each functional unit and module is merely for the convenience of distinguishing each other and are not intended to limit the scope of protection of the present disclosure. For the specific operation process of the units and modules in the above-mentioned system reference may be made to the corresponding processes in the above-mentioned method embodiments, and are not described herein. 
     In the above-mentioned embodiments, the description of each embodiment has its focuses, and the parts which are not described or mentioned in one embodiment may refer to the related descriptions in other embodiments. 
     Those ordinary skilled in the art may clearly understand that, the exemplification units and steps described in the embodiments disclosed herein may be implemented through electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented through hardware or software depends on the specific application and design constraints of the technical schemes. Those ordinary skilled in the art may implement the described functions in different manners for each particular application, while such implementation should not be considered as beyond the scope of the present disclosure. 
     In the embodiments provided by the present disclosure, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other manners. For example, the above-mentioned apparatus/terminal device embodiment is merely exemplary. For example, the division of modules or units is merely a logical functional division, and other division manner may be used in actual implementations, that is, multiple units or components may be combined or be integrated into another system, or some of the features may be ignored or not performed. In addition, the shown or discussed mutual coupling may be direct coupling or communication connection, and may also be indirect coupling or communication connection through some interfaces, devices or units, and may also be electrical, mechanical or other forms. 
     The units described as separate components may or may not be physically separated. The components represented as units may or may not be physical units, that is, may be located in one place or be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of this embodiment. 
     In addition, each functional unit in each of the embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional unit. 
     When the integrated module/unit is implemented in the form of a software functional unit and is sold or used as an independent product, the integrated module/unit may be stored in a non-transitory computer-readable storage medium. Based on this understanding, all or part of the processes in the method for implementing the above-mentioned embodiments of the present disclosure are implemented, and, may also be implemented by instructing relevant hardware through a computer program. The computer program may be stored in a non-transitory computer-readable storage medium, which may implement the steps of each of the above-mentioned method embodiments when executed by a processor. In which, the computer program includes computer program codes which may be the form of source codes, object codes, executable files, certain intermediate, and the like. The computer-readable medium may include any primitive or device capable of carrying the computer program codes, a recording medium, a USB flash drive, a portable hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), electric carrier signals, telecommunication signals and software distribution media. It should be noted that the content contained in the computer readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to the legislation and patent practice, a computer readable medium does not include electric carrier signals and telecommunication signals. 
     The above-mentioned embodiments are merely intended for describing but not for limiting the technical schemes of the present disclosure. Although the present disclosure is described in detail with reference to the above-mentioned embodiments, it should be understood by those skilled in the art that, the technical schemes in each of the above-mentioned embodiments may still be modified, or some of the technical features may be equivalently replaced, while these modifications or replacements do not make the essence of the corresponding technical schemes depart from the spirit and scope of the technical schemes of each of the embodiments of the present disclosure, and should be included within the scope of the present disclosure.