Patent ID: 12214497

Like reference symbols in the various drawings indicate like elements.

DETAILED DESCRIPTION

As a robot maneuvers about an environment, various system of the robot communicate with the robot to indicate where to step (e.g., foot placement for a leg), how to move legs of the robot (e.g., swing trajectories), how to avoid obstacles, how to maintain balance, etc. One of the more critical aspects of robot movement is understanding the robot's relationship with its environment. For instance, one aspect is understanding how a structure of the robot interacts with itself or objects in the environment (e.g., a traction surface of the environment). During movement, these interactions depend not only on the environment, but also on characteristics of the robot (e.g., speed, gait, location, etc.). To understand these relationships for the robot, the robot uses various techniques for impact detection and/or impact reaction.

With a legged-robot, it may be important to accurately detect when a leg of the robot has contacted a traction surface (e.g., a ground surface). For instance, when a leg of the robot contacts the ground surface (i.e., touches down), but the robot does not detect the leg's touchdown, the robot will continue moving this leg downward, essentially attempting to push the leg through a ground surface. Here, when continuing to move the leg downward, the robot experiences a normal force from the ground surface acting on the leg. Problematically, this normal force may disrupt the balance of the robot and, in some situations, cause the robot to turn over (e.g., flip over). When the robot detects touchdown late, it affects lateral stability for the robot because the robot experiences a larger than expected impact. In other words, the robot does not anticipate ground impact when it actually occurs. Therefore, the robot may not account for, or properly counterbalance, such an impact.

Besides accurately detecting touchdown, the robot also employs impact detection to detect trips. A trip generally refers to a disruption in a gait of the robot. In other words, the robot may have a planned movement trajectory (e.g., based on a gait controller), but trip due to contact with some object, element of the environment, or itself. For both trips and traction surface contact (e.g., touchdown of a leg), it may be advantageous to not only detect such conditions, but also to minimize a time it takes to detect and/or to respond to such a condition. Often with trips, the more quickly a robot detects a trip, the less force (or rotation) the robot will experience from the trip.

Additionally, there may be a proportional relationship between how fast a trip is detected and a likelihood that a robot will overcome the trip condition (e.g., stabilize or continue with motion). In other words, quickly detecting a trip may allow systems of the robot to quickly react and/or respond to a trip. Therefore, trip detection aims to minimize disruptions to the robot during movement about the environment.

Referring toFIG.1A, the robot100includes a body110with locomotion based structures such as legs120a-dcoupled to the body110that enable the robot100to move about the environment10. In some examples, each leg120is an articulable structure such that one or more joints J permit members122of the leg120to move. For instance, each leg120includes a hip joint JHcoupling an upper member122,122uof the leg120to the body110and a knee joint JKcoupling the upper member122uof the leg120to a lower member122Lof the leg120. For impact detection, the hip joint JHmay be further broken down into abduction-adduction rotation of the hip joint JHdesignated as “JHx” for occurring in a frontal plane of the robot100(i.e., a X-Z plane extending in directions of a x-direction axis Axand the z-direction axis AZ) and a flexsion-extension rotation of the hip joint JHdesignated as “JHy” for occurring in a sagittal plane of the robot100(i.e., a Y-Z plane extending in directions of a y-direction axis AYand the z-direction axis AZ). AlthoughFIG.1Adepicts a quadruped robot with four legs120a-d, the robot100may include any number of legs or locomotive based structures (e.g., a biped or humanoid robot with two legs) that provide a means to traverse the terrain within the environment10.

In order to traverse the terrain, each leg120has a distal end124that contacts a surface of the terrain (i.e., a traction surface). In other words, the distal end124of the leg120is the end of the leg120used by the robot100to pivot, plant, or generally provide traction during movement of the robot100. For example, the distal end124of a leg120corresponds to a foot of the robot100. In some examples, though not shown, the distal end124of the leg120includes an ankle joint JAsuch that the distal end124is articulable with respect to the lower member122Lof the leg120.

The robot100has a vertical gravitational axis (e.g., shown as a Z-direction axis AZ) along a direction of gravity, and a center of mass CM, which is a point where the weighted relative position of the distributed mass of the robot100sums to zero. The robot100further has a pose P based on the CM relative to the vertical gravitational axis AZ(i.e., the fixed reference frame with respect to gravity) to define a particular attitude or stance assumed by the robot100. The attitude of the robot100can be defined by an orientation or an angular position of the robot100in space. Movement by the legs120relative to the body110alters the pose P of the robot100(i.e., the combination of the position of the CM of the robot and the attitude or orientation of the robot100). Here, a height generally refers to a distance along the z-direction. The sagittal plane of the robot100corresponds to the Y-Z plane extending in directions of a y-direction axis AYand the z-direction axis AZ. In other words, the sagittal plane bisects the robot100into a left and right side. Generally perpendicular to the sagittal plane, a ground plane (also referred to as a transverse plane) spans the X-Y plane by extending in directions of the x-direction axis AXand the y-direction axis AY. The ground plane refers to a ground surface12where distal ends124of the legs120of the robot100may generate traction to help the robot100move about the environment10. Another anatomical plane of the robot100is the frontal plane that extends across the body110of the robot100(e.g., from a left side of the robot100with a first leg120ato a right side of the robot100with a second leg120b). The frontal plane spans the X-Z plane by extending in directions of the x-direction axis AXand the z-direction axis Az.

When a legged-robot moves about the environment10, the legs120of the robot undergo a gait cycle. Generally, a gait cycle begins when a leg120touches down or contacts a ground surface12and ends when that same leg120once again contacts the ground surface12. The gait cycle may predominantly be divided into two phases, a swing phase and a stance phase. During the swing phase, a leg120performs (i) lift-off from the ground surface12(also sometimes referred to as toe-off and the transition between the stance phase and swing phase), (ii) flexion at a knee joint JKof the leg120, (iii) extension of the knee joint JKof the leg120, and (iv) touchdown back to the ground surface12. Here, a leg120in the swing phase is referred to as a swing leg120SW. As the swing leg120SWproceeds through the movement of the swing phase120SW, another leg120performs the stance phase. The stance phase refers to a period of time where a distal end124(e.g., a foot) of the leg120is on the ground surface12. During the stance phase a leg120performs (i) initial ground surface contact which triggers a transition from the swing phase to the stance phase, (ii) loading response where the leg120dampens ground surface contact, (iii) mid-stance support for when the contralateral leg (i.e., the swing leg120SW) lifts-off and swings to a balanced position (about halfway through the swing phase), and (iv) terminal-stance support from when the robot's COM is over the leg120until the contralateral leg120touches down to the ground surface12. Here, a leg120in the stance phase is referred to as a stance leg120ST.

In order to maneuver about the environment10, the robot100includes a sensor system130with one or more sensors132,132a-n(e.g., shown as a first sensor132,132aand a second sensor132,132b). The sensors132may include vision/image sensors, inertial sensors (e.g., an inertial measurement unit (IMU)), force sensors, and/or kinematic sensors. Some examples of sensors132include a camera such as a stereo camera, a scanning light-detection and ranging (LIDAR) sensor, or a scanning laser-detection and ranging (LADAR) sensor. In some examples, the sensor132has a corresponding field(s) of view Fvdefining a sensing range or region corresponding to the sensor132. For instance,FIG.1Adepicts a field of a view FVfor the robot100. Each sensor132may be pivotable and/or rotatable such that the sensor132may, for example, change the field of view FVabout one or more axis (e.g., an x-axis, a y-axis, or a z-axis in relation to a ground plane).

In some implementations, the sensor system130includes sensor(s)132coupled to a joint J. In some examples, these sensors132couple to a motor that operates a joint J of the robot100(e.g., sensors132,132a-b). Here, these sensors132generate joint dynamics134,134JDin the form of joint-based sensor data134. Joint dynamics134JDcollected as joint-based sensor data134may include joint angles (e.g., an upper member122Urelative to a lower member122L), joint speed (e.g., joint angular velocity or joint angular acceleration), and/or joint torques experienced at a joint J (also referred to as joint forces). Here, joint-based sensor data134generated by one or more sensors132may be raw sensor data, data that is further processed to form different types of joint dynamics134JD, or some combination of both. For instance, a sensor132measures joint position (or a position of member(s)122coupled at a joint J) and systems of the robot100perform further processing to derive velocity and/or acceleration from the positional data. In other examples, a sensor132is configured to measure velocity and/or acceleration directly.

When surveying a field of view FVwith a sensor132, the sensor system130generates sensor data134(also referred to as image data) corresponding to the field of view FV. In some examples, the sensor data134is image data that corresponds to a three-dimensional volumetric point cloud generated by a three-dimensional volumetric image sensor132. Additionally or alternatively, when the robot100is maneuvering about the environment10, the sensor system130gathers pose data for the robot100that includes inertial measurement data (e.g., measured by an IMU). In some examples, the pose data includes kinematic data and/or orientation data about the robot100, for instance, kinematic data and/or orientation data about joints J or other portions of a leg120of the robot100. With the sensor data134, a perception system200of the robot100may generate maps182for the terrain about the environment10.

While the robot100maneuvers about the environment10, the sensor system130gathers sensor data134relating to the terrain of the environment10and/or structure of the robot100(e.g., joint dynamics and/or odometry of the robot100). For instance,FIG.1Adepicts the sensor system130gathering sensor data134about a room as the environment10of the robot100. As the sensor system130gathers sensor data134, a computing system140is configured to store, to process, and/or to communicate the sensor data134to various systems of the robot100(e.g., the control system170, the perception system180, the odometry system190, and/or the impact detector200). In order to perform computing tasks related to the sensor data134, the computing system140of the robot100includes data processing hardware142and memory hardware144. The data processing hardware142is configured to execute instructions stored in the memory hardware144to perform computing tasks related to activities (e.g., movement and/or movement based activities) for the robot100. Generally speaking, the computing system140refers to one or more locations of data processing hardware142and/or memory hardware144.

In some examples, the computing system140is a local system located on the robot100. When located on the robot100, the computing system140may be centralized (i.e., in a single location/area on the robot100, for example, the body110of the robot100), decentralized (i.e., located at various locations about the robot100), or a hybrid combination of both (e.g., where a majority of centralized hardware and a minority of decentralized hardware). To illustrate some differences, a decentralized computing system140may allow processing to occur at an activity location (e.g., at motor that moves a joint of a leg120) while a centralized computing system140may allow for a central processing hub that communicates to systems located at various positions on the robot100(e.g., communicate to the motor that moves the joint of the leg120).

Additionally or alternatively, the computing system140includes computing resources that are located remotely from the robot100. For instance, the computing system140communicates via a network150with a remote system160(e.g., a remote server or a cloud-based environment). Much like the computing system140, the remote system160includes remote computing resources such as remote data processing hardware162and remote memory hardware164. Here, sensor data134or other processed data (e.g., data processing locally by the computing system140) may be stored in the remote system160and may be accessible to the computing system140. In some examples, the computing system140is configured to utilize the remote resources162,164as extensions of the computing resources142,144such that resources of the computing system140may reside on resources of the remote system160.

In some implementations, as shown inFIGS.1A and1B, the robot100includes a control system170and a perception system180. The perception system180is configured to receive the sensor data134from the sensor system130and to process the sensor data134into maps182. With the maps182generated by the perception system180, the perception system180may communicate the maps182to the control system170in order perform controlled actions for the robot100, such as moving the robot100about the environment10. In some examples, by having the perception system180separate from, yet in communication with the control system170, processing for the control system170may focus on controlling the robot100while the processing for the perception system180focuses on interpreting the sensor data134gathered by the sensor system130. For instance, these systems170,180execute their processing in parallel to ensure accurate, fluid movement of the robot100in an environment10.

In some examples, the control system170includes at least one controller172, a path generator174, a step locator176, and a body planner178. The control system170may be configured to communicate with at least one sensor system130and any other system of the robot100(e.g., the perception system180, the odometry system190, and/or the impact detector200). The control system170performs operations and other functions using hardware140. The controller172is configured to control movement of the robot100to traverse about the environment10based on input or feedback from the systems of the robot100(e.g., the control system170, the perception system180, the odometry system190, and/or the impact detector200). This may include movement between poses and/or behaviors of the robot100. For example, the controller172controls different footstep patterns, leg patterns, body movement patterns, or vision system sensing patterns.

In some examples, the controller172includes a plurality of controllers172where each of the controllers172has a fixed cadence. A fixed cadence refers to a fixed timing for a step or swing phase of a leg120. For example, the controller172instructs the robot100to move the legs120(e.g., take a step) at a particular frequency (e.g., step every 250 milliseconds, 350 milliseconds, etc.). With a plurality of controllers172where each controller172has a fixed cadence, the robot100can experience variable timing by switching between controllers172. In some implementations, the robot100continuously switches/selects fixed cadence controllers172(e.g., re-selects a controller170every 3 milliseconds) as the robot100traverses the environment10.

Referring toFIG.1B, the path generator174is configured to determine horizontal motion for the robot100. For instance, the horizontal motion refers to translation (i.e., movement in the X-Y plane) and/or yaw (i.e., rotation about the Z-direction axis AZ) of the robot100. The path generator174determines obstacles within the environment10about the robot100based on the sensor data134. The path generator174communicates the obstacles to the step locator176such that the step locator176may identify foot placements for legs120of the robot100(e.g., locations to place the distal ends124of the legs120of the robot100). The step locator176generates the foot placements (i.e., locations where the robot100should step) using inputs from the perceptions system180(e.g., map(s)182). The body planner178, much like the step locator176, receives inputs from the perceptions system180(e.g., map(s)182). Generally speaking, the body planner178is configured to adjust dynamics of the body110of the robot100(e.g., rotation, such as pitch or yaw and/or height of COM) to successfully move about the environment10.

The perception system180is a system of the robot100that helps the robot100to move more precisely in a terrain with various obstacles. As the sensors132collect sensor data134for the space about the robot100(i.e., the robot's environment10), the perception system180uses the sensor data134to form one or more maps182for the environment10. Once the perception system180generates a map182, the perception system180is also configured to add information to the map182(e.g., by projecting sensor data134on a preexisting map) and/or to remove information from the map182.

Referring further toFIG.1B, the odometry system190is configured to measure where the robot100is located within a world reference frame (e.g., the environment10) and how fast the robot100is moving in that world reference frame. In other words, the odometry system190generates odometry information192as one or more estimations (e.g., measurements) for a characteristic of the robot100relative to a world reference frame. In some examples, the odometry system190receives sensor data134from a sensor132such as an IMU (e.g., accelerometer(s) and/or gyro(s)). With the sensor data134, the odometry system190may generate odometry information192based on an assumption that when a distal end124of a leg120is in contact with the ground surface12and not slipping, the distal end124is stationary. By combining this assumption with the sensor data134, the odometry system190generates odometry information192regarding robot motion relative to the world reference frame (e.g., the environment10). In other words, the odometry system190accounts for kinematics and inertial measurements to produce estimations about the robot100with respect to the world reference frame.

The impact detection system200of the robot100is configured to receive inputs from other systems of the robot100(e.g., the sensor system130, the computing system140, the remote system160, the control system170, the perception system180, and/or the odometry system190). By capitalizing on information from the other systems of the robot100, the impact detection system200attempts to make an informed decision to identify an impact202, classify the impact202, and, in some instances, generate a response204to the impact202(e.g., invoke a trip response). In some examples, the impact detection system200receives joint dynamics134JDas inputs. For example, the impact detection system200receives joint forces134JD,134JDa to enable the impact detection system200to detect when a distal end124(also referred to as a foot124of the robot100) contacts the ground surface12. In some examples, the impact detection system200receives joint angles134JD,134JDb to detect when the distal end124stops moving and/or a location of the distal end124relative to the ground surface12(e.g., as perceived by the robot100). In some implementations, the impact detection system200receives angular velocities134JD,134JDc to detect a velocity of the distal end124(e.g., to detect whether the foot has stopped moving relative to the ground). AlthoughFIG.2Adepicts the impact detection system200receiving each of the joint forces, joint angles, and angular velocities134JDa-c, some types of detection by the impact detection system200may use more or less inputs.

In some configurations, in addition to the joint dynamics134JD, the impact detection system200receives odometry192of the robot100. The odometry enables the impact detection system200to determine estimations for dynamics of the structure of the robot100by accounting for both kinematics of a world reference frame (e.g., the robot100with respect to the environment10) and kinematics of a relative reference frame (e.g., the body110or legs120of the robot100with respect to the robot100itself). For example, a velocity of a foot124of the robot100is equal to a velocity of the body110in the world reference frame (e.g., as determined by the odometry system190) plus a velocity of the foot124relative to the body110(e.g., as sensed by the sensor system130).

Additionally or alternatively, the impact detection system200receives a map182from the perception system180(e.g., in addition to joint dynamics134JDand/or odometry192). In some examples, the impact detection system200accounts for failures of the perception system180to understand trip conditions or traction surface contact. Yet in some implementations, maps182of the perception system180allow the impact detection system200to classify a type of trip. For example, it is more likely that the foot124touched down against the ground surface when a map182perceived the ground surface as near to the foot124than if the map182did not perceive the foot124in the vicinity of the ground surface12.

Referring toFIG.2A, the impact detection system200includes a detector210. The detector210is configured to identify unexpected force(s) F (or torques T) on one or more legs120(e.g., feet) of the robot100. When the impact detection system200identifies an unexpected force F (or torque T), the impact detection system200determines whether the unexpected force F corresponds to an impact202. During movement, a swing leg120SWtypically generates a minimal amount of torque T. This minimal amount of torque T is generally predictable by inverse dynamics. In other words, based on inverse dynamics and gravity, there is an expected amount of torque T that acts on the swing leg120SW. This expected amount of torque T is equal to a torque contributed by gravity (i.e., gravitational torque Tg) in combination with a torque T that is expected in order to accelerate the leg120to achieve gait motion (i.e., torque from inverse dynamics TID). Because a majority of the torque T experienced by a swing leg120SWis known, the detector210is configured to determine an unexpected torque T (also referred to as a compensated torque Tcomp). For example, Equation (1) below illustrates an equation that the detector210uses to determine the compensated torque Tcom.
Tcomp=Tmeasured−Tg−TID(1)
Here, Tmeasuredis the torque measured from joint sensor(s)132. In other words, the detector210is configured to receive and/or to determine a measured amount of torque Tmeasuredacting on the swing leg120SWbased on sensor data134(e.g., from sensors132at joints J of the robot100) from the sensor system130. With the measured amount of torque Tmeasuredand the expected torques T (i.e., the gravitational torque Tgand the torque from inverse dynamics TID), the detector210determines the compensated torque Tcomp. Based on the compensated torque Tcomp, the detector210is configured to identify when the swing leg120SWexperiences an impact202.

In some implementations, the detector210determines whether the torque T on the swing leg120SWcorresponds to an impact202on the swing leg120SWby determining whether the compensated torque Tcompsatisfies an impact torque threshold Tth(e.g., the detector210determines that the compensated torque Tcompexceeds a value set as the impact torque threshold Tth). Here, the impact torque threshold TTHis configured to represent how much noise the impact detection system200expects during impact detection. In other words, the detector210may be configured with an impact torque threshold TTHto ensure that the detector210generally does not falsely detect an impact202based on sensor noise. When the compensated torque Tcompsatisfies the impact torque threshold Tth, the detector210communicates that an impact202(or a potential impact202) has occurred on the swing leg120SW. Conversely, when the compensated torque Tcompfails to satisfy the impact torque threshold Tth, the detector210continues monitoring the torque T on the swing leg120SW.

In some examples, the detector210monitors the torque T on each joint J of the swing leg120SW. For instance, the detector210monitors the torque T at the knee joint JK, the abduction-adduction component of the hip joint JHx, and the flexion-extension component of the hip joint JHy. In some implementations, for each joint J, the detector210generates evidence212corresponding to a potential impact. Here, as shown inFIG.2A, the evidence212refers to an area under a curve representing the compensated torque TCOMPthat is greater than the impact torque threshold TTH. For instance, the detector210generates the evidence212as the integral under the curve representing the compensated torque TCOMP, but above the impact torque threshold TTH. As shown inFIG.2A, to determine the evidence212, the detector210integrates the absolute value of the compensated torque TCOMP.

When the detector210generates evidence212, the detector210may be further configured to remove evidence212when the value of the compensated torque TCOMPindicates an impact202has not occurred. In some examples, the curve representing the compensated torque TCOMPwill exceed the impact torque threshold TTH, but then drop to a value below the impact torque threshold TTH. When this drop below the impact torque threshold TTHoccurs, the detector210removes the evidence212to reduce the likelihood that the impact detection system200identifies a false impact. Similarly, in some implementations, the value of the compensated torque TCOMPindicates an impact202has not occurred because the value of the compensated torque TCOMPchanges signs (e.g., from positive to negative or vice versa). Here, the detector210clears any evidence212prior to the sign change because the impact detection system200is configured to identify an impact202as a having the same sign (e.g., based on the integral of the absolute value of the compensated torque TCOMP). Therefore, a change in sign indicates a potential false impact.

In some examples, the detector210determines that an impact202occurs on the swing leg120SWwhen a sum of the evidence212for all joints J of a leg120satisfy an evidence threshold212TH. Here, the evidence threshold212THrefers to a required amount of evidence that the swing leg120SWexperiences overall across all joints J that indicates an impact202. AlthoughFIG.2Adepicts the detector210determining whether the sum of the evidence212for all joints J of a leg120satisfies an evidence threshold212TH, other configurations of the detector210may perform this determination for less than all joints J of a leg120(e.g., for the knee joint JKand one component of the hip joint JHor both components of the hip joint JHwithout knee joint JK).

In some configurations, the evidence threshold212THdepends on the swing phase. For instance, the impact detection system200is configured to have greater sensitivity when a touchdown of the swing leg120SWis expected than when a touchdown of the swing leg120SWis not expected. Based on this, the evidence threshold212THmay be scaled or modified based on a percent completion of the swing phase. Here, other systems of the robot100may communicate the percent completion of the swing phase to the impact detector system200. For example, the control system170communicates the percent completion of the swing phase to the impact detection system200because the percent completion may be derived based on a controller172performing the gait cycle for the swing leg120SWhaving a fixed cadence. In some examples, the impact detection system200is configured to determine the percent completion of the swing phase based on a current location of the swing leg120SW(e.g., as determined by sensor data134) and a controller172operating the swing leg120SW.

Referring toFIG.2B, in some implementations, the detector210is configured to perform further analysis on a potential impact or an impact202. In other words, there may be circumstances where although the required evidence212THindicates an impact202, the actual circumstances of the detected impact202may be analyzed to confirm whether the detected impact202is an actual impact. In some examples, such asFIG.2B, the structure of a leg120of the robot100includes end stops126that restrict a range of motion of the leg120. For instance, the range of motion is limited by an end stop126to prevent damage to the structure of the leg120or based on limitations of one or more motors that articulate the leg120. When an end stop restricts126a range of motion of the leg120, the joint sensors132of the leg120may experience a sudden measured torque caused by a force F exerted by the end stop126. Even though this force F from the end stop126generates a correct sign for the torque T, the detector210is configured to ignore an impact202that occurs subsequent to when the detector210identifies the swing leg120SWis near an end stop126.

Referring toFIGS.2C-2E, the detector210may be configured to ensure that a torque T causing a potential impact202always opposes motion of the leg120(e.g., motion of a foot124of the swing leg120SW). In other words, the impact detection system200does not classify a condition as an impact202when the condition results in a measured torque that speeds up an associated joint J of the leg120(e.g., speeds up the leg120itself).FIGS.2C-2Eillustrate conditions where the detector210determines whether a torque T corresponding to an impact202opposes motion of the leg120(e.g., shown as motion of the foot124of the leg120). In each example, the foot124is moving in a direction of travel designated DT. Referring toFIG.2C, the foot124is traveling to the left such that the detector210only considers a first torque T, T1at the knee joint JK and not a second torque T, T2at the knee joint JKbecause the second torque T2contributes to the motion of the foot124. InFIG.2D, the direction of travel DTfor the foot124is downward. Since a torque T in either direction may include opposing this downward motion, the detector210is configured to consider acting torque T in either direction (e.g., the first torque T1or the second torque T2) when the potential torque contribution is unclear. InFIG.2E, the direction of travel DTfor the foot124is mostly downward, but with a directional component moving to the right. Here, the detector210may apply a weight to torques T in each direction such that the detector210requires a larger torque T in a direction that opposes the directional component of the foot124(e.g., the second torque T2) to ensure that the torque T corresponds to an impact202(or vice versa).

In some examples, the detector210identifies the velocity of the foot124and then determines whether a potential impact torque T increases or decreases the velocity of the foot124. When the potential impact torque T contributes to an increase in the velocity of the foot124, the detector210ignores this impact202(or potential impact). On the other hand, when the potential impact torque T contributes to a decrease in the velocity of the foot124, the detector210may permit this impact202to be detected as an actual impact (e.g., subject to other criteria of the detector210).

Referring toFIGS.2A and2F, when the detector210detects an impact202, the detector210is configured to communicate that impact202to a classifier220of the impact detection system200. The classifier220is configured to determine a condition of the impact202. In other words, what type of impact202occurred or is occurring to the leg120of the robot100and, depending on the type of impact202, generate a response204. In some examples, the classifier220communicates the response204to the other systems of the robot100(e.g., the control system170). When the control system170receives the response204, a controller172of the control system170may generate an appropriate movement to overcome the impact202(e.g., stabilize the robot100). In some examples, the classifier220is configured in a hierarchy such that the classifier220attempts to classify certain types of impacts202before other types of impacts202(e.g., touchdown contact before a trip condition).

In some examples, the classifier220utilizes characteristics of the distal end124(i.e., the foot124) of the leg120that experiences the impact202to classify the impact202. More particularly, the classifier220may use the position of the foot124in the world (e.g., derived from some combination of joint dynamics134JDand odometry information192), a velocity of the foot124in the world (e.g., derived from some combination of joint dynamics134JDand odometry information192), and forces on the foot124in the world frame. Here, the forces on the foot124in the world frame may be derived in two different ways. In a first approach, the classifier220may determine the forces in the world frame by deriving the forces from joint torques T of the joint dynamics134JDbased on Jacobian transformation. In a second approach, classifier220uses inverse dynamics to generate the force on the foot124in the world frame. In some examples, the classifier220determines the force on the foot124based on both the first approach and the second approach, but uses whichever approach generates a force on the foot124in the world frame closer to zero to perform classification. In some implementations, when classifying a type of impact202, the classifier220associates an importance to the dynamics of the foot124. For instance, the force on the foot124is the most important characteristic, followed by the velocity of the foot124, and followed by the position of the foot124as the least important for classification purposes.

Referring toFIG.2F, in some examples, the classifier220first determines whether the impact202corresponds to a touchdown of the swing leg120SW. To classify the impact202as a touchdown, the classifier220generally determines whether a distal end124of the leg120experiences significant vertical forces (e.g., indicating a normal force from the ground surface12) and/or whether the velocity of the distal end124is no longer moving downward even though the control system170is instructing the distal end124to move downward. Here, the classifier220may be configured with a vertical force threshold that indicates a significant magnitude of a vertical force (e.g., that risks destabilization of the robot100). In other words, when the classifier220determines that a vertical force on the distal end124satisfies the vertical force threshold (e.g., exceeds a value set as the vertical force threshold), the classifier220classifies the impact202on the swing leg120SWas a touchdown. In some implementations, the classifier220modifies the vertical force threshold based on certain circumstances. For instance, the classifier220modifies the vertical force threshold when the robot100(e.g., via the control system170) is responding to a trip (e.g., the impact detection system200generated a response204to an impact202). In another examples, the classifier220modifies the vertical force threshold based on the swing phase (e.g., a percent completion of the swing phase). To illustrate, the classifier220may modify the vertical force threshold when the classifier220is aware the swing leg120SWis early in the swing phase (e.g., shortly after lift-off). Additionally or alternatively, when the classifier220identifies the impact202as a standard trip, but the robot100is unable to elevate the foot124as a response204to the standard trip, the classifier220classifies the impact202instead as a touchdown.

In some configurations, when the swing leg120SWis near an end of completion for the swing phase, the classifier220classifies any new impact202as a touchdown. For example, when a percent completion of the swing phase is greater than or equal to 95% complete, the classifier220classifies an impact202as a touchdown. Here, this condition protects the robot100from risking destabilization by a sudden undetected touchdown or other issues that may result from late detection of touchdown.

In some examples, the classifier220classifies the impact202as touchdown when the swing leg120SWis late in the swing phase and a planned trajectory for the swing leg120SWis moving downward toward the ground surface12. For example, when a percent completion of the swing phase is about greater than or equal to 65% complete and the planned trajectory for the swing leg120SWis moving downward toward the ground surface12, the classifier220classifies an impact202as a touchdown.

In these examples, where the swing leg120SWis late in the swing phase and the planned trajectory for the swing leg120SWis moving downward toward the ground surface12, there may be additional criterial, but alternative criteria that the classifier220uses to classify an impact202as a touchdown. A first additional criteria may be that the force on the foot124is significantly large (e.g., satisfies the vertical force threshold or some version thereof). A second additional criterial may be that an impulse experienced by the foot124is quite large. For example, much like the vertical force threshold, the classifier220is configured with an impulse threshold that indicates an amount of force over time experienced by the foot124that indicates a touchdown (e.g., an impulse on the foot124exceeds the impulse threshold configured at the impact detection system200). A third additional criteria may be that a measured velocity of the foot124is not as downward (in the z-direction towards the ground surface12) as intended (e.g., by the control system170). For example, the change in position of the foot124is greater than a threshold velocity of the foot124(e.g., greater than −0.5 m/s) such that although the control system170intended the foot124to be moving downwards in the z-direction towards the ground surface12, the foot124is actually moving in decreasing speed towards the ground surface12.

Referring further toFIG.2F, when the impact202does not correspond to a touchdown of the swing leg120SW, the classifier220is configured to identify whether the impact202was caused by a special type of trip condition222,222SP. Some examples of special trip conditions222SPinclude the following: when the thigh122Uof the leg120is on the body110; when the foot124experiences liftoff scuffing; when the swing leg120SWcrosses a contralateral leg120CL(FIGS.2G and2H); when the swing leg120SWstrikes the contralateral leg120CL(FIG.21); when the knee of the robot100is on terrain behind the robot100(FIG.2J); and when the knee of the leg120is on terrain under the robot100(FIG.2K). In some implementations, the classifier220identifies special trip condition222SPbased on joint dynamics134JDand/or odometry information192of the robot100. When the classifier220identifies a special type of trip condition222SP, depending on the type of trip, the classifier220(i) may generate a response204to overcome the trip and/or to reduce the effects of the trip. When the classifier220determines that the impact202was not caused by a special type of trip condition222SP, the classifier220classifies the impact202as caused by a standard trip condition222,222STD(e.g., shown inFIG.2L). For a standard trip condition222STD, the classifier220generates a response204to elevate the foot124that experienced the impact202(i.e., to increase the height of the foot124in the z-direction).FIG.2Lillustrates a standard trip condition222STDfor the robot100.

In some configurations, when the classifier220determines that the impact202was not caused by a touchdown of the swing leg120SWand the swing leg120SWis early in the swing phase (e.g., less than 10% complete with the swing phase), the classifier220classifies the impact202as a liftoff scuffing. In some examples, the classifier220determines that a percent completion of the swing phase satisfies a swing phase threshold (e.g., is less than the swing phase threshold) that indicates that the swing leg120SWis early in the swing phase. Here, a liftoff scuffing refers to when the swing leg120SWcontacts the ground surface as the swing leg120SWlifts-off the ground surface12(e.g., the robot100stubs its foot124as it takes off). In this special trip condition222SP, the swing leg120SWwill continue lifting off the ground surface12and thus clear the ground surface12that caused the impact202. Therefore, the classifier220does not generate a response204based on this liftoff scuffing because the swing leg120SWdoes not risk any resistance as a result of the impact202.

In some configurations, the classifier220determines that a position of the thigh (i.e., upper member122Lof the leg120) on the body110of the robot100caused the detected impact202. Here, this special trip condition222SPoccurs when the joint dynamics134JDindicate that the abduction-adduction component of the hip joint JHxexceeds a predetermined limit. Generally speaking, each component of the hip joint JHx,zmay be configured with a predetermined limit. For instance, the control system170or another system of the robot100may set the predetermined limits based on the dimensions of the robot100and/or a payload of the robot100. In some examples, the abduction-adduction component of the hip joint JHxhas a predetermined limit that is a function of the flexion-extension component of the hip joint JHy. In other words, when the abduction/adduction exceeds the predetermined limit for the abduction-adduction component of the hip joint JHx, the upper member122Uof the leg120risks collision with the body110of the robot100. Much like the special trip condition222SPof liftoff scuffing, the classifier220does not generate a response204based on this type of impact202and the swing leg120SWcontinues performing the swing phase of the gait.

Referring toFIGS.2G and2H, in some examples, the classifier220determines that the swing leg120SW(e.g., shown as a first leg120a) crossed the contralateral leg120CL(e.g., shown as a second leg120b,120ST) to cause the impact202. Here, a contralateral leg120CLrefers to a relationship between two legs120a-bsuch that the first leg120ais opposite the sagittal plane (i.e., the Y-Z plane) from the second leg120b. For instance, with a quadruped robot (as shown) the stance leg120STis generally a contralateral leg120CLto the swing leg120SW(and vice versa) to maintain balance for the robot100during movement. In some examples, the classifier220determines this type of special trip condition222SPby determining whether the swing leg120SWis near a contralateral leg120CLbased on kinematics of the robot100. In these examples, determining whether the swing leg120SWis near a contralateral leg120CLmay include determining a vector distance between a first closest point of the swing leg120SWto the contralateral leg120CLand a second closest point of the contralateral leg120CLto the swing leg120SW. Here, when the vector distance indicates the swing leg120SWcrossed the contralateral leg120CL, the classifier220may additionally determine whether the feet124a-bof each leg120SW,120CLare crossed and/or whether the knee joints JK, JKa-b of each leg120SW,120CLare crossed. In some configurations, the classifier220is configured to determine that one or more of these conditions are true (e.g., one is true, two are true, or all three are true). In other words, the classifier220determines whether (i) the vector distance indicates the legs120SW,120CLare crossed, (ii) the position of the knee joints JK, JKa-b indicates that the legs120SW,120CLare crossed, or (iii) the position of the feet124a-bindicates that the legs120SW,120CLare crossed. When the swing leg120SWcrossing the contralateral leg120CLresults in an impact202, the classifier220may generate a response204to uncross the legs120SW,120C. For instance, the classifier220moves the knee joint JKof the swing leg120SWtowards a vertical axis in the z-direction of the hip joint JHof the swing leg120SW.

Referring toFIG.21, in some implementations, the classifier220determines that the swing leg120SWis near a contralateral leg120CL, but not crossed over the contralateral leg120CL(e.g., based on kinematics of the robot100). Here, the classifier220may determine the swing leg120SWcontacted the contralateral leg120CLcausing the impact202, but did not cross over the contralateral leg120CLduring, before, or shortly thereafter the impact202. In some examples, the classifier220forms this determination by identifying (i) that the swing leg120SWis not too late in the swing phase (e.g., less than 65% complete) and (ii) that the evidence212, which the detector210used to determine the impact202, was predominantly from the abduction-adduction component of the hip joint JHx. In other words, the torque of lateral rotation of the hip joint JHxwas a greater contributor to the evidence212identifying the impact202than torques of the flexion-extension component of the hip joint JHyand the knee joint JK(e.g., each separately or combined together). When the swing leg120SWcollides with the contralateral leg120CLcausing the impact202, the classifier220may generate a response204to separate the legs120SW,120CL(e.g., by moving one or more legs120SW,120CLaway from each other in the frontal plane). In other words, swing leg120SWmay continue its originally planned motion within the sagittal plane while overriding the originally planned motion in the frontal plane (e.g., along the x-axis AX) to separate from the stance leg120ST.

Referring toFIGS.2K-2L, in some configurations, the classifier220determines that a collision between the leg120(e.g., the swing leg120SW) and terrain either behind (e.g., shown inFIG.2J) or beneath (e.g., shown inFIG.2K) the body110of the robot100causes the impact202. In the case of a special trip condition222SPwhere the impact202occurs based on contact between a knee joint JKof the swing leg120SWand the terrain behind the robot100, the classifier220determines (i) that the knee joint JKof the swing leg120SWis moving backwards away from the body110(e.g., in the y-direction) and (ii) that the evidence212indicates that the flexion-extension component of the hip joint JHyfor the swing leg120SWis the largest contributor of joint torque. In some examples, the flexion-extension component of the hip joint JHyfor the swing leg120SWis larger than the joint torque T contribution of both the knee joint JKand the abduction-adduction component of the hip joint JHx(e.g., separately or in combination). In some implementations, the classifier220uses the kinematics of the swing leg120SWto determine that the knee joint JKof the swing leg120SWis moving backwards away from the body110. When the classifier220classifies the impact202as a collision between the leg120and the terrain behind the robot100, the classifier220generates a response204to move bring the knee joint JKforward (e.g., towards the CM of the robot100along the y-direction).

Referring toFIG.2L, here, the classifier220determines that a collision between the leg120and the terrain beneath the body110of the robot100caused the impact202. To determine this special trip condition222SP, the classifier220determines that the knee joint JKis moving forward (e.g., forward in the y-direction with respect to the hip joint JHof the swing leg120SW) and that the evidence212producing the impact202occurred in joints J other than the knee joint JK. Additionally or alternatively, the classifier220may use one or more maps182of the perception system180to determine that the knee joint JKof the swing leg120SWis closer to the ground surface12than the foot124of the swing leg120SW. The map182may also indicate that the knee joint JKof the swing leg120SWwas recently perceived near the ground surface12. In this special trip condition222SP, the classifier220is configured to not generate a response204because the forward motion of the swing leg120SWby continuing the swing phase of the gait will carry the knee joint JKpast the object causing the impact202.

In some examples, the impact detection system200may be advantageous when the perception system180is either off or not functioning well (e.g., issues with sensor(s)132). When this occurs, the impact detection system200may be used as a primary system for detection within the environment10. Conversely, when the perception system180is working properly, the impact detection system200may offer a robust auxiliary system for detection, especially in instances when the perception system180makes a mistake or encounters a temporary issue.

FIG.3is an arrangement of operations to perform a method300of footstep contact detection. At operation302, the method300receives joint dynamics134JDfor a swing leg120SWof a robot100. Here, the swing leg120SWperforms a swing phase of a gait of the robot100. At operation304, the method300receives odometry192that defines an estimation of characteristics (e.g., a pose) of the robot100relative to a world reference frame. At operation306, the method300determines whether a torque T on the swing leg120SWcorresponds to an impact202on the swing leg120SW. When the torque T on the swing leg120SWcorresponds to the impact202, at operation308,308a, the method300determines whether the impact202is indicative of a touchdown of the swing leg120SWon a ground surface12about the robot100based on the odometry192of the robot100and the joint dynamics134JDof the swing leg120SW. At operation308,308b, when the impact202is not indicative of the touchdown of the swing leg120SW, the method300classifies a cause of the impact202based on the odometry192of the robot100and the joint dynamics134JDof the swing leg120SW.

FIG.4is schematic view of an example computing device400that may be used to implement the systems (e.g., the sensor system130, the computing system140, the remote system160, the control system170, the perception system180, the odometry system190, and/or the impact detection system200) and methods (e.g., method300) described in this document. The computing device400is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The components shown here, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed in this document.

The computing device400includes a processor410(e.g., data processing hardware), memory420(e.g., memory hardware), a storage device430, a high-speed interface/controller440connecting to the memory420and high-speed expansion ports450, and a low speed interface/controller460connecting to a low speed bus470and a storage device430. Each of the components410,420,430,440,450, and460, are interconnected using various busses, and may be mounted on a common motherboard or in other manners as appropriate. The processor410can process instructions for execution within the computing device400, including instructions stored in the memory420or on the storage device430to display graphical information for a graphical user interface (GUI) on an external input/output device, such as display480coupled to high speed interface440. In other implementations, multiple processors and/or multiple buses may be used, as appropriate, along with multiple memories and types of memory. Also, multiple computing devices400may be connected, with each device providing portions of the necessary operations (e.g., as a server bank, a group of blade servers, or a multi-processor system).

The memory420stores information non-transitorily within the computing device400. The memory420may be a computer-readable medium, a volatile memory unit(s), or non-volatile memory unit(s). The non-transitory memory420may be physical devices used to store programs (e.g., sequences of instructions) or data (e.g., program state information) on a temporary or permanent basis for use by the computing device400. Examples of non-volatile memory include, but are not limited to, flash memory and read-only memory (ROM)/programmable read-only memory (PROM)/erasable programmable read-only memory (EPROM)/electronically erasable programmable read-only memory (EEPROM) (e.g., typically used for firmware, such as boot programs). Examples of volatile memory include, but are not limited to, random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), phase change memory (PCM) as well as disks or tapes.

The storage device430is capable of providing mass storage for the computing device400. In some implementations, the storage device430is a computer-readable medium. In various different implementations, the storage device430may be a floppy disk device, a hard disk device, an optical disk device, or a tape device, a flash memory or other similar solid state memory device, or an array of devices, including devices in a storage area network or other configurations. In additional implementations, a computer program product is tangibly embodied in an information carrier. The computer program product contains instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer- or machine-readable medium, such as the memory420, the storage device430, or memory on processor410.

The high speed controller440manages bandwidth-intensive operations for the computing device400, while the low speed controller460manages lower bandwidth-intensive operations. Such allocation of duties is exemplary only. In some implementations, the high-speed controller440is coupled to the memory420, the display480(e.g., through a graphics processor or accelerator), and to the high-speed expansion ports450, which may accept various expansion cards (not shown). In some implementations, the low-speed controller460is coupled to the storage device430and a low-speed expansion port490. The low-speed expansion port490, which may include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet), may be coupled to one or more input/output devices, such as a keyboard, a pointing device, a scanner, or a networking device such as a switch or router, e.g., through a network adapter.

The computing device400may be implemented in a number of different forms, as shown in the figure. For example, it may be implemented as a standard server400aor multiple times in a group of such servers400a, as a laptop computer400b, or as part of a rack server system400c.

Various implementations of the systems and techniques described herein can be realized in digital electronic and/or optical circuitry, integrated circuitry, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

These computer programs (also known as programs, software, software applications or code) include machine instructions for a programmable processor, and can be implemented in a high-level procedural and/or object-oriented programming language, and/or in assembly/machine language. As used herein, the terms “machine-readable medium” and “computer-readable medium” refer to any computer program product, non-transitory computer readable medium, apparatus and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term “machine-readable signal” refers to any signal used to provide machine instructions and/or data to a programmable processor.

The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read only memory or a random access memory or both. The essential elements of a computer are a processor for performing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto optical disks, or optical disks. However, a computer need not have such devices. Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto optical disks; and CD ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

To provide for interaction with a user, one or more aspects of the disclosure can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube), LCD (liquid crystal display) monitor, or touch screen for displaying information to the user and optionally a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer. Other kinds of devices can be used to provide interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input. In addition, a computer can interact with a user by sending documents to and receiving documents from a device that is used by the user; for example, by sending web pages to a web browser on a user's client device in response to requests received from the web browser.

A number of implementations have been described. Nevertheless, it will be understood that various modifications may be made without departing from the spirit and scope of the disclosure. Accordingly, other implementations are within the scope of the following claims.