Positioning system having a master-slave configuration

A positioning system for generating pose information of a machine includes a primary positioning controller and a plurality of local positioning controllers coupled to the primary positioning controller. The primary positioning controller is configured to generate primary pose information of the machine. The primary pose information reflects an overall pose of the machine. Each of the local positioning controllers is configured to generate local pose information based on at least the primary pose information. The local pose information reflects a local pose of a portion of the machine corresponding to the local positioning controller.

TECHNICAL FIELD

The present disclosure relates to a positioning system and, more particularly, to a positioning system having a master-slave configuration.

BACKGROUND

Excavation machines such as backhoe loaders, haul trucks, wheel loaders, scrapers, and other types of heavy equipment, are used to perform a variety of tasks. Some of these tasks involve carrying large, awkward, loose, and/or heavy loads along rough and crowded roadways. And because of the size of the machines and/or poor visibility provided to operators of the machines, these tasks can be difficult to complete effectively. For this reason, some machines are equipped with perception systems that provide views of a machine's environment to the operator.

Conventional perception systems include one or more perception sensors, such as LIDAR (light detection and ranging) devices, that capture different images, which are then combined to form a surround view. To combine the images, poses of the perception sensors need to be determined. For this purpose, a rate sensing device can be used to provide pose information at the location of each perception sensor. The pose information may include for example, one or more of position, orientation, linear velocity, angular velocity, and acceleration. The rate sensing device may include one or more rate sensors and may be, for example, an inertial measurement unit (IMU) or a visual odometry device. Such a perception system could measure and compensate for, e.g., vibration or movement of the perception sensors, to provide better results for the perception.

The rate sensing devices, however, are not ideal, and may have noises, bias, or drifts due to, for example, aging and temperature. The noises, bias, and drifts of the rate sensing devices may accumulate, affecting the accuracy of the determined pose of the perception sensors. In conventional technology, a state update source is used to update the readings of the rate sensing devices. The state update source can be a global navigation satellite system (GNSS), such as a global positioning system (GPS). The state update source can also be another system that is capable of providing state update, such as a pseudolite system, a perception based positioning system, a ranging radio system, a speedometer, an inclinometer, or an accelerometer. Such a configuration is not only used in perception systems, but may also be used in other systems using rate sensing devices, such as IMUs.

U.S. Pat. No. 8,457,891 of Vallot et al., which issued on Jun. 4, 2013 (the '891 patent), discloses a navigation system using an IMU for navigating a vehicle and a GPS to correct accumulating errors from the IMU. However, in such a conventional perception system, when GPS signal is not available, the readings of the IMUs cannot be updated. In this scenario, local disturbances may affect the result of the perception.

The disclosed system is directed to overcoming one or more of the problems set forth above and/or other problems of the prior art.

SUMMARY

In one aspect, the present disclosure is directed to a positioning system for generating pose information of a machine. The positioning system includes a primary positioning controller and a plurality of local positioning controllers coupled to the primary positioning controller. The primary positioning controller is configured to generate primary pose information of the machine. The primary pose information reflects an overall pose of the machine. Each of the local positioning controllers is configured to generate local pose information based on at least the primary pose information. The local pose information reflects a local pose of a portion of the machine corresponding to the local positioning controller.

In another aspect, the present disclosure is directed to a method for generating pose information of a machine. The method includes generating primary pose information of the machine and generating local pose information based on at least the primary pose information. The primary pose information reflects an overall pose of the machine. The local pose information reflects a local pose of a portion of the machine.

In yet another aspect, the present disclosure is directed to a perception system for providing a surround view of a machine. The perception system includes a plurality of perception sensors and a positioning system coupled to the perception sensors. The positioning system includes a primary positioning controller and a plurality of local positioning controllers. The primary positioning controller is configured to generate primary pose information of the machine. The primary pose information reflects an overall pose of the machine. Each of the local positioning controllers is coupled to the primary positioning controller and one of the perception sensors, and is configured to generate local pose information based on at least the primary pose information. The local pose information reflects a local pose of the corresponding perception sensor.

DETAILED DESCRIPTION

FIG. 1Aillustrates an exemplary perception system100consistent with embodiments of the present disclosure. Perception system100includes a plurality of perception sensors102, which, as shown inFIG. 1B, can be mounted on outside surfaces of a machine104to provide a surround view of machine104. Perception sensors102may include devices that are capable of providing scene data describing an environment in the vicinity of machine104, such as detecting and ranging objects located around machine104. For example, each perception sensor102may be embodied by a LIDAR (light detection and ranging) device, a RADAR (radio detection and ranging) device, a SONAR (sound navigation and ranging) device, a camera device, or another device known in the art, in one example, each perception sensor102may include an emitter that emits a detection beam, and an associated receiver that receives a reflection of that detection beam. Based on characteristics of the reflected beam, a distance and a direction from an actual sensing location of perception sensor102on machine104to a portion of a sensed physical object may be determined. By utilizing beams in a plurality of directions, perception sensors102may generate a picture of the surroundings of machine104. For example, if perception sensors102are embodied by LIDAR devices or other devices using multiple laser beams, perception sensors102may generate a cloud of points as the scene data describing an environment in the vicinity of machine104.

According to the present disclosure, images acquired by perception sensors102are stitched together to thrill the surround view. To ensure the acquired images are properly stitched, local pose information, such as position orientation and velocity, of perception sensors102are needed.

Referring again toFIG. 1A, perception system100further includes a positioning system106consistent with embodiments of the present disclosure. Positioning system106has a “master-slave” configuration and includes a primary positioning controller (also referred to as a “master positioning controller”)108and a plurality of local positioning controllers (also referred to as “slave positioning controllers”)106. The primary positioning controller108may implement a primary Kalman filter (also referred to as a “master Kalman filter”) to provide overall pose information of machine104. The overall pose information is also referred to as primary or master pose information, which includes, for example, the velocity and orientation of machine104. As shown inFIG. 1A, each perception sensor102is coupled to one of the plurality of local positioning controllers110. Each local positioning controller110may implement a local Kalman filter (also referred to as a “slave Kalman filter”) to provide the local pose information to its corresponding perception sensor102. The local pose information includes, for example, the velocity and orientation of corresponding perception sensor102. Details of primary positioning controller108and local positioning controllers110are described below.

As shown inFIG. 1A, positioning system106also includes a primary rate sensing device112and a plurality of local rate sensing devices114. Each of primary rate sensing device112and local rate sensing devices114is configured to provide pose information, such as a rate of change of a position-related parameter, including, for example, linear velocity, angular velocity, or acceleration. Each of primary rate sensing device112and local rate sensing devices114may include one or more rate sensors and may be, for example, an inertial measurement unit (IMU) or a visual odometry device. Specifically, primary rate sensing device112may include devices that provide, for example, angular rates and acceleration of machine104. Similarly, each local rate sensing device114may include devices that provide, for example, angular rates and acceleration of a corresponding perception sensor102. For example, each of primary rate sensing device112and local rate sensing devices114may include a 6-degree of freedom (6 DOF) IMU. A 6 DOF IMU includes a 3-axis accelerometer, 3-axis angular rate gyros, and sometimes a 2-axis inclinometer. The 3-axis angular rate gyros may provide signals indicative of the pitch rate, yaw rate, and roll rate of machine104or corresponding perception sensor102. The 3-axis accelerometer may provide signals indicative of the acceleration of machine104or corresponding perception sensor102in the x, y, and z directions.

According to the present disclosure, primary positioning controller108is coupled to primary rate sensing device112, and receives primary pose incremental information, such as the angular rates and/or acceleration of machine104, from primary rate sensing device112. In some embodiments, primary positioning controller108may also receive other machine wide measurement inputs, such as position and/or velocity information provided by one or more state update sources116and other applicable state information of machine104. A state update source116may be, for example, a GNSS receiver, a pseudolite system, a perception based positioning system, a ranging radio system, a speedometer, or an inclinometer. Similarly, each local positioning controller110is coupled to one of local rate sensing devices114, and receives local pose incremental information, such as the angular rates and/or acceleration, of corresponding perception sensor102, from the corresponding local rate sensing device114.

As described above, each of primary positioning controller108and local positioning controllers106may implement a Kalman filter. The Kalman filter can determine accurate values of measurements observed over time, such as measurements taken in a time series. The Kalman filter's general operation involves two phases, i.e., a propagation or “predict” phase and a measurement or “update” phase. In the predict phase, the value estimate from the previous timestep in the time series is used to generate an a priori value estimate. In the update phase, the a priori estimate calculated in the predict phase is combined with an estimate of the accuracy of the a priori estimate (e.g., the variance or the uncertainty), and a current measurement value to produce a refined, i.e., updated, a posteriori estimate.

FIG. 2Aillustrates an exemplary embodiment of primary positioning controller108. Primary positioning controller108includes a primary pose propagation unit108-1and a primary pose measurement update unit108-2. Primary pose propagation unit108-1may implement the “predict” phase of the primary Kalman filter. Specifically, primary pose propagation unit108-1receives signals (primary pose incremental information) from primary rate sensing device112. Primary pose propagation unit108-1may also receive other signals (other machine wide measurement inputs) reflecting additional machine-wide state information of machine104. By utilizing these signals, primary pose propagation unit108-1may propagate or “predict” certain states of machine104, such as position, linear velocity, angular velocities, and angular orientation (attitude) of machine104. Primary pose propagation unit108-1outputs a propagated primary pose estimate of one or more of above states of machine104to primary pose measurement update unit108-2.

Primary pose measurement update unit108-2may implement the measurement update (or “update”) phase of the primary Kalman filter. In the measurement update phase, an updated primary pose estimate is determined for machine104by updating the propagated primary pose estimate using, e.g., state update information provided by one or more state update sources116. In the scenario where the state update information from state update sources116are not available, for example, if state update sources116are GNSS receivers and lose the connection with GNSS satellites, primary pose measurement update unit108-2may update the propagated primary pose estimate using, for example, previously received state update information. The updated primary pose estimate generated by primary pose measurement update unit108-2is output as the primary pose information to local positioning controllers110.

FIG. 2Billustrates an exemplary embodiment of local positioning controller110. Local positioning controller110includes a local pose propagation unit110-1and a local pose measurement update unit110-2. Local pose propagation unit110-1implements the “predict” phase of the local Kalman filter. Specifically, local pose propagation unit110-1receives signals (local pose incremental information) from local rate sensing device114. By utilizing these signals, local pose propagation unit110-1may propagate or “predict” certain states of corresponding perception sensor102. For example, local pose propagation unit110-1may predict position, forward velocity, angular velocities, and angular orientation (attitude) of corresponding perception sensor102. Local pose propagation unit110-1outputs a propagated local pose estimate of one or more of above states of the corresponding perception sensor102to local pose measurement update unit110-2.

Local pose measurement update unit110-2may implement the measurement update (or “update”) phase of the local Kalman filter. In the measurement update phase, an updated local pose estimate is determined for corresponding perception sensor102by updating the propagated local pose estimate using the primary pose information provided by primary positioning controller108.

Thus, as described above, primary positioning controller108and local positioning controllers110form the “master-slave” configuration, with primary positioning controller108being the “master” and local positioning controllers110being the “slaves.” Primary positioning controller108provides unified updating information (the primary pose information) to update pose estimates of all local positioning controllers110. That is, the pose estimate update process in all local positioning controllers110is “controlled” by the same source, and thus the difference among local pose estimates of different local positioning controllers110can be reduced.

According to the present disclosure, when state update information from state update sources116is available to positioning system106, for example, when GNSS signals are available to positioning system106using GNSS receivers as state update sources116, the propagated primary pose estimate output by primary pose propagation unit108-1can be updated using the state update information from the state update sources116, and therefore primary pose measurement update unit108-2can output a relatively accurate updated primary pose estimate. When state update information from state update sources116is not available to positioning system106, the updated primary pose estimate generated by primary pose measurement update unit108-2may be based on old state update information form state update sources116, and thus may have a relatively low accuracy, i.e., a relatively large error. However, such an error is passed to each of local positioning controllers110. That is, all local positioning controllers110have the same error in the update phase. As a result, although local positioning controllers110may drift over time, they drift in a same direction, and thus the discrepancy among different local positioning controllers110is still small even in the situation where state update information from state update sources116is not available.

In some embodiments, primary rate sensing device112and local rate sensing devices114are identical to each other. In some embodiments, primary rate sensing device112may have a relatively higher quality than local rate sensing devices114. For example, primary rate sensing device112has smaller drift over time than local rate sensing devices114, and thus can provide better, more accurate, and/or more stable measurement results than local rate sensing devices114. Therefore, with the “master-slave” configuration, positioning system106can achieve a relatively good perception result using only one high quality primary rate sensing device112and several relatively low quality local rate sensing devices114, or using primary rate sensing device112and local rate sensing devices114that are both of relatively low quality. In contrast, for a positioning system without the “master slave” configuration, i.e., a positioning system having only local rate sensing devices and local positioning controllers coupled to perception sensors, to achieve a similar result, much higher quality and more stable local rate sensing devices are needed, in other words, positioning system106of the present disclosure can provide a good perception result with a relatively low cost.

Referring again toFIG. 1A, in some embodiments, positioning system106also includes one or more processors118and one or more storage medium120. Processors118may be configured to perform various operations consistent with embodiments of the present disclosure. The term “processor” may include any physical device having an electric circuit that performs a logic operation on input. For example, processor118may include one or more integrated circuits, microchips, microcontrollers, microprocessors, all or part of a central processing unit (CPU), graphics processing unit (GPU), digital signal processor (DSP), field programmable gate array (FPGA), or other circuits suitable for executing instructions or performing logic operations.

Storage medium120may be a non-transitory computer-readable storage medium, such as a memory device (e.g., random access, flash memory, or the like), an optical medium (e.g., a CD, DVD, BluRay®, or the like), firmware (e.g., an EPROM), or any other storage medium. Storage medium120may store data captured by, for example, perception sensors102. Storage medium120may also store a computer program containing instructions for execution by processor118to cause positioning system106to perform particular operations, such as operations consistent with embodiments of the present disclosure.

In some embodiments, instead of or in addition to storage medium storing the above-described computer program, positioning system106may also include hardware modules comprised of connected logic units, such as gates and flip-flops, and/or comprised of programmable units, such as programmable gate arrays or processors, for example, each of which is configured to perform part or all of the operations consistent with embodiments of the present disclosure.

FIG. 3is a flow chart illustrating an exemplary method300for generating pose information. Method300may be executed by positioning system106consistent with embodiments of the present disclosure.

As shown inFIG. 3, at302, primary rate sensing device112generates and sends primary pose incremental information to primary positioning controller108. The primary pose incremental information may include angular rates and/or acceleration of machine104.

At304, primary positioning controller108generates a propagated primary pose estimate based on the primary pose incremental information. In some embodiments, other machine wide measurement inputs reflecting additional machine-wide state information of machine104may also be taken into consideration while generating the propagated primary pose estimate.

At306, primary positioning controller108updates the propagated primary pose estimate using state update information provided by one or more state update sources116to generate an updated primary pose estimate and sends the updated primary pose estimate as primary pose information to local positioning controllers110. In some embodiments, when state update information from state update sources116is not available, primary positioning controller108may update the propagated primary pose estimate using previously received state update information.

At308, each local rate sensing device114generates and sends local pose incremental information to corresponding local positioning controller110. The local pose incremental information may include angular rates and/or acceleration of corresponding perception sensor102.

At310, each local positioning controller110generates a propagated local pose estimate based on the local pose incremental information.

At312, each local positioning controller110updates the corresponding propagated local pose estimate using the primary pose information to generate an updated local pose estimate and sends the updated local pose estimate to corresponding perception sensor102.

AlthoughFIG. 3shows the above processes in a particular order, one skilled in the art will appreciate that this does not constitute a requirement that the processes consistent with the present disclosure are performed in such an order. For example, generating the local pose incremental information by local rate sensing device114(308inFIG. 3) can be performed before or after updating the propagated primary estimate by primary positioning controller108(306inFIG. 3).

INDUSTRIAL APPLICABILITY

In the embodiments described above, the disclosed positioning system is described in connection with a perception system. However, the disclosed positioning system may also be applicable to any machine that includes a system that is sensitive to consistency among different units of the system and localized disturbances. The disclosed positioning system may provide a more robust solution and better long-term consistency. For example, with the disclosed positioning system, a perception system may enhance operator awareness by reducing mismatching of images generated by different perception sensors in the perception system. In particular, the disclosed positioning system may reduce or eliminate the mismatching by creating a root-mean-square best machine level pose and use this machine level pose to update local pose of each perception sensor. As such, random drifting of the perception sensors is reduced and a higher long term relative accuracy can be obtained.

It will be apparent to those skilled in the art that various modifications and variations can be made to the disclosed imaging system. Other embodiments will be apparent to those skilled in the art from consideration of the specification and practice of the disclosed imaging system. It is intended that the specification and examples be considered as exemplary only, with a true scope being indicated by the following claims and their equivalents.