Patent Description:
Some navigation approaches for robotic devices includes utilizing imaging systems to identify objects in an environment for mapping and localization purposes. Such systems may include one or more image sensors to perform object detection, wall tracking, and so on. For example, such systems may include multiple image sensors that each have a different field of view.

One such navigation and localization approach includes utilizing a simultaneous localization and mapping (SLAM) algorithm with image sensor data as an input. Often, multiple image sensors get utilized to ensure that front, back, and side views get captured for purposes of ensuring that environmental features/obstructions are factored into navigation decisions. Multiple image sensors can be particularly important when a robotic device can move in potentially any direction based on rotation about a center axis of the same. This ensures that the robotic device collects a sufficient amount of environmental data from each field of view to prevent collisions, falling down stairs, and so on. However, image sensors increase both cost and complexity in manufacturing robotic devices, as well as necessitate having sufficient hardware/software resources to capture and process multiple simultaneous image data streams.

<CIT> describes an autonomous vacuum cleaner including an autonomously moveable housing carrying a vacuum collection system for generating a working air flow for removing dirt from the surface to be cleaned and storing the dirt in a collection space. Distance sensors for position sensing are mounted to the housing, behind a sensor cover.

<CIT> describes a robot cleaner that cleans a room using a serpentine room clean and a serpentine localized clean. Sensors can include an object following sensor, a stairway detector and bumper sensors.

<CIT> describes an electric vacuum cleaner comprising a main body case, drive wheels, a cleaning unit, an extraction means, and a control means.

<CIT> describes a mobile robot that includes a robot body having a forward drive direction, a drive system supporting the robot body above a cleaning surface for maneuvering the robot across the cleaning surface, and a robot controller in communication with the drive system. The robot also includes a bumper movably supported by a forward portion of the robot body and an obstacle sensor system disposed on the bumper.

According to a first aspect of the invention, there is provided a robotic surface cleaning device according to claim <NUM>.

According to a second aspect of the invention, there is provided a computer-implemented method for navigation of a robotic surface cleaning device according to claim <NUM>.

Optional and/or preferable features may be provided according to the dependent claims.

These and other features and advantages will be better understood by reading the following detailed description, taken together with the drawings wherein:.

In general, the present disclosure is directed to a time of flight (ToF) sensor arrangement that may be utilized by a robot device, e.g., a robotic surface cleaning device (or vacuum) or other robotic device, to identify and detect objects in a surrounding environment for mapping and localization purposes. According to the invention, a robot is disclosed that includes a plurality of ToF sensors disposed about a housing of the robot. Two or more ToF sensors may be angled/aligned to establish at least partially overlapping field of views to form redundant detection regions around the robot. Objects that appear simultaneously, or nearly simultaneously, may then be detected by the robot and utilized to positively identify, e.g., with a high degree of confidence, the presence of an object. The identified objects may then be utilized as data points by the robot to build/update a map. The identified objects may also be utilized during pose routines that allow the robot to orient itself within the map with a high degree of confidence.

Although the following aspects and embodiments specifically reference robotic vacuums, this disclosure is not limited in this regard. In addition, the ToF sensor arrangement disclosed variously herein may be utilized in a robot without an image sensor system for object identification/tracking, or alternatively, may be used in combination with an image sensor system.

The ToF sensor arrangement disclosed herein allows for environmental information to be collected in a relatively simple manner that can be used alone or in combination with other sensory such as image sensors. When used as a replacement for image sensors, the ToF sensor arrangement advantageously significantly reduces cost, complexity and computational load on hardware resources of the robotic device.

As generally referred to herein, a ToF sensor refers to any sensor device capable of measuring the relative distance between the sensor and an object in an environment. Preferably, infrared-type ToF sensors may be utilized, wherein each infrared ToF sensor includes an IR transmitter and receiver. However, other sensor types may be utilized such as acoustic ToF sensors that emit and receive sound waves, e.g., ultrasound, for measurement purposes.

Referring now to <FIG>, a block diagram illustrates an example robot navigation system <NUM> in accordance with an embodiment of the present disclosure. The robot navigation system <NUM> includes a housing <NUM>, a navigation controller <NUM>, a plurality of time of flight (ToF) sensors shown collectively as <NUM> and individually as <NUM>-<NUM> to <NUM>-n, and a motor <NUM>. The navigation controller <NUM> may comprise any number of chips, circuitry, and sensory for support robot functions such as field programmable gate arrays (FPGA), processors, memory, gyroscopes, and inertial sensors/acceleration sensors, and so on. The navigation controller (or simply controller) is not necessarily limited to navigation functions and may generally control all aspects of the robot including cleaning processes. The navigation controller <NUM> may implement routines such as Simultaneous Localization and Mapping (SLAM) or any other suitable navigation routine. The motor <NUM> may be controllable via signaling from the navigation controller <NUM> and may comprise one or more motors for driving wheels <NUM> (<FIG>) to cause the robot to travel along a surface. The wheels <NUM> may be implemented as a tread driving gear/wheel, and the particular example wheels shown in <FIG> should not be construed as limiting.

The housing <NUM> may have any shape and is not necessarily limited to the shape shown in the figures (e.g., circular). For example, the housing <NUM> may have a square shape, a D-shape, a triangular shape, a circular shape, a hexagonal shape, a pentagonal shape, and/or any other suitable shape. In some instances, the positioning of the ToF sensors <NUM>, relative to the housing <NUM>, may be based, at least in part, on the shape of the housing <NUM>.

Each of the plurality of time of flight sensors <NUM>-<NUM> to <NUM>-n may comprise any sensor capable of measuring relative distance between the sensor and an object in a surrounding environment and converting the same into a representational electrical signal. For example, the time of flight sensors <NUM>-<NUM> to <NUM>-n may comprise infrared laser-type sensors that utilize infrared wavelengths to output a measurement distance signal, which may be referred to herein as simply a measurement signal. In other examples, the time of flight sensors <NUM>-<NUM> to <NUM>-n may comprise sensors capable of measuring distance acoustically via soundwaves, e.g., ultrasound. In any event, the time of flight sensors <NUM>-<NUM> to <NUM>-n may comprise short-range sensors capable of measurements from a few centimeters to a meter, or long-range sensors capable of measurements from <NUM> meter to hundreds of meters, or a combination of both short and long-range ToF sensors.

As discussed in further detail below, the navigation controller <NUM> may receive measurement signals from the ToF sensors <NUM>-<NUM> to <NUM>-n to identify objects the environment of the robot. In an embodiment, the location of the identified objects relative to a known position of the robot may be utilized to update/build a map from a point cloud, e.g., a plurality of points that may utilized to generate a map. The identified objects may also be utilized to calculate robot odometry and pose in order to localize the robot within the map. The ToF sensors <NUM>-<NUM> to <NUM>-n may be used exclusively to identify objects in an environment, e.g., without the aid of image sensors or other like device, or may be used in combination with image sensor(s).

<FIG> shows a side perspective view of an example embodiment 106A of the robot navigation system <NUM> implemented in a robot <NUM>. As shown, the robot <NUM> is a vacuum-type robot including a nozzle and dust cup (not shown) for dirt collection. The housing <NUM> of the robot <NUM> includes a plurality of ToF sensors, which are more clearly shown in FIGs. <NUM>-<NUM>, but for ease of explanation and clarity the embodiment of <FIG> shows a single forward-facing ToF <NUM>-<NUM>. As generally referred to herein, forward-facing refers to a direction which is substantially parallel (or forms an acute angle) with an imaginary line representing the direction of travel for the robot as the same moves forward during operation (see <FIG>). On the other hand, rearward-facing as generally used herein refers to a direction which is substantially parallel (or forms an acute angle) with an imaginary line representing the direction of travel for the robot as the same moves in reverse during operation.

The ToF <NUM>-<NUM> includes a field of view (FOV) <NUM> which can have a generally conical shape. Each FOV may also be referred to herein as detection regions. When observed from the top, e.g., as shown in <FIG>, the ToF <NUM>-<NUM> may include a relatively narrow angle α. In an embodiment, the angle α of each of the FOVs for the ToF sensors is about <NUM> degrees, although other configurations are within the scope of this disclosure. For example, each FOV can include relatively wide-angled FOVs, e.g., <NUM> degrees or more, or relatively narrow angled FOVs, e.g., under <NUM> degrees, or a combination of different angles such that one or more ToF sensors have a FOV with a first angle that is greater than one or more other TOFs sensors with a second angle.

Turning to <FIG>, another example embodiment 106B of the robot <NUM> is shown. As shown, the robot <NUM> includes at least two ToF sensors, namely first and second ToF sensors <NUM>-<NUM> and <NUM>-<NUM>. In this embodiment, the ToF sensors <NUM>-<NUM> and <NUM>-<NUM> may be disposed in vertical alignment with each other such that an imaginary line drawn <NUM> substantially transverse from the surface to be cleaned intersects both the first and second ToF sensors <NUM>-<NUM> and <NUM>-<NUM>. Each FOV <NUM>-<NUM>, <NUM>-<NUM> of the first and second ToF sensors <NUM>-<NUM>, <NUM>-<NUM> may then at least partially overlap, as shown.

This configuration shown in <FIG> therefore be referred to as a vertically stacked ToF arrangement, whereby the FOV of two or more vertically-aligned sensors may be utilized to allow the robot navigation system <NUM> to determine a height of objects that appear within each associated FOV. In one example, height may be determined for an object by the robot navigation system <NUM> detecting object 212in the overlapping region <NUM> of each of the first and second FOVs <NUM>-<NUM>, <NUM>-<NUM>. The height of the detected object 212may then be determined based on a relatively simple calculation that utilizes the height/position of each of the first and second ToF sensors <NUM>-<NUM>, <NUM>-<NUM> relative to a known reference point (e.g., the surface to be cleaned or the body <NUM>), the relative distance from the robot <NUM> to the object <NUM>, the geometry of the first and second FOVs <NUM>-<NUM>, <NUM>-<NUM>, and more particularly, the geometry and dimensions of the overlapping region.

Thus, in response to the object appearing in the overlapping region <NUM>, a height determination for the object <NUM> may then be calculated. The calculated height for the object <NUM> may then be used by the robot navigation system <NUM> during localization and navigation. For example, height determinations by the robot navigation system <NUM> using the vertically-stacked ToF arrangement may be advantageously utilized to distinguish between objects/obstructions in an environment that can be navigated around, e.g., furniture, toys, and so on, by the robot <NUM> versus objects/obstructions that cannot be navigated around such as walls and windows.

Each of the first and second ToF sensors <NUM>-<NUM>, <NUM>-<NUM> may optionally be part of first and second ToF arrays <NUM>-<NUM>, <NUM>-<NUM>. Each of the ToF sensors in the second ToF arrays <NUM>-<NUM>, <NUM>-<NUM> can include ToF sensors disposed in a uniform manner relative to each other about the body <NUM> of the robot, such as shown, or may be disposed at varying distances relative to each other. As further shown in <FIG>, each of the ToF sensors of the first array of ToF sensors <NUM>-<NUM> may be vertically aligned with corresponding ToF sensors of the second array of ToF sensors <NUM>-<NUM>. Each pair of sensors vertically-aligned in this manner can be configured to have overlapping FOVs, as discussed above. Accordingly, the first and second arrays of ToF sensors <NUM>-<NUM>, <NUM>-<NUM> may allow for height determinations in multiple directions about the robot <NUM>, e.g., up to <NUM> degrees around the body <NUM> of the robot <NUM>.

<FIG> shows another example embodiment 106C of the robot <NUM>. As shown, the robot <NUM> includes at least two ToF sensors, namely first and second ToF sensors <NUM>-<NUM> and <NUM>-<NUM>, in a staggered configuration. In this embodiment, the ToF sensors <NUM>-<NUM> and <NUM>-<NUM> may be disposed in a displaced/staggered arrangement such that that an imaginary line drawn <NUM> substantially transverse from the surface to be cleaned intersects with only one of the first and second ToF sensors <NUM>-<NUM> and <NUM>-<NUM>. Each FOV <NUM>-<NUM>, <NUM>-<NUM> of the first and second ToF sensors <NUM>-<NUM>, <NUM>-<NUM> can overlap, as shown, for height detection purposes of discussed above.

In addition, each of the first and second ToF sensors <NUM>-<NUM>, <NUM>-<NUM> may optionally be part of first and second ToF arrays <NUM>-<NUM>, <NUM>-<NUM>, respectively. In this embodiment, the first and second ToF arrays <NUM>-<NUM>, <NUM>-<NUM> may be disposed in a staggered manner. To this end, the robot navigation system <NUM> may utilize overlapping FoVs from the first and second ToF sensors <NUM>-<NUM>, <NUM>-<NUM>, or from first and third ToF sensors <NUM>-<NUM>, <NUM>-<NUM>. Accordingly, this staggered configuration provides the robot navigation system <NUM> with flexibility as to which combinations of sensors, and by extension, which overlapping FOVs to utilize when calculating heights of objects/instructions within the same. Note while the embodiments of <FIG> show two arrays of ToF sensors, this disclosure is not limited in this regard. The present disclosure is equally applicable to embodiments including N number of arrays such as, for instance, three, four, or five arrays of ToF sensors arranged in a vertically-aligned configuration, staggered configuration, or combination thereof.

As is known, light-based ToF sensors measure relative distances to objects by reflecting light off of an object and measuring the duration of time for the light to be reflected back to the sensor. These calculations operate, in part, based on the speed of light remaining constant. As a robot travels forward, the change of a reported distance combined with the time interval between the reported changes in distance may be used to calculate the real-time speed of the robot, for instance. Therefore, the speed of a robot may be given by the following equation: <MAT>.

<FIG> shows another example embodiment of the robot <NUM> consistent with the present disclosure. The robot <NUM> of <FIG> includes seven (<NUM>) sensors, namely ToF <NUM>-<NUM> to <NUM>-<NUM>, although other numbers of sensors may be utilized and this is not intended to be limiting. For instance, the robot <NUM> may include two or more arrays of vertically stacked ToFs, as discussed above with regard to <FIG>. The ToFs <NUM>-<NUM> to <NUM>-<NUM> are disposed around the housing of the robot <NUM> at various locations. The spacing between sensors may be uniform, or may vary. As shown, the first ToF sensor <NUM>-<NUM> is forward facing and has a FOV that is substantially parallel with the direction of forward travel (represented by imaginary line <NUM>) for the robot, which is shown more clearly in <FIG>. Sensors <NUM>-<NUM> to <NUM>-<NUM> are also forward-facing but are arranged such that they have associated FOVs that are angled relative to an imaginary line <NUM> that represents a forward direction of travel for the robot <NUM>.

According to the invention, the angles of the FOV may allow for overlap of regions between the FOV for two or more ToF sensors. These overlapped regions are also referred to as redundant detection regions. For example, the embodiment of <FIG> shows that the FOV <NUM>-<NUM> for the ToF sensor <NUM>-<NUM> overlaps the FOV <NUM>-<NUM> for the sensor <NUM>-<NUM>. This overlapping arrangement may be utilized to establish predefined regions that allow for detection/identification of objects disposed in those redundant regions. In the event an object is affirmatively/positively identified in such a region, the information may be utilized to allow for tracking by a single sensor, e.g., sensor <NUM>-<NUM>, even after the object is no longer in the FOV <NUM>-<NUM> of the ToF sensor <NUM>-<NUM>, which is described in greater detail below.

In one specific example embodiment, the ToF <NUM>-<NUM> may be initially relied upon to track odometry. If the ToF <NUM>-<NUM> is not registering any objects, e.g., measurements are at or below a threshold floor value for distance, or tracking an object that has yet to enter the FOV of the other ToF sensors <NUM>-<NUM> to <NUM>-<NUM>, the navigation controller <NUM> may utilize the other sensors that have overlapping FOVs to positively identify objects through multi-sensor detection, e.g., by sensors <NUM>-<NUM> and <NUM>-<NUM>. In response to the detection of an object, the navigation controller <NUM> may "hand off" tracking to the ToF sensor with the FOV that is more likely to continue to have the object in view.

For example, consider a scenario where the navigation controller <NUM> detects object <NUM> simultaneously, or nearly simultaneously, entering the FOV <NUM>-<NUM> of ToF sensor <NUM>-<NUM> and the FOV <NUM>-<NUM> of ToF sensor <NUM>-<NUM>. The robot <NUM> may then "hand off" tracking to the sensor <NUM>-<NUM> as the associated FOV <NUM>-<NUM> can continue to detect the object <NUM> over the entire distance D as the robot <NUM> moves along direction <NUM>. On the other hand, the sensor <NUM>-<NUM> only remains capable of detecting object <NUM> for distance d before the same is outside the detection range of FOV <NUM>-<NUM>. Accordingly, distance d represents the extent of the redundant region by which both ToF sensors <NUM>-<NUM> and <NUM>-<NUM> can track object <NUM> while distance D represents the extent of the entire region by which ToF sensor <NUM>-<NUM> can detect presence/distance of the object <NUM> (assuming forward movement along direction <NUM>). In a general sense, this "handing off" allows for objects to be tracked by a single sensor with a relatively high degree of confidence that the object is present, and the relative distance of the robot <NUM> to the objects. This can be particularly advantageous when attempting to maintain a particular distance from an object such as a wall, furniture or other obstruction, or to otherwise adjust operation of the robot <NUM> based on the presence and distance of the object <NUM>.

Alternatively, or in addition to the "hand off" scenario discussed above, the navigation controller <NUM> may continue to track objects via two or more ToF sensors. For example, in the embodiment of <FIG>, the navigation controller <NUM> may use both ToF sensors <NUM>-<NUM> and <NUM>-<NUM> as a pair and triangulate relative distance to the object <NUM>. These triangulated measurements may thus allow the navigation controller <NUM> to establish a higher confidence level during odometry calculations, map building/updating, and/or general navigation.

In any such cases, and in accordance with an embodiment, objects may be detected by the navigation controller <NUM> when two or more ToF sensors with overlapping FOVs output a measurement signal that indicates the presence of an object and its relative distance from the robot <NUM>. In some cases, multiple measurements from each of the ToF sensors may be utilized by the navigation controller <NUM> to minimize or otherwise reduce instances of false positives. Thus, upon successive data points from each FOV (e.g., derived from the respective measurement signals), the navigation controller <NUM>, and by extension the robot <NUM>, can detect an object in the environment, optionally continue to track that object via one or more ToF sensors, and use those data points to increase the confidence of the robot odometry processes.

<FIG> shows another example embodiment for the robot <NUM> wherein the ToF sensors form multiple redundant regions for detection. As shown, the ToF sensors can be arrayed to form a "detection halo" about the robot whereby multiple detection regions both proximate the robot <NUM>, e.g., regions <NUM>-<NUM> to <NUM>-<NUM>, and distant from the robot <NUM>. According to the invention, regions <NUM>-<NUM> and <NUM>-<NUM> are utilized for object detection/tracking. Notably, pairs of forward and rear-facing ToFs, e.g., <NUM>-<NUM> and <NUM>-<NUM>; <NUM>-<NUM> and <NUM>-<NUM>, may be utilized to create overlapping regions (e.g., regions <NUM>-<NUM> and <NUM>-<NUM>) proximate to either side of the robot <NUM>. These regions may therefore be used for object detection and mapping functions, as well as positive identification for objects to be used in odometry calculations. This configuration may also be accurately described as shared ToF detection (or ToF Redundancy), and may be utilized to positive identify objects with a high degree of confidence to store a representation of the same within a point cloud. The point cloud may then be utilized to map the environment about the robot. In some instances, the regions <NUM>-<NUM> and <NUM>-<NUM> may be used during obstacle and/or wall following.

<FIG> shows another example embodiment of the robot <NUM> consistent with the present disclosure. In this embodiment, the robot <NUM> may utilize the plurality of ToF sensors <NUM>-<NUM> to <NUM>-<NUM> to perform a pose routine. As referred to herein, the term pose refers to a robots ability to orient itself within a map. Therefore, if the robot rotates <NUM> degrees about a center axis of its housing, pose refers to the ability to detect, measure and verify that the robot <NUM> has fully and correctly rotated <NUM> degrees.

In an embodiment, ToF redundant detection regions (labeled Regions <NUM> to <NUM>) can be utilized during pose calculations. Consider a scenario where the robot <NUM> of <FIG> detects an object in region <NUM> using one or more of the aforementioned ToF detection schemes discussed above. Then, the robot initiates a <NUM> degree rotation to the left (e.g., counter clockwise). The robot <NUM> may then use Region <NUM> to positively detect and identify the same object in an estimated window of time based on the executed movement sequence, e.g., based on wheel rotation estimates, real-time wheel encoders, or other suitable approach. Thus, the robot <NUM> may determine when the movement sequence, e.g., rotation about a center axis of the housing <NUM> of the robot <NUM>, has completed, and whether the estimates comport with the real-world data received from the ToF sensors. This validation/calibration sequence may therefore be utilized by the robot to calibrate movement systems, detect when the robot is stuck or otherwise impeded, and/or to detect the robot has fully transitioned into a desired orientation within its environment.

In accordance with an aspect of the present disclosure, whereby said aspect is not according to the invention and is present for illustration purposes only, a robotic surface cleaning device is disclosed. The robotic surface cleaning device comprising a housing, a motor coupled to at least one wheel to drive the robot, at least a first Time of Flight (ToF) sensor and a second ToF sensor coupled to the housing, the first and second ToF sensors having a first and a second field of view (FOV), respectively, the first and second FOV at least partially overlapping each other to form a first redundant detection region, a navigation controller disposed in the housing, the navigation controller to receive a first and second measurement signal from the first and second ToF sensors, respectively, and detect an object based, at least in part, on the first and second measurement signals indicating a presence of an object within the first redundant detection region.

In accordance with another aspect of the present disclosure, whereby said aspect is not according to the invention and is present for illustration purposes only, a robotic surface cleaning device to navigate in a surrounding environment to perform cleaning operations is disclosed. The robotic surface cleaning device comprising a housing with a first plurality of Time of Flight (ToF) sensors to identify and/or track objects in an environment surrounding the housing, wherein at least a first and a second ToF sensor of the first plurality of ToF sensors have detection regions at least partially overlapping each other to form a first redundant detection region, and a controller disposed in the housing to determine a location of an object in the surrounding environment relative to the housing based at least in part on the first redundant detection region.

In accordance with another aspect of the present disclosure, whereby said aspect is not according to the invention and is present for illustration purposes only, a computer-implemented method for navigation of a robotic surface cleaning device is disclosed. The method comprising establishing at least a first redundant detection region based at least in part on first and second Time of Flight (ToF) sensors having associated detection regions that at least partially overlap each other, receiving, by a controller, first and second measurement signals from the first and second ToF sensors, respectively, and detecting, by the controller, a location of an object relative to the robotic surface cleaning device based on the first and second measurement signals indicating presence of the object within the first redundant detection region.

Claim 1:
A robotic surface cleaning device (<NUM>) comprising:
a housing (<NUM>);
a motor (<NUM>) coupled to at least one wheel (<NUM>) to drive the robotic surface cleaning device (<NUM>);
a first Time of Flight, ToF, sensor (<NUM>-<NUM>) having a first field of view, FOV (<NUM>-<NUM>), a second ToF sensor (<NUM>-<NUM>) having a second FOV (<NUM>-<NUM>), a third ToF sensor (<NUM>-<NUM>) having a third FOV, and a fourth ToF sensor (<NUM>-<NUM>) having a fourth FOV, each ToF sensor being coupled to the housing (<NUM>) , the first (<NUM>-<NUM>) and second (<NUM>-<NUM>) FOV at least partially overlapping each other to form a first redundant detection region (<NUM>-<NUM>), the third and fourth FOV at least partially overlapping each other to form a second redundant detection region (<NUM>-<NUM>);
a navigation controller (<NUM>) disposed in the housing (<NUM>), the navigation controller (<NUM>) to:
receive a first and second measurement signal from the first (<NUM>-<NUM>) and second (<NUM>-<NUM>) ToF sensors, respectively;
detect an object based, at least in part, on the first and second measurement signals indicating a presence of the object within the first redundant detection region (<NUM>-<NUM>);
cause the robotic surface cleaning device (<NUM>) to rotate about a center axis of the housing (<NUM>) for a predetermined rotation angle;
receive a third and fourth measurement signal from the third and fourth ToF sensors, respectively;
detect the object based, at least in part, on the third and fourth measurement signals indicating a presence of the object within the second redundant region (<NUM>-<NUM>);
track the detected object (<NUM>) moving between the first redundant detection region (<NUM>-<NUM>) and the second redundant detection region (<NUM>-<NUM>)based, at least in part, on the first, second, third, and fourth measurement signals;
determine an actual rotation angle based, at least in part, on the tracked movement of the detected object; and
compare the actual rotation angle to the predetermined rotation angle to determine whether the actual rotation angle comports with the predetermined rotation angle.