Patent Description:
As an important constituent part of a smart home, ground-sweeping robots have appeared in the lives of thousands of households. A ground-sweeping robot needs a 3D vision function, including instant positioning, distance measurement, map construction, obstacle avoidance, and recognition of an obstacle in the direction of travel, such as a table, a chair, a sofa, a cabinet, or a home appliance. In the existing technology, the ground-sweeping robot usually carries a plurality of sensors to implement the 3D vision function, and may also project structured light through a diffractive optical element (DOE), and then implement obstacle recognition and obstacle avoidance by using a camera and a recognition algorithm.

Structured light is a system structure composed of a projection apparatus and a camera. Specific optical information is projected by the projection apparatus onto the surface and background of an object, and then is collected by the camera. Information such as the position and depth of the object is calculated according to the change of an optical signal caused by the object, and then the entire three-dimensional space is restored.

An existing solution for obstacle avoidance with structured light in a ground-sweeping robot usually cannot recognize a suspension space below furniture such as a sofa or a cabinet, causing the ground-sweeping robot to be stuck at the suspension space; or cannot recognize a shorter obstacle on the forward path of the ground-sweeping robot; or possibly cannot accurately recognize an elongated obstacle in the vertical direction, such as a chair leg.

EP patent application <CIT> discloses a mobile robot including a main body and a pattern irradiation unit emitting a cross-shaped optical pattern including a horizontal line optical pattern and a vertical line optical pattern intersecting the horizontal line optical pattern. The pattern irradiation unit includes a light source and a lens converting light emitted from the light source into the cross-shaped optical pattern, the lens includes convex cells on an incidence surface upon which the emitted light is incident, the incidence surface is divided into a first area converting the light emitted from the light source into the horizontal line optical pattern and a second area converting the light emitted from the light source into the vertical line optical pattern, vertical convex cells extended in parallel in the vertical direction are formed in the first area, and horizontal convex cells extended in parallel in the horizontal direction are formed in the second area.

EP patent application <CIT> discloses a mobile includes a first pattern emission unit for emitting first pattern light toward a floor in a cleaning area in front of a body and a second pattern emission unit for emitting second pattern light upward in front of the body, acquires an image obtained by emitting light of each of the patterns emitted from the first pattern emission unit and the second pattern unit through an image acquirer, detects a pattern from the acquired image to determine an obstacle, detects a cliff depending on a shape or position of the pattern, and is driven along a path along which the mobile robot does not fall off the cliff to avoid the cliff, and thus, preferentially determines whether a cliff is present, differentiates between a threshold and a cliff such as a stair to be driven or avoid the obstacle, is effectively driven due to the rapid determination and operation of the mobile obstacle, and prevents the mobile robot from falling off the cliff such as a stair.

The contents in the Background are only technologies known by the disclosers, and do not necessarily represent the existing technology in the field.

In view of at least one of the defects in the existing technology, the disclosure provides an obstacle detection system for a mobile apparatus as defined in the appended claims.

Preferred embodiments of the disclosure provide an obstacle detection system based on structured light, a mobile apparatus carrying the obstacle detection system, and a ground-sweeping robot. The obstacle detection system detects a higher obstacle within a large field of view on a path of travel by projecting a first lateral detection line in a horizontal direction; detects a suspension space below furniture such as a sofa or a cabinet by projecting a first longitudinal detection line in a vertical direction, thereby preventing the mobile apparatus from being stuck; and detects a low obstacle by projecting a second longitudinal detection line in the vertical direction, eliminating a blind zone caused when there is only a detection line in the horizontal direction; meanwhile, an elongated obstacle in the vertical direction can also be effectively detected with a plurality of longitudinal detection lines.

The drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification, and are used to explain the present disclosure along with the embodiments of the present disclosure, but do not constitute any limitation to the disclosure. In the drawings:.

The disclosure below provides many different embodiments or examples so as to realize different structures described herein. In order to simplify the disclosure herein, the following will give the description of the parts and arrangements embodied in specific examples. Of course, they are only for the exemplary purpose, not intended to limit the present disclosure. Besides, the present disclosure may repeat a reference number and/or reference letter in different examples, and such repeat is for the purpose of simplification and clarity, which does not represent any relation among various embodiments and/or arrangements as discussed. In addition, the present disclosure provides examples of various specific processes and materials, but those skilled in the art can also be aware of application of other processes and/or use of other materials.

The preferred embodiments of the present disclosure will be described below with reference to the drawings. It should be appreciated that the preferred embodiments described herein are only used to illustrate and explain the present disclosure, but not to limit the present disclosure.

According to a preferred embodiment of the disclosure, <FIG> schematically illustrates a mobile apparatus <NUM>, on which an obstacle detection system <NUM> is mounted. As shown in <FIG>, the obstacle detection system <NUM> includes a structured light projection module <NUM>, a camera module <NUM>, and an image processing module <NUM>. In the travelling process of the mobile apparatus <NUM>, the structured light projection module <NUM> projects a structured light <NUM> on a forward path of the mobile apparatus <NUM>, the structured light <NUM> including at least one lateral detection line in a horizontal direction and at least one longitudinal detection line in a vertical direction. The structured light <NUM> is irradiated onto obstacles <NUM>, <NUM> and <NUM> to form lateral or longitudinal bright lines. The camera module <NUM> captures the lateral or longitudinal bright lines, that is, an image of the structured light. The image processing module <NUM> communicates with the camera module <NUM> to receive the image of the structured light, and can calculate, according to the image of the structured light, the distances and positions of the obstacles <NUM>, <NUM> and <NUM> on the forward path.

According to a preferred embodiment of the disclosure, <FIG> illustrates one form of a pattern of the structured light <NUM> projected by the structured light projection module <NUM>, that is, the pattern of the structured light <NUM> includes a first lateral detection line <NUM>, a first longitudinal detection line <NUM> and a second longitudinal detection line <NUM>. The structured light projection module <NUM> projects the first lateral detection line <NUM> according to a first preset angle range, and the zero degree in the horizontal direction is set directly in front of the structured light projection module <NUM>. Preferably, the structured light projection module <NUM> projects the first lateral detection line <NUM> according to an angle range of -<NUM>° to +<NUM>° in the horizontal direction. The first lateral detection line <NUM> can be used to detect the higher obstacle <NUM> located within a large field of view on the path of travel of the mobile apparatus <NUM> (see <FIG>).

As shown in <FIG>, the structured light projection module <NUM> projects the first longitudinal detection line <NUM> according to a second preset angle range, and the zero degree in the vertical direction is set directly in front of the structured light projection module <NUM>. Preferably, the structured light projection module <NUM> projects the first longitudinal detection line <NUM> according to an angle range of <NUM>° to <NUM>° in the vertical direction, and the first longitudinal detection line <NUM> is located above the first lateral detection line <NUM>. As shown in <FIG>, with regard to a suspension space <NUM> below furniture such as sofa or a cabinet, the suspension space <NUM> is narrow, and the mobile apparatus <NUM> tends to be stuck after entering the suspension space <NUM>; alternatively, in the case where a height of the suspension space <NUM> is lower than that of the mobile apparatus <NUM>, the mobile apparatus <NUM>, when travelling to the position of the suspension space <NUM>, may be obstructed by the obstacle <NUM> above the suspension space <NUM> (e.g., an upper part of furniture such as a sofa or a cabinet). The first lateral detection line <NUM> is cast onto the suspension space <NUM> and cannot form an image, that is, the detection result is that there is no obstacle ahead, while the first longitudinal detection line <NUM> is cast onto the obstacle <NUM> above the suspension space <NUM> to form a longitudinal bright line. Then the image processing module <NUM> can determine, according to the longitudinal bright line, that there is the narrow suspension space <NUM> ahead, as well as the distance and position of the suspension space <NUM> relative to the mobile apparatus <NUM>. Therefore, the first longitudinal detection line <NUM> is used for detecting the suspension space <NUM> below furniture such as a sofa or a cabinet, so as to avoid the mobile apparatus <NUM> from being stuck.

Returning to <FIG>, the structured light projection module <NUM> projects the second longitudinal detection line <NUM> according to a third preset angle range. Preferably, the structured light projection module <NUM> projects the second longitudinal detection line <NUM> according to an angle range of -<NUM>° to <NUM>° in the vertical direction, and the second longitudinal detection line <NUM> is located below the first lateral detection line <NUM> and on an extension line of the first longitudinal detection line <NUM>. The first lateral detection line <NUM> is used for detecting the higher obstacle <NUM> located within a large field of view on the path of travel of the mobile apparatus <NUM>, but there is a blind zone with regard to the shorter obstacle <NUM>. As shown in <FIG>, with regard to the obstacle <NUM> having a height lower than a vertical distance between the structured light projection module <NUM> and the ground, the first lateral detection line <NUM> (a point perpendicular to the principal plane in <FIG>) cannot be cast onto the obstacle <NUM> for imaging, that is, the detection result is that there is no obstacle ahead. The second longitudinal detection line <NUM> is cast onto the obstacle <NUM> to form a longitudinal bright line, and the image processing module <NUM> can determine, according to the longitudinal bright line, that there is the shorter obstacle <NUM> ahead, as well as the distance and position of the obstacle <NUM> relative to the mobile apparatus <NUM>. Therefore, the second longitudinal detection line <NUM> is used for detecting the obstacle <NUM> having a height lower than the vertical distance between the structured light projection module <NUM> and the ground, eliminating the blind zone caused when there is only a detection line in the horizontal direction.

Returning to <FIG>, the first lateral detection line <NUM> is used for detecting the higher obstacle <NUM> located within a large field of view on the path of travel of the mobile apparatus <NUM>; the second longitudinal detection line <NUM> is located below the first lateral detection line <NUM> and on an extension line of the first longitudinal detection line <NUM>, and is used for detecting the obstacle <NUM> having a height lower than the vertical distance between the structured light projection module <NUM> and the ground, eliminating the blind zone caused when there is only a detection line in the horizontal direction. Therefore, preferably, one end point of the second longitudinal detection line <NUM> intersects with the first lateral detection line <NUM>. The first longitudinal detection line <NUM> is located above the first lateral detection line <NUM>, and is used for detecting the suspension space <NUM> below furniture such as a sofa or a cabinet, avoiding the mobile apparatus from being stuck. Since the intensity of the light source of the obstacle detection system <NUM> is constant, and the limited intensity of the light source, in the case of a shorter total length of all the detection lines of the structured light, will be evenly distributed on the shorter detection line so as to enhance the brightness of the detection line and extend the detection distance, the first longitudinal detection line <NUM> is thus preferably spaced from the second longitudinal detection line <NUM>, and there is no need to extend all the way upward from the first lateral detection line as long as an object above the suspension space can be detected. Obviously, a form of the connection between the first longitudinal detection line <NUM> and the second longitudinal detection line <NUM> also falls into the scope of protection of the disclosure.

According to a preferred embodiment of the disclosure, with regard to some vertical elongated obstacles located in the direction of travel, such as chair legs, the first lateral detection line <NUM> is cast onto an elongated obstacle to form an image approximate to a bright point, which is not easily recognized, while the first longitudinal detection line <NUM> and the second longitudinal detection line <NUM> are cast onto the elongated obstacle to form an image as a longitudinal bright line. After the camera module <NUM> captures the longitudinal bright line, the image processing module <NUM> may calculate, according to the image, the distance and position of the elongated obstacle on the forward path.

According to a preferred embodiment of the disclosure, <FIG> illustrates another form of the pattern of the structured light <NUM> projected by the structured light projection module <NUM>, that is, the pattern of the structured light <NUM> includes the first lateral detection line <NUM>, the first longitudinal detection line <NUM> and a second lateral detection line <NUM>. The structured light projection module <NUM> projects the first lateral detection line <NUM> according to the first preset angle range. Preferably, the structured light projection module <NUM> projects the first lateral detection line <NUM> according to an angle range of -<NUM>° to +<NUM>° in the horizontal direction. The structured light projection module <NUM> projects the first longitudinal detection line <NUM> according to the second preset angle range. Preferably, the structured light projection module <NUM> projects the first longitudinal detection line <NUM> according to an angle range of <NUM>° to <NUM>° in the vertical direction, and the first longitudinal detection line <NUM> is located above the first lateral detection line <NUM>. The structured light projection module <NUM> projects the second lateral detection line <NUM> according to a fourth preset angle range. Preferably, the structured light projection module <NUM> projects the second lateral detection line <NUM> according to an angle range of -<NUM>° to +<NUM>° in the horizontal direction. The second lateral detection line <NUM> is located below and parallel to the first lateral detection line <NUM>. The second horizontal detection line <NUM> can also be used for detecting the obstacle <NUM> having a height lower than the vertical distance between the structured light projection module <NUM> and the ground, eliminating the blind zone caused when there is only one detection line within a large field of view in the horizontal direction.

According to a preferred embodiment of the disclosure, <FIG> illustrates another form of the structured light <NUM> projected by the structured light projection module <NUM>, that is, the structured light <NUM> includes the first lateral detection line <NUM>, the first longitudinal detection line <NUM>, the second longitudinal detection line <NUM>, a third longitudinal detection line <NUM> and a fourth longitudinal detection line <NUM>. The structured light projection module <NUM> projects the first lateral detection line <NUM> according to the first preset angle range, and projects the first longitudinal detection line <NUM> according to the second preset angle range, the first longitudinal detection line <NUM> being located above the first lateral detection line <NUM> and at a lateral center of the first lateral detection line <NUM>. The structured light projection module <NUM> projects the second longitudinal detection line <NUM> according to the third preset angle range, the second longitudinal detection line <NUM> being located below the first lateral detection line <NUM> and on an extension line of the first longitudinal detection line <NUM>. The structured light projection module <NUM> also projects the third longitudinal detection line <NUM> and the fourth longitudinal detection line <NUM>, the third longitudinal detection line <NUM> and the fourth longitudinal detection line <NUM> being parallel to the first longitudinal detection line <NUM> and located in symmetrical positions with the first longitudinal detection line <NUM> as an axis of symmetry. The third longitudinal detection line <NUM> and the fourth longitudinal detection line <NUM> are used for improving the detection probability of detecting an elongated obstacle (e.g., a chair leg or the like), and the suspension space below furniture such as a sofa or a cabinet.

According to one preferred embodiment of the disclosure, <FIG> illustrates another form of the pattern of the structured light <NUM> projected by the structured light projection module <NUM>, that is, the pattern of the structured light <NUM> includes the first horizontal detection line <NUM>, the first longitudinal detection line <NUM>, the second lateral detection line <NUM>, the third longitudinal detection line <NUM> and the fourth longitudinal detection line <NUM>. The detection function of each detection line is the same as that in the aforesaid preferred embodiments, and will not be repeated here.

<FIG> illustrates a pattern of the structured light <NUM> according to a preferred embodiment of the disclosure. As shown in <FIG>, the pattern of the structured light <NUM> includes the first lateral detection line <NUM>, the first longitudinal detection line <NUM>, the second longitudinal detection line <NUM>, and the second lateral detection line <NUM>. The structured light projection module <NUM> projects the first lateral detection line <NUM> according to the first preset angle range, projects the first longitudinal detection line <NUM> according to the second preset angle range, projects the second longitudinal detection line <NUM> according to the third preset angle range, and projects the second lateral detection line <NUM> according to the fourth preset angle range, wherein the first longitudinal detection line <NUM> and the second longitudinal detection line <NUM> are aligned to each other and connected together, the second lateral detection line <NUM> intersects with a lower end point of the second longitudinal detection line <NUM> and is parallel to the first lateral detection line <NUM>. Preferably, the structured light projection module <NUM> projects the first longitudinal detection line <NUM> according to an angle range of <NUM>° to <NUM>° in the vertical direction, and projects the second longitudinal detection line <NUM> according to an angle range of <NUM>° to -<NUM>° in the vertical direction.

In the aforesaid preferred embodiments, various forms of the structured light <NUM> projected by the structured light projection module <NUM>, i.e., any combination of multiple detection lines, all fall into the scope of protection of the disclosure.

According to a preferred embodiment of the disclosure, the structured light projection module <NUM> includes a laser light source configured to emit a laser beam, and a diffractive optical element (DOE) disposed downstream of an optical path of the laser light source to receive the laser beam and project the structured light <NUM> having a preset pattern.

As shown in <FIG>, the camera module <NUM> and the structured light projection module <NUM> are disposed to be located in the same plane, and there is a displacement of a certain distance therebetween in each of the horizontal and vertical directions. The displacement between the camera module <NUM> and the structured light projection module <NUM> in the horizontal direction is ΔX, and the displacement therebetween in the vertical direction is ΔY (as shown in <FIG>). The camera module <NUM> has a lens group, the displacement between an equivalent optical center O" of the lens group and a central light outlet O of the structured light projection module <NUM> in the horizontal direction being ΔX, and the displacement therebetween in the vertical direction being ΔY.

Taking the light outlet O of the structured light projection module <NUM> as the coordinate origin of a spatial coordinate system (X, Y, Z), where the Z axis is the travel direction of the mobile apparatus <NUM>, the X axis is the horizontal direction, and the Y axis is the vertical direction. The structured light <NUM> emitted by the structured light projection module <NUM> is cast onto an obstacle to form a horizontal or vertical bright line. The position of any point of the bright line in the spatial coordinate system (X, Y, Z) is P (x, y, z). An image of the structured light <NUM> captured by the camera module <NUM> is imaged on the focal plane. Supposing an intersection point O' between an optical axis of the lens group of the camera module <NUM> and the focal plane as the coordinate origin of a plane coordinate system (U, V), the imaging position of any point P (x, y, z) of the bright line in the plane coordinate system is P' (u, v).

Now P (x, y, z) is used to represent the coordinates of any point on a lateral or longitudinal bright line in the spatial coordinate system, the bright line being formed by casting the structured light <NUM> onto the obstacle, while P' (u, v) is used to represent the coordinates of the imaging position of P (x, y, z) on the focal plane of the camera module <NUM>. As shown in <FIG>, O" is an equivalent optical center of the camera module <NUM>, an object distance is the distance of an obstacle along the travel direction of the mobile apparatus <NUM>, i.e., z, and an image distance is approximately a focal length f of the lens group of the camera module <NUM>.

Considering that the displacement between the equivalent optical center O" of the lens group of the camera module <NUM> and the light outlet O of the structured light projection module <NUM> in the horizontal direction is ΔX, and the displacement therebetween in the vertical direction is ΔY, then in the obstacle detection system <NUM>, with regard to a structured light bright point on a longitudinal detection line (e.g., the longitudinal detection line <NUM>, <NUM>, <NUM> or <NUM>), the image processing module <NUM> determines a spatial position of an obstacle corresponding to the structured light bright point in the following manner: <MAT> <MAT> <MAT>.

y and z can be inversely solved by a simple and calculating-power-saving method: <MAT>.

With regard to a structured light bright point on a lateral detection line (e.g., the lateral detection line <NUM> or <NUM>), the image processing module <NUM> determines a spatial position of an obstacle corresponding to the structured light bright point in the following manner: <MAT> <MAT> <MAT>.

The disclosure also provides a mobile apparatus <NUM> including the obstacle detection system <NUM> as described above. Preferably, the mobile apparatus is a ground-sweeping robot.

As shown in <FIG>, according to a preferred embodiment of the disclosure, a ground-sweeping robot carrying the obstacle detection system <NUM> is provided, the structured light projection module <NUM> projecting the first lateral detection line <NUM> according to the angle range of -<NUM>° to +<NUM>° in the horizontal direction, projecting the first longitudinal detection line <NUM> according to the angle range of <NUM>° to <NUM>° in the vertical direction, and projecting the second longitudinal detection line <NUM> according to the angle range of -<NUM>° to <NUM>° in the vertical direction. The height of the structured light projection module <NUM> from the ground is <NUM>, the displacement between the camera module <NUM> and the structured light projection module <NUM> in the horizontal direction is <NUM>, and the displacement therebetween in the vertical direction is <NUM>. According to the calculation:.

Claim 1:
An obstacle detection system (<NUM>) for a mobile apparatus (<NUM>), comprising:
a structured light projection module (<NUM>) configured to project structured light (<NUM>) on a forward path of the mobile apparatus (<NUM>), the structured light (<NUM>) comprising at least one lateral detection line in a horizontal direction and at least one longitudinal detection line in a vertical direction, wherein the structured light projection module (<NUM>) projects a first lateral detection line (<NUM>) according to a first preset angle range, projects a first longitudinal detection line (<NUM>) according to a second preset angle range, the first longitudinal detection line (<NUM>) being located above the first lateral detection line (<NUM>);
a camera module (<NUM>) configured to capture an image of the structured light (<NUM>); and
an image processing module (<NUM>) configured to calculate a distance and a position of an obstacle (<NUM>, <NUM>, <NUM>) on the forward path according to the image of the structured light (<NUM>),
wherein the structured light projection module (<NUM>) comprises a laser light source configured to emit a laser beam, and a diffractive optical element disposed downstream of an optical path of the laser light source to receive the laser beam and project the structured light,
characterized in that the structured light projection module (<NUM>) further projects a second lateral detection line (<NUM>) according to a fourth preset angle range, the second lateral detection line (<NUM>) being located below and parallel to the first lateral detection line (<NUM>),
the structured light projection module (<NUM>) further projects a second longitudinal detection line (<NUM>) according to a third preset angle range, the second longitudinal detection line (<NUM>) being located below the first lateral detection line (<NUM>) and on an extension line of the first longitudinal detection line (<NUM>), and the first longitudinal detection line (<NUM>) is spaced from the second longitudinal detection line (<NUM>).