Patent Description:
Robots have been developed for industrial use and have been a part of factory automation.

In recent years, the application of robots has been further expanded, medical robots, aerospace robots, and the like have been developed, and home robots that can be used in general homes have also been manufactured. Among these robots, a robot capable of traveling by itself is called the mobile robot. A representative example of the mobile robot used in home is a robot cleaner.

Various technologies for detecting an environment and a user around the robot cleaner through various sensors provided in the robot cleaner are known. In addition, technologies which allow the robot cleaner to learn and map a traveling area by itself and to determine a current position on a map are known. A robot cleaner that travels and cleans the traveling area in a preset manner is known.

In addition, in the prior art (<CIT>), a method of processing a map (grid map) for a cleaning area into a form that is easy for a user to check (such as changing an outline), and cleaning the cleaning area according to a cleaning command input through the map is disclosed.

On the other hand, in the prior art (<CIT>), it is related to a method for classifying areas using a feature point and a mobile cleaning robot using the same, and a technique has been disclosed in which a user can conveniently order cleaning command by dividing areas using feature points from a grid map.

However, in the case of the prior art described above, in a typical indoor environment, a lot of furniture such as a bed, the grid map created by the robot is significantly different from the actual area drawing, and it is difficult for the user to intuitively grasp information about the area.

On the other hand, in the prior art (<CIT>), it is disclosed that a area segmentation point is detected, analyzed, and the map is generated, and the area segmentation point is detected based on the width between pixels.

In addition, there is also a conventional technique in which a region is divided by a structure dividing point or erosion-expansion.

However, when the cleaning is performed by dividing the cleaning area as described above, the segmentation is performed based on the same point as the door. In a home environment, an area such as a room has a simple structure, so a room may be set as one area, but in a complex environment such as an office, there are many narrow areas, so it is highly likely that the segmentation point will not be matched where it is suitable.

In addition, there are many obstacles in one area, and thus it is not suitable for a pattern driving having straightness.

<CIT>
relates to a method for orienting an autonomous movable cleaning device, wherein the method comprises recording distance data to obstacles and acquiring image data. Based on the distance data, a cartography K of the overall area G is generated and stored in the device. Next, a potential map P is generated based on identifying the obstacles (e.g. walls) included in the cartography K. Based on the potential map, the overall area G is divided into sub areas. Moreover, the acquired images can be analyzed to detect door openings, which may be taken into consideration for dividing the overall area G. <CIT> relates to a moving body control apparatus including a distance sensor which acquires distance point data with respect to an obstacle around a moving body and a route calculation means which calculates a movement route of the moving body on a grid map showing a moving environment of the moving body. <CIT> relates to a method for controlling an autonomous mobile robot for carrying out a task in a local region of an area of application of the robot. The robot may be configured to clean floors.

A first object is to provide an area division optimized for the mobile robot traveling in a straight line by dividing the area in a map showing a cleaning area.

The second object is to provide the area division capable of minimizing the complicated movement path of the mobile robot by separating the area in the commercial space such as a large-sized office space.

Meanwhile, the office space is divided into narrow spaces and used by partition. In the case of such a separated space, the traveling path of the mobile robot becomes complicated, and the possibility of collision with the obstacle increases. The third object of the present disclosure is to provide a method for minimizing changes by minimizing the cleaning area having a bending path (as ¬-shaped) when dividing the cleaning area into a plurality of areas.

In addition, the fourth object is to provide a control method capable of minimizing the movement of the mobile robot for processing the uncleaned area by using a topology node and a distance map when dividing the cleaning area.

In an aspect, there is provided the mobile robot including: a traveling unit configured to move a main body; a cleaning unit configured to perform a cleaning function; a sensing unit configured to sense a surrounding environment; an image acquiring unit configured to acquire an image outside the main body; and a controller configured to generate a distance map indicating distance information from an obstacle for a cleaning area based on information detected and the image through the sensing unit and the image acquiring unit, divide the cleaning area into a plurality of detailed areas according to the distance information of the distance map and control to perform cleaning independently for each of the detailed areas.

The distance map is composed of a plurality of pixels, and each pixel includes distance information from an obstacle.

The controller selects a plurality of distance levels for the distance information and forms a boundary loop connecting the pixels to each other according to the plurality of distance levels.

The plurality of distance levels are defined as distance information in which the number of pixels having the same distance information is equal to or greater than a threshold value.

For each of the distance levels, the controller searches for pixels having the distance level and connects the searched pixels to neighboring pixels to form the boundary loop.

The controller forms the boundary loops in the order in which the distance levels are large.

The controller extends the boundary loop toward the obstacle to form the detailed area having a rectangular shape.

The controller cuts the pixel out of the rectangle or expands the pixel recessed in the rectangle to form a maximum rectangle included in the cleaning area.

When the cleaning area is a hallway area, the controller forms the detailed area by cutting the hallway area to a predetermined distance.

The controller controls the travelling unit to travel in a zigzag mode with respect to the detailed area.

In another aspect, there is provided a method of controlling the mobile robot to perform cleaning while moving a main body, the method comprising: performing a preceding cleaning in a cleaning area, obtaining a detection signal through a sensing unit, and photographing surrounding environment through an image acquiring unit to obtain image data; generating a distance map indicating distance information from an obstacle to the cleaning area based on the detection signal and the image data; and dividing the cleaning area into a plurality of detailed areas according to the distance information of the distance map.

The distance map is composed of a plurality of pixels, and each pixel includes distance information from the obstacle in the step of the generating the distance map.

The step of the dividing the cleaning area comprises: selecting a plurality of distance levels for the distance information, and forming a boundary loop connecting the pixels to each other according to the plurality of distance levels.

The plurality of distance levels are defined as distance information in which the number of pixels having the same distance information is equal to or greater than a threshold value in the step of the dividing the cleaning area.

The step of the dividing the cleaning area comprises: for each of the distance levels, searching for pixels having the distance level and connecting the searched pixels to neighboring pixels to form the boundary loop.

The step of the dividing the cleaning area is forming the boundary loops in the order in which the distance levels are large.

The step of the dividing the cleaning area comprises: extending the boundary loop toward the obstacle to form the detailed area having a rectangular shape.

In the step of the dividing the cleaning area, the pixel out of the rectangle is cut or the pixel recessed in the rectangle is expanded to form a maximum rectangle included in the cleaning area.

When the cleaning area is a hallway area, the detailed area is formed by cutting the hallway area to a predetermined distance.

The method further comprises; controlling travelling unit to travel in a zigzag mode with respect to the detailed area.

According to at least one of the embodiments of the present disclosure, the area division is optimized for the mobile robot traveling in a straight line by dividing the area in a map showing a cleaning area.

In addition, it is possible to minimize the complicated movement path of the mobile robot by separating the area in the commercial space such as a large-sized office space, and minimize changes by minimizing the cleaning area having a bending path (as ¬-shaped).

In addition, it is possible to minimize the movement of the mobile robot for processing the uncleaned area by using a topology node and a distance map when dividing the cleaning area.

Meanwhile, various other effects will be disclosed directly or implicitly in a detailed description according to an embodiment of the present disclosure to be described later.

The mobile robot <NUM> according to an embodiment of the present disclosure means a robot capable of moving itself using a wheel or the like, and may be a home helper robot and a robot cleaner. Hereinafter, referring to the drawings, a robot cleaner having a cleaning function among mobile robots will be described as an example, but the present disclosure is not limited thereto.

The mobile robot means a robot capable of moving itself using wheels or the like. Therefore, the mobile robot may be a guide robot, a cleaning robot, an entertainment robot, a home helper robot, a security robot, and the like, which can move by itself, and the present disclosure is not limited to the type of the mobile robot.

However, the present disclosure is not limited to these embodiments and can be modified in various forms.

On the other hand, the suffixes "module" and "part" for the components used in the following description are given simply by considering the ease of writing the present specification, and do not impart a particularly important meaning or role in itself. Therefore, the "module" and the "unit" may be used interchangeably.

Further, in this specification, terms such as first and second may be used to describe various elements, but these elements are not limited by these terms.

<FIG> shows an embodiment of the present disclosure, the mobile robot that is a cleaning robot.

The mobile robot <NUM> may be provided with a cleaning mechanism 135d such as a brush to clean a specific space while moving itself.

The mobile robot <NUM> includes sensing units <NUM>: <NUM> and <NUM> capable of detecting information about the surroundings.

The mobile robot <NUM> effectively fuses vision-based location recognition using a camera and rider-based location recognition technology using a laser to perform robust location recognition and map generation against environmental changes such as illumination changes and product location changes.

The image acquiring unit <NUM> photographs a travelling area, and may include one or more camera sensors for acquiring an image outside the main body.

In addition, the image acquisition unit <NUM> may include a camera module. The camera module may include a digital camera. The digital camera includes at least one optical lens, an image sensor (for example, a CMOS image sensor) composed of a plurality of photodiodes (for example, pixels) imaged by light passing through the optical lens, and a digital signal processor (DSP) that composes an image based on a signal output from photodiodes. The digital signal processor is capable to generate not only a still image but also a moving image composed of frames composed of still images.

In the present embodiment, the image acquisition unit <NUM> includes a front camera sensor provided to acquire an image in front of the main body, but the location and the photographing range of the image acquisition unit <NUM> are not necessarily limited thereto.

For example, the mobile robot <NUM> may include only a camera sensor that acquires an image of the front in the travelling area and perform vision-based location recognition and travelling.

Alternatively, the image acquisition unit <NUM> of the mobile robot <NUM> according to an embodiment of the present disclosure may include a camera sensor (not shown) that is disposed obliquely with respect to one surface of the main body and configured to photograph the front side and the top side together. That is, it is possible to photograph both the front side and the top side with a single camera sensor. In this case, the controller <NUM> may separate the front image and the upper image from the image acquired by the camera based on the angle of view. The separated front image may be used for vision-based object recognition with the image obtained from the front camera sensor. In addition, the separated upper image may be used for vision-based location recognition and travelling with the image acquired from an upper camera sensor.

The mobile robot <NUM> according to the present disclosure may perform a vision slam that recognizes the current location by comparing surrounding images with image-based pre-stored information or comparing acquired images.

Meanwhile, the image acquisition unit <NUM> may also include a plurality of front camera sensors and / or upper camera sensors. Alternatively, the image acquisition unit <NUM> may be provided with a plurality of camera sensors (not shown) configured to photograph the front and the top together.

In the case of this embodiment, a camera is installed on a part of the mobile robot <NUM> (ex, front, rear, bottom), and the acquired image can be continuously acquired during cleaning. Multiple cameras may be installed for each part to improve photographing efficiency. The image acquired by the camera can be used to recognize the type of material, such as dust, hair, floor, etc. in the space, whether to clean, or when to clean.

The front camera sensor may photograph a situation of an obstacle or a cleaning area existing in the front of the traveling direction of the mobile robot <NUM>.

According to an embodiment of the present disclosure, the image acquisition unit <NUM> may acquire a plurality of images by continuously photographing the periphery of the main body, and the obtained plurality of images may be stored in a storage unit.

The mobile robot <NUM> may increase the accuracy of obstacle recognition by using a plurality of images or may increase the accuracy of obstacle recognition by selecting one or more images from a plurality of images and using effective data.

The sensing unit <NUM> may include a lidar sensor <NUM> that acquires terrain information outside the main body <NUM> using a laser.

The lidar sensor <NUM> outputs the laser to provide information such as a distance, a location direction, and a material of the object that reflects the laser and can acquire terrain information of the travelling area. The mobile robot <NUM> may obtain <NUM>-degree geometry information using the lidar sensor <NUM>.

The mobile robot <NUM> according to the embodiment of the present disclosure may grasp the distance, location, and direction of objects sensed by the lidar sensor <NUM> and generate a map while travelling accordingly.

The mobile robot <NUM> according to the embodiment of the present disclosure may acquire terrain information of the travelling area by analyzing the laser reception pattern such as a time difference or signal intensity of the laser reflected and received from the outside. In addition, the mobile robot <NUM> may generate the map using terrain information acquired through the lidar sensor <NUM>.

For example, the mobile robot <NUM> according to the present disclosure compares the surrounding terrain information acquired from the lidar sensor <NUM> at the current location with the lidar sensor-based pre-stored terrain information or compares the acquired terrain information to perform a lidar slam that recognizes the current location.

More preferably, the mobile robot <NUM> according to the present disclosure effectively fuses vision-based location recognition using the camera and the lidar-based location recognition technology using the laser, and it can perform location recognition and map generation that are robust to environmental changes, such as changes in illuminance or changes in the location of the object.

Meanwhile, the sensing unit <NUM> may include sensors <NUM> for sensing various data related to the operation and state of the mobile robot <NUM>.

For example, the sensing unit <NUM> may include an obstacle detection sensor <NUM> that detects an obstacle in front. In addition, the sensing unit <NUM> may further include a cliff detection sensor for detecting the presence of a cliff on the floor in the travelling area, and a lower camera sensor for acquiring an image of the floor.

Referring to <FIG>, the obstacle detection sensor <NUM> may include a plurality of sensors installed at regular intervals on the outer circumferential surface of the mobile robot <NUM>.

The obstacle detection sensor <NUM> may include an infrared sensor, an ultrasonic sensor, an RF sensor, a geomagnetic sensor, a Location Sensitive Device (PSD) sensor, and the like.

Meanwhile, the location and type of the sensor included in the obstacle detection sensor <NUM> may vary depending on the type of the mobile robot <NUM>, and the obstacle detection sensor <NUM> may include more various sensors.

The obstacle detection sensor <NUM> is a sensor that detects a distance from an indoor wall or the obstacle, and the present disclosure is not limited to that type but will be described below by using an ultrasonic sensor.

The obstacle detection sensor <NUM> detects the object, particularly an obstacle, present in the travelling (movement) direction of the mobile robot <NUM> and transmits obstacle information to the controller <NUM>. That is, the obstacle detection sensor <NUM> may detect a projecting object, an object in the house, furniture, a wall, a wall edge, and the like, present on a movement path of the mobile robot <NUM>, in the front or side, and transmit the information to the controller <NUM>.

The mobile robot <NUM> may be provided with a display (not shown) to display a predetermined image such as a user interface screen. In addition, the display may be configured as a touch screen and used as an input means.

In addition, the mobile robot <NUM> may receive user input through touch, voice input, or the like, and display information on the object and a place corresponding to the user input on the display screen.

The mobile robot <NUM> may perform an assigned task, that is, cleaning while travelling in a specific space. The mobile robot <NUM> may perform autonomous travelling that generates a path to a predetermined destination on its own and travels and following travelling that moves while following a person or another robot. In order to prevent the occurrence of a safety accident, the mobile robot <NUM> can travel while detecting and avoiding the obstacle during movement based on the image data acquired through the image acquisition unit <NUM> and the detection data obtained from the sensing unit <NUM>.

The mobile robot <NUM> of <FIG> is capable of providing cleaning services in various spaces, for example, spaces such as airports, hotels, marts, clothing stores, logistics, hospitals, and especially large areas such as commercial spaces.

The mobile robot <NUM> may be linked to a server (not shown) that can manage and control it.

The server can remotely monitor and control the states of the plurality of robots <NUM> and provide effective service.

The mobile robot <NUM> and the server may be provided with communication means (not shown) supporting one or more communication standards to communicate with each other. In addition, the mobile robot <NUM> and the server may communicate with a PC, a mobile terminal, and other external servers. For example, the mobile robot <NUM> and the server may communicate using a Message Queuing Telemetry Transport (MQTT) method or a HyperText Transfer Protocol (HTTP) method. In addition, the mobile robot <NUM> and the server may communicate with a PC, a mobile terminal, or another server outside using the HTTP or MQTT method.

In some cases, the mobile robot <NUM> and the server support two or more communication standards and may use an optimal communication standard according to the type of communication data and the type of devices participating in the communication.

The server is implemented as a cloud server, and a user can use data stored and functions and services provided by the server through the server connected to various devices such as a PC and a mobile terminal.

The user can check or control information about the mobile robot <NUM> in the robot system through the PC, the mobile terminal, or the like.

In this specification, 'user' is a person who uses a service through at least one robot, an individual customer who purchases or rents a robot and uses it at home, and a manager of a company that provides services to employees or customers using the robot, the employees and the customers using the services provided by the company. Accordingly, the 'user' may include an individual customer (Business to Consumer: B2C) and an enterprise customer (Business to Business: B2B).

The user can monitor the status and location of the mobile robot <NUM> through the PC, the mobile terminal, and the like, and manage content and a work schedule. Meanwhile, the server may store and manage information received from the mobile robot <NUM> and other devices.

The mobile robot <NUM> and the server may be provided with communication means (not shown) supporting one or more communication standards to communicate with each other. The mobile robot <NUM> may transmit data related to space, objects, and usage to the server.

Here, the data related to the space and object are data related to the recognition of the space and objects recognized by the robot <NUM>, or image data for the space and the object obtained by the image acquisition unit <NUM>.

According to the embodiment, the mobile robot <NUM> and the server include artificial neural networks (ANN) in the form of software or hardware learned to recognize at least one of the user, a voice, an attribute of space, and attributes of objects such as the obstacle.

According to the embodiment of the present disclosure, the robot <NUM> and the server may include deep neural networks (Deep) such as Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), and Deep Belief Network (DBN), which are learned by Deep Learning. For example, the deep neural network structure (DNN) such as a convolutional neural network (CNN) may be installed on the controller (see <NUM> of <FIG>) of the robot <NUM>.

The server may transmit the updated deep neural network (DNN) structure data to the robot <NUM> after learning the deep neural network (DNN) based on data received from the mobile robot <NUM>, or data input by the user, and the like. Accordingly, the deep neural network (DNN) structure of artificial intelligence provided by the mobile robot <NUM> may be updated.

In addition, usage-related data is data obtained according to the use of a predetermined product, for example, data acquired according to the use of the robot <NUM>, and may include usage history data, sensing data obtained from the sensing unit <NUM>, and the like.

The learned deep neural network structure (DNN) may receive input data for recognition, recognize attributes of people, objects, and spaces included in the input data, and output the result.

In addition, the learned deep neural network structure (DNN) may receive input data for recognition, analyze and learn usage-related data of the mobile robot <NUM>, and recognize usage patterns, usage environments, and the like.

Meanwhile, data related to space, objects, and usage may be transmitted to the server through a communication unit (see <NUM> of <FIG>).

Based on the received data, the server may train the deep neural network (DNN) and then transmit the updated deep neural network (DNN) structure data to the mobile robot <NUM> for updating.

Accordingly, the mobile robot <NUM> becomes smarter and provides a user experience (UX) that evolves as it is used.

The robot <NUM> and the server may also use external information. For example, the server may comprehensively use external information acquired from other linked service servers to provide an excellent user experience.

According to the present disclosure, the mobile robot <NUM> and / or the server can perform voice recognition, so that the user voice can be used as an input for controlling the robot <NUM>.

Further, according to the present disclosure, the mobile robot <NUM> can provide a more diverse and active control function to the user by actively providing information or outputting a voice recommending a function or service.

On the other hand, such the mobile robot <NUM> may be implemented in the embodiment shown in <FIG>.

<FIG> is a perspective view showing the mobile robot and a charging stand for charging the mobile robot according to another embodiment of the present disclosure, and <FIG> is a block diagram showing a control relationship between main components of the mobile robot according to the embodiment of the present disclosure. The block diagram of <FIG> is applicable to both the mobile robot <NUM> of <FIG> and the mobile robot 100b of <FIG> and will be described below with the configuration of the mobile robot of <FIG>.

Referring to <FIG>, the mobile robot 100b includes a travelling unit <NUM> that moves the main body <NUM>. The travelling unit <NUM> includes at least one travelling wheel <NUM> that moves the main body <NUM>. The travelling unit <NUM> includes a travelling motor (not shown) connected to the travelling wheel <NUM> to rotate the travelling wheel. For example, the travelling wheels <NUM> may be provided on the left and right sides of the main body <NUM>, respectively, hereinafter referred to as the left wheel L and the right wheel R, respectively.

The left wheel L and the right wheel R may be driven by one travelling motor, but a left wheel travelling motor driving the left wheel L and a right wheel travelling motor driving the right wheel R may be provided as needed. The travelling direction of the main body <NUM> can be switched to the left or right side by making a difference in the rotational speeds of the left wheel L and the right wheel R.

The mobile robots <NUM> and 100b include a service unit <NUM> for providing a predetermined service. <FIG> illustrate the present disclosure as an example in which the service unit <NUM> performs a cleaning operation, but the present disclosure is not limited thereto. For example, the service unit <NUM> may be provided to provide a user with household services such as cleaning (scrubbing, suction cleaning, mopping, etc.), washing dishes, cooking, laundry, and garbage disposal. As another example, the service unit <NUM> may perform a security function for detecting external intruders or dangerous situations.

The mobile robots <NUM> and 100b may move the travelling area and clean the floor by the service unit <NUM>. The service unit <NUM> includes an inhalation device for inhaling foreign substances, brushes <NUM> and <NUM> for performing the brushing, a dust container (not shown) for storing foreign substances collected by the inhalation device or brush, and / or a mop(not shown) for performing mopping.

In the bottom part of the main body <NUM> of the mobile robot 100b of <FIG>, an intake port for inhalation of air may be formed, and in the main body <NUM>, an inhalation device(Not shown) that provides inhalation force so that air can be inhaled through the intake port and a dust container (not shown) for collecting dust sucked with air through the intake port may be provided.

The main body <NUM> may include a case <NUM> forming a space in which various components constituting the mobile robot 100b are accommodated. An opening for inserting and removing the dust container may be formed in the case <NUM>, and a dust container cover <NUM> that opens and closes the opening may be rotatably provided with respect to the case <NUM>.

A roll-type main brush having brushes exposed through the intake port, and an auxiliary brush <NUM> locationed on the front side of the bottom surface of the main body <NUM> and having a plurality of blades extending radially may be provided. The rotation of these brushes <NUM> separates dust from the floor in the travelling area, and the dust separated from the floor is sucked through the intake port and collects in the dust container.

The battery supplies not only the driving motor, but also the power required for the overall operation of the mobile robot 100b. When the battery is discharged, the mobile robot 100b may perform travelling to return to the charging stand <NUM> for charging, and during such return travelling, the mobile robot 100b may detect the location of the charging stand <NUM> by itself.

The charging stand <NUM> may include a signal transmission unit (not shown) that transmits a predetermined return signal. The return signal may be an ultrasonic signal or an infrared signal, but is not limited thereto.

The mobile robot 100b of <FIG> may include a signal detection unit (not shown) that receives the return signal. The charging stand <NUM> may transmit the infrared signal through the signal transmission unit, and the signal detection unit may include an infrared sensor that detects the infrared signal. The mobile robot 100b moves to a location of the charging stand <NUM> according to the infrared signal transmitted from the charging stand <NUM> and docks the charging stand <NUM>. By the docking, charging is performed between the charging terminal <NUM> of the mobile robot 100b and the charging terminal <NUM> of the charging stand <NUM>.

The mobile robot 100b may include the sensing unit <NUM> that senses information inside / outside the mobile robot 100b.

For example, the sensing unit <NUM> may include one or more sensors <NUM> and <NUM> sensing various types of information about the travelling area, and an image acquiring unit <NUM> for obtaining image information about the travelling area. According to the embodiment, the image acquisition unit <NUM> may be separately provided outside the sensing unit <NUM>.

The mobile robot 100b may map the travelling area through the information sensed by the sensing unit <NUM>. For example, the mobile robot 100b may perform vision-based location recognition and map generation based on the ceiling image of the travelling area acquired by the image acquisition unit <NUM>. In addition, the mobile robot 100b may perform location recognition and map generation based on a light detection and ranging (LiDAR) sensor <NUM> using a laser.

More preferably, the mobile robot 100b according to the present disclosure effectively fuses vision-based location recognition using a camera and laser-based lidar-based location recognition technology, thereby the robot 100b can perform location recognition and map generation that are robust to environmental changes, such as changes in illuminance and location of objects.

Meanwhile, the image acquisition unit <NUM> photographs the travelling area, and may include one or more camera sensors for acquiring an image outside the main body <NUM>.

In addition, the image acquisition unit <NUM> may include a camera module. The camera module may include a digital camera. The digital camera includes at least one optical lens and an image sensor (for example, a CMOS image sensor) composed of a plurality of photodiodes (for example, pixels) imaged by light passing through the optical lens, and a digital signal processor (DSP) that composes an image based on a signal output from photodiodes. The digital signal processor can generate not only a still image but also a moving image composed of frames composed of still images.

In this embodiment, the image acquisition unit <NUM> is provided on the front camera sensor 120a provided to acquire an image in front of the main body <NUM> and an upper camera sensor 120b located in the upper surface portion of the main body <NUM> and provided to acquire the image of the ceiling in the travelling area but the location and photographing range of the image acquisition unit <NUM> are not necessarily limited thereto.

For example, the mobile robot 100b may be equipped with only the upper camera sensor 120b that acquires the image of the ceiling in the travelling area, and perform vision-based location recognition and travelling.

Alternatively, the image acquisition unit <NUM> of the mobile robot 100b according to the embodiment of the present disclosure may include a camera sensor (not shown) configured disposed inclined with respect to one surface of the main body <NUM> to photograph the front and the top together. That is, it is possible to photograph both the front side and the top side with a single camera sensor. In this case, the controller <NUM> may separate the front image and the upper image from the image acquired by the camera based on the angle of view. The separated front image may be used for vision-based object recognition, such as an image obtained from the front camera sensor 120a. In addition, the separated upper image may be used for vision-based location recognition and travelling, such as an image obtained from the upper camera sensor 120b.

The mobile robot 100b according to the present disclosure may perform a vision slam of recognizing the current location by comparing surrounding images with pre-stored information based on images or comparing acquired images.

On the other hand, the image acquisition unit <NUM> may be provided with a plurality of front camera sensor 120a and / or upper camera sensor 120b. Alternatively, the image acquisition unit <NUM> may be provided with a plurality of camera sensors (not shown) configured to photograph the front and the top together.

In the case of this embodiment, a camera is installed on a part of the mobile robot (ex, front, rear, and bottom), and the captured image can be continuously acquired during cleaning. Multiple cameras may be installed for each part for photographing efficiency. The image captured by the camera can be used to recognize the type of material such as dust, hair, floor, or the like present in the space, to check whether it is cleaned, or when to clean.

The front camera sensor 120a may photograph a situation of the obstacle existing in the front of the traveling direction of the mobile robot 100b or a cleaning area.

According to the embodiment of the present disclosure, the image acquisition unit <NUM> may acquire a plurality of images by continuously photographing the surroundings of the main body <NUM>, and the obtained plurality of images may be stored in the storage unit <NUM>.

The mobile robot 100b may increase the accuracy of obstacle recognition by using a plurality of images or may increase the accuracy of obstacle recognition by selecting one or more images from a plurality of images and using effective data.

The lidar sensor <NUM> outputs a laser to provide information such as a distance, a location direction, and a material of an object that reflects the laser and can acquire terrain information of the travelling area. The mobile robot 100b may obtain <NUM>-degree geometry information with the lidar sensor <NUM>.

The mobile robot 100b according to the embodiment of the present disclosure may generate the map by grasping the distance, location, and direction of objects sensed by the lidar sensor <NUM>.

The mobile robot 100b according to the embodiment of the present disclosure may acquire terrain information of the travelling area by analyzing a laser reception pattern such as a time difference or signal intensity of a laser reflected and received from the outside. In addition, the mobile robot 100b may generate the map using terrain information acquired through the lidar sensor <NUM>.

For example, the mobile robot 100b according to the present disclosure may perform a lidar slam determining the moving direction by analyzing surrounding terrain information acquired at the current location through the lidar sensor <NUM>.

More preferably, the mobile robot 100b according to the present disclosure may effectively recognize obstacles and generate the map by extracting an optimal moving direction with a small amount of change using a vision-based location recognition using the camera and a lidar-based location recognition technology using the laser and an ultrasonic sensor.

Meanwhile, the sensing unit <NUM> may include sensors <NUM>, <NUM>, and <NUM> that sense various data related to the operation and state of the mobile robot.

For example, the sensing unit <NUM> may include an obstacle detection sensor <NUM> that detects the obstacle in front. In addition, the sensing unit <NUM> may further include a cliff detection sensor <NUM> that detects the presence of a cliff on the floor in the travelling area, and a lower camera sensor <NUM> that acquires an image of the floor.

The obstacle detection sensor <NUM> may include a plurality of sensors installed at regular intervals on the outer circumferential surface of the mobile robot 100b.

Meanwhile, the location and type of the sensor included in the obstacle detection sensor <NUM> may vary depending on the type of the mobile robot, and the obstacle detection sensor <NUM> may include more various sensors.

The obstacle detection sensor <NUM> detects the object, particularly the obstacle, present in the travelling (movement) direction of the mobile robot and transmits obstacle information to the controller <NUM>. That is, the obstacle detection sensor <NUM> may detect a projecting object present on a movement path of a mobile robot, in the front or side, a furniture in the house, furniture, a wall, a wall edge, and the like and transmit the information to the controller <NUM>.

At this time, the controller <NUM> detects the location of the obstacle based on at least two or more signals received through the ultrasonic sensor, and controls the movement of the mobile robot 100b according to the detected location of the obstacle to provide an optimal movement path when generating the map.

Depending on the embodiment, the obstacle detection sensor <NUM> provided on the outer surface of the case <NUM> may include a transmitter and a receiver.

For example, the ultrasonic sensor may be provided such that at least one transmitter and at least two receivers are staggered. Accordingly, signals can be radiated at various angles, and signals reflected by obstacles can be received at various angles.

Depending on the embodiment, the signal received from the obstacle detection sensor <NUM> may be subjected to a signal processing such as amplification and filtering, and then a distance and direction to the obstacle may be calculated.

Meanwhile, the sensing unit <NUM> may further include a travelling detection sensor that detects a travelling operation of the mobile robot 100b according to travelling of the main body <NUM> and outputs operation information. As the travelling sensor, a gyro sensor, a wheel sensor, an acceleration sensor, or the like can be used.

The mobile robot 100b may further include a battery detection unit (not shown) that detects a state of charge of the battery and transmits the detection result to the controller <NUM>. The battery is connected to the battery detection unit so that the battery level and charge status are transmitted to the controller <NUM>. The remaining battery power may be displayed on the screen of the output unit (not shown).

In addition, the mobile robot 100b includes an operation unit <NUM> capable of inputting on / off or various commands. Various control commands necessary for the overall operation of the mobile robot 100b may be received through the operation unit <NUM>. In addition, the mobile robot 100b may include an output unit (not shown), and display reservation information, battery status, operation mode, operation status, and error status, etc..

Referring to <FIG>, the mobile robot 100b includes the controller <NUM> for processing and determining various information such as recognizing a current location, and the storage unit <NUM> for storing various data. In addition, the mobile robot 100b may further include a communication unit <NUM> that transmits and receives data to and from other devices.

Among the devices that communicate with the mobile robot 100b, the external terminal has an application for controlling the mobile robot 100b, and through execution of the application, the mobile robot 100b displays the map of the travelling area to be cleaned, and specifies an area to clean a specific area on the map. The user terminal may communicate with the mobile robot 100b to display the current location of the mobile robot with the map, and information on a plurality of areas may be displayed. In addition, the user terminal updates and displays the location of the mobile robot according to the movement of the mobile robot.

The controller <NUM> controls the sensing unit <NUM>, the operation unit <NUM>, and the travelling unit <NUM> constituting the mobile robot 100b to control the overall operation of the mobile robot 100b.

The storage unit <NUM> records various information necessary for the control of the mobile robot 100b and may include a volatile or nonvolatile recording medium. The recording medium stores data that can be read by a microprocessor and is not limited to the type or implementation method.

In addition, the map for the travelling area may be stored in the storage unit <NUM>. The map may be input by the user terminal, the server, or the like capable of exchanging information with the mobile robot 100b through wired or wireless communication or may be generated by the mobile robot 100b learning by itself.

The location of the rooms in the travelling area may be displayed on the map. In addition, the current location of the mobile robot 100b may be displayed on the map, and the current location of the mobile robot 100b on the map may be updated in the travelling process. The external terminal stores the same map as the map stored in the storage unit <NUM>.

The storage unit <NUM> may store cleaning history information. Such cleaning history information may be generated each time cleaning is performed.

The map for the travelling area stored in the storage unit <NUM> includes a navigation map used for travelling during cleaning, a slam (Simultaneous localization and mapping) map used for location recognition, an obstacle, and the like. If it hits, it may be a learning map stored the corresponding information when an obstacle is encountered and use it for cleaning for learning, a global location map used for global location recognition, and an obstacle recognition map in which information about the recognized obstacle is recorded, and the like.

Meanwhile, as described above, maps may be separately stored and managed in the storage unit <NUM> for each use but the map may not be clearly classified for each use. For example, a plurality of pieces of information may be stored in one map for use in at least two or more purposes.

The controller <NUM> may include a travelling control module <NUM>, a location recognition module <NUM>, a map generation module <NUM>, and an obstacle recognition module <NUM>.

The travelling control module <NUM> controls travelling of the mobile robot 100b, and controls travelling of the travelling unit <NUM> according to the travelling setting. In addition, the travelling control module <NUM> may grasp the travelling route of the mobile robot 100b based on the operation of the travelling unit <NUM>. For example, the travelling control module <NUM> can grasp the current or past moving speed, the distance traveled, etc. of the mobile robot 100b, and also grasp the history of changing the current or past direction based on the rotational speed of the travelling wheel. Based on the travelling information of the mobile robot 100b identified, the location of the mobile robot 100b on the map may be updated.

The map generation module <NUM> may generate the map of the travelling area. The map generation module <NUM> may process an image acquired through the image acquisition unit <NUM> to generate the map. For example, the map corresponding to the travelling area and the cleaning map corresponding to the cleaning area can be generated.

In addition, the map generation module <NUM> may recognize the global location by processing the image acquired through the image acquisition unit <NUM> at each location and linking it with the map.

In addition, the map generation module <NUM> may generate the map based on information obtained through the lidar sensor <NUM>, and recognize a location based on the information obtained through the lidar sensor <NUM> at each location.

More preferably, the map generation module <NUM> may generate the map and perform location recognition based on information obtained through the image acquisition unit <NUM> and the lidar sensor <NUM>.

The location recognition module <NUM> estimates and recognizes the current location. The location recognition module <NUM> uses the image information of the image acquisition unit <NUM> to grasp the location in connection with the map generation module <NUM> and the location recognition module <NUM> may estimate and recognize the current location even though the location of the mobile robot 100b suddenly changes.

The mobile robot 100b is capable of recognizing the location during continuous travelling through the location recognition module <NUM>, and it is possible to learn the map and estimate the current location though the travelling control module <NUM>, the map generation module <NUM>, and the obstacle recognition module <NUM> without the location recognition module <NUM>.

The mobile robot 100b acquires the acquired image through the image acquisition unit <NUM> at an unknown current location. Various features such as lights, edges, corners, blobs, and ridges located on the ceiling are identified through the image.

As such, the controller <NUM> may classify the travelling area and generate the map composed of a plurality of regions, or recognize the current location of the main body <NUM> based on the pre-stored map.

In addition, the controller <NUM> may fuse the information obtained through the image acquisition unit <NUM> and the lidar sensor <NUM> to generate the map and perform location recognition.

When the map is generated, the controller <NUM> may transmit the generated map to the external terminal, the server, or the like through the communication unit <NUM>. Also, as described above, the controller <NUM> may store the map in the storage unit <NUM> when the map is received from the external terminal, the server, or the like.

In addition, when the map is updated while travelling, the controller <NUM> transmits the updated information to the external terminal so that the map stored in the external terminal and the mobile robot 100b is the same. As the map stored in the external terminal and the mobile robot 100b remains the same, for the cleaning command from the mobile terminal, the mobile robot 100b can clean the designated area, and the current location of the mobile robot 100b can be displayed on the external terminal.

At this time, the map is divided into a plurality of areas, and may include information on obstacles in the area.

When the cleaning command is input, the controller <NUM> determines whether the location on the map and the current location of the mobile robot match. The cleaning command may be input from a remote control, an operation unit or the external terminal.

If the current location does not match the location on the map, or if the current location cannot be confirmed, the controller <NUM> recognizes the current location and restores the current location of the mobile robot 100b, and then the controller <NUM> may be control to move the travelling unit <NUM> to the designated area based on the current location.

If the current location does not match the location on the map, or if the current location cannot be confirmed, the location recognition module <NUM> analyzes the acquired image from the image acquisition unit <NUM> and / or the terrain information acquired from the lidar sensor <NUM> and estimates the current location based on the map. In addition, the obstacle recognition module <NUM> or the map generation module <NUM> can also recognize the current location in the same way.

After recognizing the location and restoring the current location of the mobile robot 100b, the travelling control module <NUM> calculates a travelling route from the current location to the designated area and controls the travelling unit <NUM> to move to the designated area.

When receiving the cleaning pattern information from the server, the travelling control module <NUM> may divide the entire travelling area into a plurality of areas and set one or more areas as designated areas according to the received cleaning pattern information.

The travelling control module <NUM> processes the map generated from the map generating module <NUM> and divides the map into a plurality of detailed area. The travelling control module <NUM> divides the expanded detailed area from the boundary loop connected to nodes having similar distance levels based on the distance map from the topology node to the obstacle. At this time, the detailed area to be divided may have a quadrangular shape, and the mobile robot <NUM> may be defined as a region that can travel at a time.

In addition, the travelling control module <NUM> may calculate the travelling route according to the received cleaning pattern information, travel along the travelling route, and perform cleaning.

When the cleaning for the set designated area is completed, the controller <NUM> may store a cleaning record in the storage unit <NUM>.

In addition, the controller <NUM> may transmit the operation state or the cleaning state of the mobile robot 100b to the external terminal or the server at a predetermined cycle through the communication unit <NUM>.

Accordingly, the external terminal displays the location of the mobile robot 100b along with the map on the screen of the running application based on the received data, and also outputs information about the cleaning state.

The mobile robot 100b according to the embodiment of the present disclosure moves in one direction until an obstacle or a wall surface is sensed, and when the obstacle recognition module <NUM> recognizes the obstacle, the robot 100b may determine travelling patterns such as straight and rotating.

For example, if the recognized obstacle attribute is a kind of obstacle that can be passed, the mobile robot 100b may continue to go straight. Or, if the attribute of the recognized obstacle is an obstacle that cannot be passed, the mobile robot 100b rotates to move a certain distance, and then moves to a distance in which the obstacle is detected in the opposite direction of the initial movement direction to travel in a zigzag form.

The mobile robot 100b according to an embodiment of the present disclosure may perform human or object recognition, and avoidance based on machine learning.

The controller <NUM> may include the obstacle recognition module <NUM> that recognize an obstacle previously learned by machine learning from an input image, and the travelling control module <NUM> that controls the travelling of the travelling unit <NUM> based on the attribute of the obstacle recognized.

The obstacle recognition module <NUM> may include an artificial neural network (ANN) in the form of software or hardware in which the attributes of the obstacle are learned.

For example, the obstacle recognition module <NUM> may include a deep neural network (DNN) such as a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), or a Deep Belief Network (DBN) trained by Deep Learning.

The obstacle recognition module <NUM> may determine the attribute of the obstacle included in input image data based on weights between nodes included in the deep neural network (DNN).

Meanwhile, the mobile robot 100b may further include an output unit <NUM> to display predetermined information as an image or output it as sound.

The output unit <NUM> may include a display (not shown) that displays information corresponding to the user's command input, a processing result corresponding to the user's command input, an operation mode, an operation state, and an error state.

According to an embodiment, the display may be configured as a touch screen by forming a mutual layer structure with a touch pad. In this case, the display composed of the touch screen may be used as an input device capable of inputting information by a user's touch in addition to the output device.

In addition, the output unit <NUM> may include an audio output unit (not shown) that outputs an audio signal. Under the control of the controller <NUM>, the sound output unit may output an alert message such as a warning sound, an operation mode, an operation state, an error state, information corresponding to a user's command input, and a processing result corresponding to a user's command input as sound. The audio output unit may convert the electrical signal from the controller <NUM> into an audio signal and output the converted audio signal. To this end, a speaker or the like may be provided.

Hereinafter, a control method for processing maps of the mobile robots <NUM> and 100b of <FIG> having the configuration diagram of <FIG> will be described.

<FIG> is a flowchart illustrating a control method of the mobile robot according to an embodiment of the present disclosure, and <FIG> are views for reference to the description of the control method of <FIG>.

Referring to <FIG>, the mobile robot <NUM> according to an embodiment of the present disclosure extracts the distance map stored by the command of the controller <NUM> (S10).

At this time, the distance map may be binarization data as shown in <FIG> and may indicate whether the obstacle exists or not.

In this case, the pixel at the point where the obstacle is present may be displayed as <NUM>, and the pixel at the point where the obstacle does not exist may be displayed as <NUM>.

In this distance map, a driving node that the mobile robot <NUM> traveled while forming the distance map may be displayed as a topology node.

The topology node indicates a point where the mobile robot <NUM> is located at regular time intervals, and the obstacle detected at a corresponding node of the mobile robot <NUM> is represented by a pixel representing <NUM>.

At this time, the pixel at the point, where the obstacle does not exist, of the distance map includes information on the distance from the obstacle.

Next, the controller <NUM> divides the corresponding cleaning area into detailed areas (S20).

Specifically, the distance information of each pixel is read from the distance map, and a distance level is extracted per step(S30).

For example, when the distance map is formed as shown in <FIG>, the closer the obstacle is, the smaller the distance information is, and at this time, the distance information of the pixel located in the center has the largest value.

Since such distance information may have different values depending on the boundary formed by the obstacle, they may have different distance information even if they are located in the same horizontal or column.

Therefore, the distance information having the largest value and the number of pixels having each distance information value are read together to set a reference distance level.

For example, when the distance information of the largest value is <NUM>, when the number of pixels having <NUM> as distance information is <NUM>, <NUM> is not recognized as the distance level.

Among the distance information having the value less than <NUM>, distance information in which the number of corresponding pixels is greater than or equal to a threshold value may be extracted.

For example, when pixels below the threshold meet <NUM> to <NUM> pixels and the number of pixels having distance information of <NUM> is greater than or equal to the threshold, <NUM> may be set as the first distance level D1.

Next, among the values smaller than the first distance level D1, distance information having a number of pixels equal to or greater than a threshold value may be set as the second distance level D2.

In this order, a plurality of distance levels may be set, for example, three distance levels may be set.

At this time, the controller <NUM> generates the boundary loop according to the first distance level D1 (S40).

Specifically, as illustrated in <FIG>, the pixel having the first distance level D1 at a starting position is set as a first node G1 and the controllers <NUM> explores pixels having the same first distance level D1 around the first node G <NUM>.

The pixel at the shortest distance having the first distance level D1 around the first node G1 is set as the second node G2, and the pixel at the shortest distance having the first distance level D1 around the second node G2 is set as the third node G3, and the above operation is repeated to continuously define neighboring pixels having the first distance level D1.

When a plurality of nodes are defined in this way, as shown in <FIG>, a first boundary loop C1 connecting between each node is formed.

That is, when forming the first boundary loop C1, the first node G1 is connected to the second node G2, and the second node G2 is connected to the third node G3 while the calculation process in which the first boundary loop C1 is extended is repeated.

At this time, when two consecutive nodes are connected, it is determined whether the corresponding space is a hallway (S50). That is, it is determined whether pixels having the same distance information are continuously searched in the same horizontal or column.

As such, when the corresponding area is not the hallway, the first boundary loop C1 is formed as shown in <FIG> through the connection of neighboring nodes to form a closed curve.

When the line connecting the nodes located at the first distance level D1 from the obstacle forms the closed loop as shown in <FIG>, the controller <NUM> performs area expansion from the first boundary loop C1 toward the obstacle (S60).

At this time, the area expansion from the first boundary loop C1 may proceed based on the first distance level D1, and the contour of the expanded area is eroded to have a rectangle.

That is, when some areas protrude without forming the rectangle, the area is eroded to form a side of the rectangle to define the largest rectangle that can be included in the cleaning area.

At this time, when the sides of all the squares are within the boundary pixels forming the obstacle to satisfy the maximum size square, if only a predetermined number of pixels are recessed from the maximum size square, the corresponding pixel may be expanded to satisfy the square.

At this time, the corresponding pixel may be marked so that it can be controlled to be careful when the mobile robot <NUM> is running.

As described above, the rectangular detailed area R1 of the region in which the first boundary loop C1 forms the closed loop among the regions of the distance map is divided with respect to the first distance level D1.

Next, for the region in which the first boundary loop C1 forming the closed loop for the first distance level D1 is not formed, for the second distance level D2 smaller than the first distance level D1, the boundary loop is formed (S70).

That is, for the second distance level D2 of <FIG>, the node having the same second distance information is searched, and each node is connected to form the second boundary loop C2 for the second distance level D2.

At this time, when the corresponding area satisfies the hallway area as shown in <FIG>, the number of pixels having the same distance information may be counted to divide the areas R2, R3, R4,<IMG> into a predetermined length.

At this time, in the case of the hallway area, if there is a portion where the second boundary loop C2 has an inflection point, the area may be divided based on the inflection point.

In this case, if it is determined that the hallway area it may correspond to an aisle in an office, and in the case of the aisle made of a chair or a desk rather than a wall aisle, the aisle may be formed with different distance information.

In the case of having different distance information as described above, it is possible to divide the cleaning area to minimize the uncleaned area by dividing at the inflection point of the distance information.

Therefore, even in the case of <FIG> in which the hallway areas are continuously connected, it can be divided into detailed areas R2, R3, R4<IMG> so as to have different rectangles based on some inflection points.

Next, the controller <NUM> of the mobile robot <NUM> may perform the same operation on the third distance level D3 to divide the cleaning area into detailed areas having a plurality of squares.

If it is determined that there is no unexpanded area, that is, when it is determined that all of the cleaning areas are divided into the rectangular detailed area, the controller <NUM> ends the area division of the distance map and stores the corrected distance map including the information in the storage unit <NUM> (S80).

The mobile robot <NUM> may control the travelling unit <NUM> to perform cleaning for each detailed area of the cleaning area based on the corrected distance map.

<FIG> is a view showing a cleaning area partitioned according to an embodiment of the present disclosure.

In the case of a large-area such as the office area as shown in <FIG>, hallway areas having different distances from each other are continuously formed by furniture such as a partition or a desk or chair.

In the case that the mobile robot <NUM> cleans the large area, such as the office area, when performing continuous travelling through the distance map through preceding cleaning, the mobile robot <NUM> must repeatedly travel an unnecessary path or a lot of uncleaned areas are generated.

Accordingly, by filtering is performed by setting each distance level according to the distance from the obstacle as in the present disclosure and dividing the boundary loop formed through the filtering into detailed areas of the rectangle having the maximum value, a calibrated distance map as shown in <FIG> can be generated.

Through this calibrated distance map, the mobile robot <NUM> may clean the inside of the rectangle of the corresponding detailed area without the uncleaned area through the optimized driving mode such as the zigzag mode or the edge mode. At this time, the mobile robot <NUM> may clean the entire cleaning area in the order of cleaning the neighboring divided areas after independently ending the cleaning of one divided area.

Therefore, the connection between divided areas is cut off, and the unclean area can be minimized by performing driving with distance information optimized for each area.

The mobile robot <NUM> according to the present disclosure is not limited to the configuration and method of the embodiments described as described above, the embodiments are all or part of each embodiment is optional so that various modifications can be made It may be configured in combination.

Likewise, although the operations are depicted in the drawings in a particular order, it should not be understood that such operations should be performed in the particular order shown or in sequential order, or that all shown actions should be performed in order to obtain the desired result. In certain cases, multitasking and parallel processing may be advantageous.

Claim 1:
A mobile robot (<NUM>, 100b) comprising:
a traveling unit (<NUM>) configured to move a main body (<NUM>);
a cleaning unit (<NUM>) configured to perform a cleaning function;
a sensing unit (<NUM>) configured to sense a surrounding environment;
an image acquiring unit (<NUM>) configured to acquire an image outside the main body (<NUM>); and
a controller (<NUM>) configured to:
perform a preceding cleaning in a cleaning area,
generate a distance map indicating distance information from an obstacle for a cleaning area based on information detected through the sensing unit (<NUM>) and the image acquired by the image acquiring unit (<NUM>), wherein the distance map is composed of a plurality of pixels, and each pixel includes distance information from an obstacle,
divide the cleaning area into a plurality of detailed areas according to the distance information of the distance map, and
control to perform cleaning independently for each of the detailed areas.