Patent ID: 12235653

DETAILED DESCRIPTION

Terms used in the disclosure will be briefly described, and the disclosure will be described in detail.

The terms used in describing the embodiments of the disclosure are general terms selected that are currently widely used considering their function herein. However, the terms may change depending on intention, legal or technical interpretation, emergence of new technologies, and the like of those of ordinary skill in the related art. Further, in certain cases, there may be terms arbitrarily selected, and in this case, the meaning of the term will be disclosed in greater detail in the corresponding description. Accordingly, the terms used herein should be defined based on the meaning of the term and the overall context of the disclosure, and not simply by its designation.

Various modifications may be made to the embodiments of the disclosure, and there may be various types of embodiments. Accordingly, specific embodiments will be illustrated in drawings, and the embodiments will be described in detail in the detailed description. However, it should be noted that the various embodiments do not limit the scope of the disclosure to a specific embodiment, but they should be understood to include all modifications, equivalents or alternatives of the embodiments included in the ideas and the technical scopes disclosed herein.

The terms such as “first,” and “second” may be used in describing the various elements, but the elements are not to be limited by the terms. The terms may be used only to distinguish one element from another.

A singular expression includes a plural expression, unless otherwise specified. In the disclosure, it is to be understood that the terms such as “consist” or “include” are used herein to designate a presence of a characteristic, number, step, operation, element, component, or a combination thereof, and not to preclude a presence or a possibility of adding one or more of other characteristics, numbers, steps, operations, elements, components or a combination thereof.

Embodiments of the disclosure will be described in detail below with reference to the accompanying drawings to aid in the understanding of those of ordinary skill in the art. However, the disclosure may be realized in various different forms and it should be noted that the disclosure is not limited to the various embodiments described herein. Further, in the drawings, parts not relevant to the description have been omitted to clearly describe the disclosure, and like reference numerals may be used to indicate like elements throughout the disclosure.

FIG.1is a block diagram illustrating a configuration of a robot according to an embodiment of the disclosure.

Referring toFIG.1, a robot100may include a light emitter110, a camera120, a driver130, a memory140, and a processor150.

The light emitter110may output light for obtaining distance information to an object. Herein, distance information to an object may mean a distance between the robot100and the object. The light emitter110may output light at a pre-set time interval. For example, the light emitter110may output light for a first time period, and not output light for a second time period. The light emitter110may output a laser having various forms. For example, the light emitter110may output a laser of a line beam form in a direction horizontal with a ground. The light emitter110may be implemented as various laser devices. For example, the light emitter110may be implemented as a laser diode (LD) or a Vertical Cavity Surface Emitting Laser (VCSEL).

The camera120may capture the surroundings of the robot100, and obtain reflective light that is reflected from an object. Specifically, the camera120may obtain reflective light that is reflected from the object after being output from the light emitter110. In addition, the camera120may obtain external light. Here, the external light may include sunlight. The camera120may include a sensor device of various types. For example, the camera120may include a charge-coupled device (CCD) sensor, a complementary metal-oxide-semiconductor (CMOS) sensor, a photodiode (PD) and an avalanche photodiode (APD).

The driver130may be a configuration for moving the robot100. Specifically, the driver130may include an actuator for the driving of the robot100.

The memory140may store an operating system (OS) for controlling the overall operation of the elements of the robot100and commands or data associated with the elements of the robot100. To this end, the memory140may be implemented as a non-volatile memory (e.g., hard disk, solid state drive (SSD), flash memory), a volatile memory, and the like.

The processor150may control the overall operation of the robot100. The processor150may include an image obtaining module151, a distance information obtaining module152, a corrected image obtaining module153, a driving path generating module154, and an external light area identifying module155.

The image obtaining module151may obtain an image of the surroundings of the robot100using the camera120. The image obtaining module151may generate an image based on light obtained by the camera120. In the image obtained through the image obtaining module151, an object present in the surroundings of the robot100and light reflected from the object after being output from the light emitter110may be included. For example, the image obtaining module151may obtain a first image while the light emitter110is outputting light, and obtain a second image while the light emitter110is not outputting light. At this time, light output from the light emitter110may be included in the first image, but light output from the light emitter110may not be included in the second image.

The distance information obtaining module152may analyze an image obtained through the image obtaining module151and obtain distance information to an object included in the image. For example, the distance information obtaining module152may obtain distance information to an object based on triangulation. Specifically, the distance information obtaining module152may identify a pixel (or, a pixel group) having a size within a threshold range from among pixels in an image. Then, the distance information obtaining module152may calculate a distance to an object corresponding to the identified pixel (or pixel group) based on a position of the identified pixel (or pixel group). The distance information obtaining module152may calculate the distance to an object based on the position of the pixel (or pixel group) corresponding to light output from the light emitter110on the image.

Based on the distance information obtained through the distance information obtaining module152, the robot may generate a driving path and drive according to the generated driving path. The light obtained by the camera120may include not only light output from the light emitter110, but also external light (e.g., sunlight). Accordingly, an external light area (or external light) may be present in the image obtained through the image obtaining module151. When distance information is obtained based on the image in which the external light area is included and the driving path is generated, accuracy of the distance information and driving efficiency of the robot100may be reduced because the external light area is processed like an object (or obstacle). That is, despite an object not actually being present, based on the external light being recognized as an object, a driving path that bypasses the external light area may be generated.

As described above, in order to prevent driving efficiency of the robot100from being reduced because of the presence of external light, the corrected image obtaining module153may obtain a new image with the external light area removed. As shown inFIG.2, the corrected image obtaining module153may generate a third image30based on a first image10obtained while the light emitter110is outputting light and a second image20obtained while the light emitter110is not outputting light. Specifically, the corrected image obtaining module153may generate the third image30by deducting a pixel value of the second image20from a pixel value of the first image10. As shown inFIG.2, the first image10may include an area A corresponding to light output from the light emitter110(or, laser area) and an external light area B, but the third image30may not include the external light area B.

The distance information obtaining module152may obtain third distance information31based on the third image30with the external light area removed. The distance information obtaining module152may obtain third distance information31based on a position of the laser area A on the third image30. The third distance information31may include distance data r that corresponds to each of a plurality of angles (Θ).

The driving path generating module154may generate a driving path based on the third distance information31. The processor150may control the driver130for the robot100to drive along the generated driving path. As described above, by obtaining distance information based on the third image, accuracy of the distance information may be enhanced and driving efficiency of the robot100may be enhanced.

As shown inFIG.3, the laser area A and the external light area B may overlap. Because the corrected image obtaining module153generates the third image30by deducting the pixel value of the second image20from the pixel value of the first image10, a portion from among not only the external light area B but also the laser area A may be removed. In this case, when a driving path is generated based on the third image30, the removed laser area A is analyzed when generating the driving path, and the removal of laser area A may cause the robot100to collide with an object present in the external light area B.

To prevent problems such as the above, when the laser area A and the external light area B are overlapped, the external light area identifying module155may identify the external light area B, and the driving path generating module154may generate a driving path to drive bypassing the identified external light area B.

The external light area identifying module155may identify the external light area B based on information about an object. Here, the information about an object may include an image which captured an object and distance information to an object. For example, the information about an object may include the first image captured while the light emitter110is outputting light and the second image captured while the light emitter110is not outputting light. Alternatively, the information about an object may include distance information to an object included in the first image and distance information to an object included in the second image.

A method of identifying the external light area according to various embodiments will be described below.

According to an embodiment, the external light area identifying module155may identify the external light area B by analyzing the first image10. Specifically, the external light area identifying module155may identify a pixel (or, a pixel group) having a size greater than or equal to a threshold value from among pixels of the first image10. The external light area identifying module155may identify an area that corresponds to the identified pixel (or, a pixel group). The external light area identifying module155may obtain thickness information of the identified area. The external light area identifying module155may obtain thickness information of the identified area based on position information of a pixel (or pixel group) positioned at a lower end and a pixel (or pixel group) positioned at an upper end from among the plurality of pixels (or pixel groups) having a size greater than or equal to a threshold value.

The external light area identifying module155may compare obtained information about a thickness with pre-stored information about a thickness of light and identify whether the area that corresponds to the identified pixel (or pixel group) corresponds to the external light area B. Here, the pre-stored information about the thickness of light may include a look-up table in which information about a position within the image and the thickness of light is matched and stored. For example, if a thickness of an area that corresponds to the identified pixel (or pixel group) is greater than a thickness stored in the look-up table, the external light area identifying module155may identify the area that corresponds to the identified pixel (or pixel group) as the external light area B. Alternatively, if a difference between the thickness of the area that corresponds to the identified pixel (or pixel group) and the thickness stored in the look-up table is greater than or equal to a threshold value, the external light area identifying module155may identify the area that corresponds to the identified pixel (or pixel group) as the external light area B.

The external light area identifying module155may identify the external light area B by analyzing a pattern of the first image10. Specifically, the external light area identifying module155may identify the pixel (or pixel group) having a size greater than or equal to a threshold value from among the pixels of the first image10. The external light area identifying module155may compare a pattern of an area that corresponds to the identified pixel (or pixel group) with a pre-stored pattern and identify the external light area B. For example, the external light area identifying module155may identify, based on obtaining a measure of similarity between a pre-stored pattern and the pattern of the area that corresponds to the identified pixel (or pixel group), and the obtained similarity being less than or equal to a threshold value, the identified area as the external light area B. Here, the pre-stored pattern may be defined based on a form of light (e.g., line beam form) that is output from the light emitter110.

According to another embodiment, the external light area identifying module155may identify the external light area B by analyzing the second image20. Specifically, the external light area identifying module155may identify a pixel (or pixel group) having a size greater than or equal to a threshold value from among the pixels of the second image20. The external light area identifying module155may identify the area that corresponds to the identified pixel (or pixel group) as the external light area B. That is, because the second image20is an image obtained while the light emitter110is not outputting light, the external light area identifying module155may determine, based on a pixel (or pixel group) having a size greater than or equal to a threshold value being present in the second image20, that the corresponding pixel (or pixel group) has a size greater than or equal to a threshold value according to the external light.

The external light area identifying module155may identify the external light area based on the distance information obtained through the distance information obtaining module152. The distance information may include information of a distance from the robot100, and an angle (or position) based on the robot100. As shown inFIG.4A, the distance information may include distance data r that corresponds respectively to the plurality of angles (0). As described above, the distance information obtaining module152may obtain, using triangulation, first distance information11based on the first image10, and second distance information21based on the second image20.

The external light area identifying module155may identify the external light area B based on the second distance information21. For example, the external light area identifying module155may obtain distance data (r3, r4, r5, r6, r7) of a threshold range from among the second distance information21. Then, the external light area identifying module155may obtain position information of pixels in the second image20corresponding to the obtained distance data (r3, r4, r5, r6, r7). The external light area identifying module155may obtain position information of the obtained pixel (or pixel group) as position information about the external light area B.

The distance information obtaining module152may obtain third distance information31by deducting the second distance information21from the first distance information11. Then, the driving path generating module154may generate a driving path based on the third distance information31and position information about the external light area B. In an example, the driving path generating module154may generate a driving path that bypasses the laser area A and the external light area B.

When the external light area B is identified based on the second distance information21, the external light area identifying module155may change a value of distance data that corresponds to the identified external light area B as a pre-set value and store the same. For example, as shown inFIG.4B, the external light area identifying module155may change the value of distance data (r3, r4, r5, r6, r7) from among the second distance information21to a pre-set value c and store the same. Accordingly, the third distance information31obtained through the distance information obtaining module152may include distance data reflected with the pre-set value c. In this case, the driving path generating module154may generate a driving path that bypasses the area corresponding to the distance data (r3, r4, r5, r6, r7) based on the value of distance data (r3, r4, r5, r6, r7). As described above, the external light area identifying module155may change the distance data corresponding to the external light area B to a pre-set value so that it can be reflected when generating the driving path and store the same.

InFIG.4AandFIG.4B, although an embodiment of the distance information obtaining module152obtaining the third distance information31by deducting the second distance information21from the first distance information11has been described, the distance information obtaining module152may obtain the third distance information31based on the third image30obtained through corrected image obtaining module153.

Referring toFIG.4C, the corrected image obtaining module153may generate the third image30by deducting the pixel value of the second image20from the pixel value of the first image10. The robot100may move in-between a time point at which the first image10is captured and a time point at which the second image is captured. Accordingly, a matching of the positions of the pixel (or pixel group) of the first image10and the pixel (or pixel group) of the second image20may be needed. The corrected image obtaining module153may correct the position of the pixel (or pixel group) of the first image10or the second image20prior to deducting the pixel value of the second image20from the pixel value of the first image10. Then, the distance information obtaining module152may generate the third image30by deducting the pixel value of the second image20from the pixel value of the first image10. The distance information obtaining module152may obtain third distance information31which includes distance information to the laser area A based on the position of the pixel (or pixel group) corresponding to the laser area A included in the third image30. That is, the distance information obtaining module152may obtain the third distance information31based on triangulation.

Each module of the processor150described above may be implemented with software, or implemented in a combined form with software and hardware.

InFIG.1, a plurality of modules151to155are shown as one configuration of the processor150. However, this is merely one embodiment, and the plurality of modules151to155may be stored in the memory140. At this time, the processor150may load the plurality of modules151to155stored in the memory140from the non-volatile memory to the volatile memory and execute respective functions of the plurality of modules151to155.

FIG.5is a map illustrating a driving path of a robot according to an embodiment of the disclosure.

As shown inFIG.5, the robot100may generate a driving path P that bypasses the laser area A and the external light area B. Because a robot of the related art generates a driving path based only on the third distance information31ofFIG.4AandFIG.4Bwithout taking into consideration the external light area B, the area at which a distance value from the third distance information31is 0 has been determined as a free space, and a driving path passing therethrough has been generated. Accordingly, the robot of the related art generated a driving path that passed through the external light area B, and if an object is present in the external light area B, there has been the problem of a collision accident occurring. On the other hand, because the robot100according to an embodiment of the disclosure generates a driving path taking into consideration not only the third distance information31but also the external light area B, a driving path that bypasses the external light area B may be generated, and driving stability may be enhanced according thereto.

In the above, the robot100has been described as obtaining the first image10while the light emitter110is outputting light, and obtaining the second image20while the light emitter110is not outputting light. However, this is merely one embodiment, and the robot100may be implemented such that the second image20can be obtained while the light emitter110is outputting light.

FIG.6is a block diagram of a robot illustrating a method of obtaining a second image according to an embodiment of the disclosure. Referring toFIG.6, the camera120may include a first sensor121and a second sensor122. At this time, the second sensor122may include a filter (e.g., band reject filter) which rejects light corresponding to a wavelength of light (e.g., 905 nm) that is output from the light emitter110. Accordingly, the second sensor122may obtain the second image20which does not include the laser area even when the light emitter110is outputting light. Further, the first sensor121may obtain the first image10which includes the laser area. The processor150may obtain the third image30with the external light area removed based on the first image10and the second image20.

FIG.7is a flowchart illustrating a control method of a robot according to an embodiment of the disclosure.

The robot100may obtain first information about an object using a camera while the light emitter is outputting light to obtain distance information to the object, and obtain second information about the object using the camera while the light emitter is not outputting light (S710). Here, the first information and the second information may be an image captured through the camera. Alternatively, the first information and the second information may be distance information obtained based on the image. In the disclosure, information may include an image and distance information.

The robot100may obtain third information about an object based on the first information and the second information (S720). Based on the first information and the second information being an image, the robot100may deduct a pixel value of the second information from a pixel value of the first information and obtain the third information (third image). If the first information and the second information are distance data to each area within an image, the robot100may obtain the third information by deducting the second information from the first information.

The robot100may obtain information about the external light area based on at least one from among the first information, the second information, and the third information (S730). In an example, the robot100may obtain information about the external light area by analyzing the second information. Specifically, based on the second information being an image, the robot100may obtain position information of a pixel (or pixel group) having a size greater than or equal to a threshold value from among the pixels of the second information. The robot100may obtain the obtained position information of the pixel (or pixel group) as position information about the external light area. Based on the second information being distance information, the robot100may identify distance information within a threshold range from among the second information. The robot100may obtain position information that corresponds to the identified distance information, and obtain the obtained position information of the pixel (or pixel group) as position information about the external light area.

The robot100may generate a driving path of the robot100based on information about the external light area (S740). The robot100may generate a driving path that bypasses the external light area. Accordingly, driving stability of the robot100may be enhanced.

FIG.8is a flowchart illustrating a control method of a robot according to an embodiment of the disclosure.

The robot100may obtain the first image using the camera while the light emitter is outputting light to obtain distance information to an object, and obtain the second image using the camera while the light emitter is not outputting light (S810). Accordingly, while information about light output from the light emitter may be included in the first image, information about light output from the light emitter may not be included in the second image.

The robot100may obtain the third image by deducting the pixel value of the second image from the pixel value of the first image (S820). Accordingly, information about the external light area may not be included in the third image. The robot100may obtain distance information to an object included in the third image by analyzing the third image (S830). At this time, the robot100may obtain distance information to an object based on a position of light output from the light emitter in the third image.

The robot100may obtain information about the external light area based on at least one from among the first image, the second image, and the third image (S840). In an example, the robot100may identify a pixel (or pixel group) having a size greater than or equal to a threshold value from among the pixels of the first image. At this time, the robot100may obtain thickness information of an area that corresponds to the identified pixel (or pixel group). Then, the robot100may compare the obtained thickness information with the pre-stored thickness information and identify whether the area corresponding to the identified pixel (or pixel group) corresponds to the external light area. In another example, the robot100may identify the external light area included in the first image by comparing and analyzing a pattern of the first image with a pre-stored pattern.

The robot100may generate a driving path based on distance information to an object and information about the external light area (S850). The robot100may generate a driving path so as to drive spaced apart by a pre-set distance or more from the object and the external light area.

FIG.9is a flowchart illustrating a control method of a robot according to an embodiment of the disclosure.

The robot100may obtain the first image using the camera while the light emitter is outputting light to obtain distance information to an object, and obtain the second image using the camera while the light emitter is not outputting light (S910). Because the operation corresponds to operation S810inFIG.8, the detailed description thereof will be omitted.

The robot100may obtain first distance information based on the first image, and obtain second distance information based on the second image (S920). The robot100may obtain the first distance information and the second distance information based on triangulation. Then, the robot100may obtain third distance information by deducting the second distance information from the first distance information (S930).

The robot100may obtain information about the external light area based on the second distance information (S940). The robot100may identify distance information within a threshold range from among the second distance information, and obtain position information of a pixel of the second image that corresponds to the identified distance information. The robot100may obtain the obtained position information of the pixel as position information about the external light area.

The robot100may generate a driving path based on the third distance information and information about the external light area (S950). The robot100may generate, based on the distance information to an object and the position information of the external light area included in the third distance information, a driving path so as to drive spaced apart by a pre-set distance or more from the object and the external light area. That is, the robot100may generate a driving path that bypasses the object and the external light area.

The various embodiments described above may be implemented in a computer or in a recording medium readable by a device similar to a computer using a software, a hardware, or a combination thereof. In some cases, the embodiments described in the disclosure may be implemented by the processor on its own. Based on a software implementation, embodiments of processes and functions described in the disclosure may be implemented with separate software modules. Each of the above-described software modules may perform at least one function and operation described in the disclosure.

Computer instructions for performing processing operations according to the various embodiments described above may be stored in a non-transitory computer-readable medium. The computer instructions stored in this non-transitory computer-readable medium may cause a specific device to perform a processing operation according to the above-described various embodiments when executed by a processor.

The non-transitory computer readable medium may refer to a medium that stores data semi-permanently rather than storing data for a very short time, such as a register, a cache, a memory, or the like, and is readable by a device. Specific examples of the non-transitory computer readable medium may include, for example, and without limitation, a compact disc (CD), a digital versatile disc (DVD), a hard disc, a Blu-ray disc, a USB, a memory card, a ROM, and the like.

While the disclosure has been illustrated and described with reference to various example embodiments thereof, it will be understood that the various example embodiments are intended to be illustrative, not limiting. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the true spirit and full scope of the disclosure, including the appended claims and their equivalents.