Patent Publication Number: US-2023157507-A1

Title: Image-Viewing Method, Terminal, and Cleaner

Description:
CROSS-REFERENCE TO RELATED APPLICATION 
     This application is a 35 U.S.C. 371 national phase application of PCT International Application No. PCT/CN2021/070154 filed on Jan. 4, 2021, which claims priority over Chinese Patent Application No. 202010275131.6 filed on Apr. 9, 2020, both of which are incorporated herein by reference in their entireties. 
    
    
     TECHNICAL FIELD 
     The present disclosure relates to the field of image-processing technologies, and in particular, to an image-viewing method, a terminal, and a cleaning machine. 
     BACKGROUND 
     At present, after being equipped with a camera, a cleaning robot or a cleaning machine can effectively avoid an obstacle in a process of cleaning, and display an avoidance result such as information about an obstacle position and the like on a map in an application. However, when a user views the map using the application, the obstacle that the cleaning robot or the cleaning machine avoids cannot be seen, thereby causing a poor user experience. 
     It needs to be noted that information disclosed in forgoing the background section is merely for improving understanding of the background of the present disclosure, and therefore, information that does not constitute the prior art known to those ordinarily skilled in the art may be included. 
     SUMMARY 
     According to a first aspect of the present invention, there is provided an image-viewing method, applied to a terminal on which a client for a cleaning machine is installed. The method includes receiving, by using the client, an operation instruction for viewing an obstacle; sending, in response to the operation instruction for viewing the obstacle, a request to view the obstacle wherein the viewing request carries identification information used to uniquely identify an image of the obstacle or uniquely identify the obstacle; receiving the image of the obstacle identified by the identification information; and controlling, by the client, a display of the image of the obstacle. 
     According to an embodiment of the present invention, receiving, by the client, the operation instruction for viewing the obstacle includes receiving, by the client, a click operation instruction on an icon corresponding to the obstacle; or receiving, by the client, a touch operation instruction on an icon corresponding to the obstacle. 
     According to an embodiment of the present invention, controlling, by the client, the display of the image of the obstacle includes displaying, by the client, the image of the obstacle in a display area in which the icon corresponding to the obstacle is located. 
     According to a second aspect of the present invention, there is further provided an image-viewing method, applied to a cleaning machine-. The method includes receiving a request to view an obstacle wherein the request carries identification information used to uniquely identify an image of the obstacle or uniquely identify the obstacle; selecting the image of the obstacle identified by the identification information from an obstacle image database; and feeding back the image of the obstacle identified by the identification information. 
     According to an embodiment of the present invention, the method further includes, while feeding back the image of the obstacle identified by the identification information, feeding back at least one of the following pieces of information related to the obstacle: type of the obstacle, confidence in the obstacle, and a quantity of times the cleaning machine avoids the obstacle. 
     According to an embodiment of the present invention, the method further includes, after selecting the image of the obstacle identified by the identification information from the obstacle image database, cropping the image of the obstacle, and correspondingly, the feeding back of the image of the obstacle identified by the identification information including feeding back the cropped image. 
     According to an embodiment of the present invention, the method further includes receiving a request to view an original image of the cropped image; and feeding back, in response to the request to view the original image, the image of the obstacle identified by the identification information. 
     According to an embodiment of the present invention, the method further includes, before selecting the image of the obstacle identified by the identification information from the obstacle image database, storing one or more images into the obstacle image database in advance wherein each image represents a respective obstacle; and allocating, to each of the one or more images, a piece of identification information used to uniquely identify the image. 
     According to an embodiment of the present invention, the method further includes, before selecting the image of the obstacle identified by the identification information from the obstacle image database, storing one or more images and one or more identifiers into the obstacle image database in advance wherein each image represents a respective obstacle, and each identifier is associated with a respective image and is used to uniquely identify the obstacle represented by the image. 
     According to a third aspect of the present invention, there is further provided a terminal on which a client for a cleaning machine is installed. The terminal includes a receiving module, a sending module, and a display control module. The receiving module is configured to receive, by the client, an operation instruction for viewing an obstacle; the sending module is configured to send, in response to the operation instruction for viewing the obstacle, a request to view the obstacle wherein the request carries identification information used to uniquely identify an image of the obstacle or uniquely identify the obstacle; the receiving module is further configured to receive the image of the obstacle identified by the identification information; and the display control module is configured to control, by the client, a display of the image of the obstacle. 
     According to an embodiment of the present invention, the receiving module is further configured to receive, by the client, a click operation instruction on an icon corresponding to the obstacle; or receive, by the client, a touch operation instruction on an icon corresponding to the obstacle. 
     According to an embodiment of the present invention, the display control module is further configured to display, by the client, the image of the obstacle in a display area in which the icon corresponding to the obstacle is located. 
     According to a fourth aspect of the present invention, there is still provided a cleaning machine. The cleaning machine includes a receiving module, configured to receive a request to view an obstacle, wherein the request carries identification information used to uniquely identify an image of the obstacle or uniquely identify the obstacle; a selection module, configured to select the image of the obstacle identified by the identification information from an obstacle image database; and a feedback module, configured to feed back the image of the obstacle identified by the identification information. 
     According to an embodiment of the present invention, the feedback module is further configured to, while feeding back the image of the obstacle identified by the identification information, feeding back at least one of the following pieces of information related to the obstacle: type of the obstacle, a confidence in the obstacle, and a quantity of times that the cleaning machine avoids the obstacle. 
     According to an embodiment of the present invention, the cleaning machine further includes a cropping module, configured to, after selecting the image of the obstacle identified by the identification information from the obstacle image database by the selection module, crop the image of the obstacle; and correspondingly, the feedback module is configured specifically to feed back the cropped image. 
     According to an embodiment of the present invention, the receiving module is further configured to receive a request to view an original image of the cropped image; and the feedback module is further configured to feed back, in response to the request to view the original image, the image of the obstacle identified by the identification information. 
     According to another aspect of the present disclosure, there is provided an image-viewing apparatus including a processor and memory wherein the memory stores computer program instructions, and the computer program instructions perform the image-viewing method described in any one of the foregoing embodiments when being executed by the processor. 
     According to still another aspect of the present disclosure, there is further provided a non-transitory computer-readable storage medium including computer program instructions wherein the computer program performs the image-viewing method described in any one of the foregoing embodiments when being executed by the processor. 
     It needs to be understood that teaching of the present disclosure does not need to implement all the foregoing beneficial effects, but specific technical effects may be achieved through specific technical solutions; moreover, other embodiments of the present disclosure can further implement beneficial effects that are not mentioned above. 
    
    
     
       BRIEF DESCRIPTION OF DRAWINGS 
       The foregoing and other objectives, features, and advantages of exemplary embodiments of the present disclosure become easier to understand through following detailed descriptions with reference to the accompanying drawings. In the accompanying drawings, several embodiments are illustrated as examples without limitations. 
       In the accompanying drawings, a same or corresponding reference numeral indicates a same or corresponding part. 
         FIG.  1    is an interaction schematic flowchart illustrating implementation of an image-viewing method according to an embodiment of the present disclosure; 
         FIG.  2    is a schematic flowchart illustrating implementation of an image-viewing method at a terminal side according to an embodiment of the present disclosure; 
         FIG.  3    is a schematic flowchart I illustrating implementation of an image-viewing method at a cleaning machine side according to an embodiment of the present disclosure; 
         FIG.  4    is a schematic flowchart II illustrating implementation of an image-viewing method at a cleaning machine side according to an embodiment of the present disclosure; 
         FIG.  5    is a compositional structural schematic view of a terminal according to an embodiment of the present disclosure; and 
         FIG.  6    is a compositional structural schematic view of a cleaning machine according to an embodiment of the present disclosure. 
     
    
    
     DESCRIPTION OF EMBODIMENTS 
     The principles and the spirits of the present disclosure are described in the following with reference to several exemplary embodiments. It should be understood that the embodiments are provided only for those ordinarily skilled in the art to better understand and then implement the present disclosure, and are not to limit in any manner the scope of the present disclosure. On the contrary, the embodiments are provided for the present disclosure to be relatively clear and thorough, and for the scope of the present disclosure to be fully conveyed to those ordinarily skilled in the art. 
     Technical solutions of the present disclosure are further described in detail in the following with reference to the accompanying drawings and specific embodiments. 
       FIG.  1    is an interaction schematic flowchart illustrating implementation of an image-viewing method according to an embodiment of the present disclosure. 
     Referring to  FIG.  1   , according to this embodiment of the present disclosure, a procedure of implementing the image-viewing method includes information interaction between a terminal installed with a client and a cleaning machine. Specifically, an operation instruction for viewing an obstacle is first received from a user by a client installed on the terminal; a request to view the obstacle is then sent to the cleaning machine by the terminal in response to the operation instruction for viewing the obstacle wherein the request carries identification information; after the cleaning machine receives the request, an image of the obstacle identified by the identification information from an obstacle image database is selected by the cleaning machine; the image of the obstacle identified by the identification information is fed back to the terminal; and at last, a display of the image of the obstacle is controlled by the terminal through the client. 
       FIG.  2    is a schematic flowchart illustrating implementation of an image-viewing method at a terminal side according to an embodiment of the present disclosure. 
     Referring to  FIG.  2   , according to this embodiment of the present disclosure, the image-viewing method is applicable to the terminal wherein the client for the cleaning machine is installed on the terminal. The method includes operation  201  of receiving, by the client, the operation instruction for viewing the obstacle; operation  202  of sending, in response to the operation instruction for viewing the obstacle, a request to view the obstacle wherein the request carries identification information of the obstacle; operation  203  of receiving the image of the obstacle identified by the identification information; and operation  204  of controlling, by the client, display of the image of the obstacle. 
     Specifically, the operation instruction for viewing the obstacle may be a click operation instruction or a touch operation instruction. Correspondingly, operation  201  includes receiving, by the client, a click operation instruction on an icon corresponding to the obstacle; or receiving, by the client, a touch operation instruction on an icon corresponding to the obstacle. 
     Specifically, when the user performs a click operation or a touch operation on an icon corresponding to a to-be-viewed obstacle on the client used to display a cleaning map of the cleaning machine, the terminal receives the operation instruction for viewing the obstacle by the client. 
     In operation  202 , the terminal sends, in response to the operation instruction for viewing the obstacle, the request to view the obstacle to instruct the cleaning machine to select the image of the obstacle identified by identification information from the obstacle image database wherein the request carries identification information. Specifically, the identification information is used to uniquely identify the identification information of the image of the obstacle or uniquely identify the obstacle. 
     In this way, when receiving the request to view the obstacle that carries the identification information, the cleaning machine may directly select the image of the obstacle identified by the identification information from the obstacle image database according to the identification information used to uniquely identify the image of the obstacle. Alternatively, when receiving the request to view the obstacle that carries the identification information, the cleaning machine may also directly select the image of the obstacle identified by the identification information from the obstacle image database according to the identification information used to uniquely identify the obstacle. Specifically, in the image of the obstacle, a position of the obstacle in the image is marked in a manner that is identifiable to the user, for example, a rectangular block is marked out around the obstacle. 
     In operations  203  and  204 , after receiving the image of the obstacle that is identified by the identification information and is fed back by the cleaning machine, the terminal may directly display, by the client, the image of the obstacle in a specific display area in which the icon corresponding to the obstacle is located. As an example, the display area in which the icon corresponding to the obstacle is located may be a display area including an icon corresponding to the obstacle, or a display area at an upper left corner, a lower left corner, an upper right corner, or a lower right corner of the icon corresponding to the obstacle. 
     According to the embodiment of the present disclosure, in the image-viewing method, the operation instruction for viewing the obstacle is first received from the user by the client installed on the terminal; the request to view the obstacle is then sent, in response to the operation instruction for viewing the obstacle, to the cleaning machine, to instruct the cleaning machine to select the image of the obstacle identified by the identification information from the obstacle image database; and the image of the obstacle that is identified by the identification information and is fed back from the cleaning machine is received, and the display of the image of the obstacle is controlled by the client. In this way, according to the present disclosure, through information interaction between the client installed on the terminal and the cleaning machine, based on an operation of a user for viewing an obstacle, the image of the obstacle can be displayed in real time by the client. Therefore, the user can visually see the obstacle that the cleaning machine avoids, thereby effectively improving the user experience. 
       FIG.  3    is a schematic flowchart I illustrating implementation of an image-viewing method at a cleaning machine side according to an embodiment of the present disclosure. 
     Referring to  FIG.  3   , according to an embodiment of the present disclosure, the image-viewing method is applicable to the cleaning machine, and includes operation  301  of receiving a request to view an obstacle from a terminal wherein the request carries identification information; operation  302  of selecting an image of the obstacle identified by the identification information from an obstacle image database; and operation  303  of feeding back the image of the obstacle identified by the identification information to the terminal. 
     In operation  301 , after receiving the request to view the obstacle, the cleaning machine may analyze the request to obtain the identification information. Specifically, the identification information is used to uniquely identify the image of the obstacle or to uniquely identify the obstacle. 
     In operation  302 , an image of the obstacle is stored in the obstacle image database in advance. 
     In an example, when storing an image of the obstacle, the cleaning machine may allocate, to one image correspondingly, a piece of identification information (for example, identification code), which is used to uniquely identify the image every time, save the image, and send the identification information and coordinates of the corresponding obstacle in the image to a client in real time. In this way, when performing image viewing, that is, when receiving a request to view the obstacle that carries the identification information, the cleaning machine may select the image of the obstacle identified by the identification information quickly from the obstacle image database according to the identification information used to uniquely identify the image of the obstacle. 
     In another example, when storing an image of the obstacle, the cleaning machine directly and correspondingly stores, in the obstacle image database, the image of the obstacle and the identification information that is used to uniquely identify the obstacle. In this way, when performing image viewing, the cleaning machine selects, according to the identification information that is used to uniquely identify the obstacle, the image of the obstacle identified by the identification information from the obstacle image database. 
     In operation  303 , when feeding back the image of the obstacle identified by the identification information, the cleaning machine may feed back at least one of the following related pieces of information of the obstacle at the same time: type of the obstacle, a confidence in the obstacle, and a quantity of times the cleaning machine avoids the obstacle. 
       FIG.  4    is a schematic flowchart II illustrating implementation of an image-viewing method on the side of a cleaning machine according to an embodiment of the present disclosure. 
     Referring to  FIG.  4   , according to this embodiment of the present disclosure, the image-viewing method is applicable to a cleaning machine, and includes operation  401  of receiving, from a terminal, a request to view an obstacle wherein the request carries identification information; operation  402  of selecting an image of the obstacle identified by the identification information from an obstacle image database; operation  403  of cropping the image of the obstacle; and operation  404  of feeding back a cropped image. 
     In operation  401 , after receiving the request to view the obstacle, the cleaning machine may analyze the request to obtain the identification information. Specifically, the identification information is used to uniquely identify the image of the obstacle or uniquely identify the obstacle. 
     In operation  402 , the cleaning machine may store the image of the obstacle in the obstacle image database in advance. 
     In an example, when storing an image of the obstacle, the cleaning machine may allocate, to one image correspondingly, a piece of identification information (for example, identification code) that is used to uniquely identify the image every time; save the image; and send the identification information and coordinates of the obstacle in the image to a client in real time. In this way, when performing image viewing—that is, when receiving a request to view the obstacle that carries the identification information—the cleaning machine may quickly select, according to the identification information that is used to uniquely identify the image of the obstacle, the image of the obstacle identified by the identification information from the obstacle image database. 
     In another example, when storing an image of the obstacle, the cleaning machine directly and correspondingly stores the image of the obstacle and the identification information that is used to uniquely identify the obstacle in the obstacle image database. In this way, when performing image viewing, the cleaning machine selects, according to the identification information that is used to uniquely identify the obstacle, an image of the obstacle identified by the identification information from the obstacle image database. 
     In operations  403  and  404 , the image of the obstacle fed back by the cleaning machine may be cropped, and a principle for cropping is that at least the obstacle is included. In this way, the image fed back by the cleaning machine to a terminal client is a cropped image. Compared with an entire image (that is, an original image) of the obstacle, the cropped image has a relatively small size so that a good real-time feedback can be achieved, thereby effectively improving user experience. 
     In an embodiment of the present disclosure, an interface for a user to select and view the original image of the obstacle stored by the cleaning machine is provided in advance on the client of the terminal. In this way, after operation  404 , the method further includes further receiving, by the cleaning machine, a request to view an original image of the cropped image; and feeding back, in response the request to view the original image, the image of the obstacle identified by the identification information. 
     In this way, according to the present disclosure, through information interaction between the client installed on the terminal and the cleaning machine, based on the operation of the user for viewing the obstacle, the image of the obstacle can be displayed in real time by the client. Therefore, the user can visually see what obstacle the cleaning machine avoids, thereby effectively improving user experience. 
     Similarly, as with the image-viewing method described above, an embodiment of the present disclosure also provides a terminal wherein a client for a cleaning machine is installed on the terminal As illustrated in  FIG.  5   , the terminal  50  includes a receiving module  501 , a sending module  502 , and a display control module  503 . Specifically, the receiving module  501  is configured to receive, by the client, an operation instruction for viewing an obstacle; the sending module  502  is configured to send, in response to the operation instruction for viewing the obstacle, a request to view the obstacle wherein the request carries identification information that is used to uniquely identify the image of the obstacle or identification information used to uniquely identify the obstacle; the receiving module  501  is further configured to receive the image of the obstacle identified by the identification information; and the display control module  503  is configured to control, by the client, a display of the image of the obstacle. 
     According to an embodiment of the present disclosure, the receiving module  501  is specifically configured to: receive, by the client, a click-operation instruction on an icon corresponding to the obstacle; or receive, by the client, a touch-operation instruction on an icon corresponding to the obstacle. 
     According to an embodiment of the present disclosure, the display control module  503  is specifically configured to display, by the client, the image of the obstacle in a specific display area in which the icon corresponding to the obstacle is located. 
     It should be noted here that foregoing descriptions in the embodiment of the terminal are similar to foregoing descriptions in the method embodiments as illustrated in  FIG.  1    and  FIG.  2    and have similar beneficial effects to those of the foregoing method embodiments as illustrated in  FIG.  1    and  FIG.  2   . Therefore, details are not provided again. For technical details not disclosed in the embodiment of the terminal of the present disclosure, the foregoing descriptions in the method embodiments as illustrated in  FIG.  1    and  FIG.  2    of the present disclosure are referred to for understanding and, for brevity, details are not provided again. 
     Further, based on the image-viewing method described above, an embodiment of the present disclosure further provides a cleaning machine. As illustrated in  FIG.  6   , the cleaning machine  60  includes a receiving module  601 , configured to receive a request to view an obstacle wherein the request carries identification information that is used to uniquely identify an image of the obstacle or to uniquely identify the obstacle; a selection module  602 , configured to select the image of the obstacle identified by the identification information from an obstacle image database; and a feedback module  603 , configured to feed back the image of the obstacle identified by the identification information. 
     According to an embodiment of the present disclosure, the feedback module  603  is further configured to, while feeding back the image of the obstacle identified by the identification information, feed back at least one of following pieces of information related to the obstacle at the same time: type of the obstacle, confidence in the obstacle, and a quantity of times the cleaning machine avoids the obstacle. 
     According to an embodiment of the present disclosure, as illustrated in  FIG.  6   , the cleaning machine  60  further includes a cropping module  604 , configured to, after selecting the image of the obstacle identified by the identification information by the selection module from the obstacle image database, crop the image of the obstacle; and, correspondingly, the feedback module  603 , specifically configured to feed back a cropped image of the obstacle. 
     According to an embodiment of the present disclosure, the receiving module  601  is further configured to receive a request to view an original image of the cropped image; and the feedback module  603  is further configured to feed back, in response to the request to view the original image, the image of the obstacle identified by the identification information. 
     It should be noted here that foregoing descriptions in the embodiment of the cleaning machine are similar to foregoing descriptions in the method embodiments as illustrated in  FIG.  3    and  FIG.  4    and have similar beneficial effects to those of the foregoing method embodiments as illustrated in  FIG.  3    and  FIG.  4   . Therefore, details are not provided again. For technical details not disclosed in the embodiment of the cleaning machine of the present disclosure, the foregoing descriptions in the method embodiments as illustrated in  FIG.  3    and  FIG.  4    of the present disclosure are referred to for understanding and, for brevity, details are not provided again. 
     It should be noted that, in this specification, the terms “include,” “comprise,” or any other variants thereof are intended to cover a non-exclusive inclusion so that a process, a method, an article, or an apparatus that includes a list of elements not only includes those elements but also includes other elements that are not expressly listed, or further includes elements inherent to such process, method, article, or apparatus. In a case in which there are no other limitations, an element limited by words “include/comprise a . . . ” does not exclude that there are additional identical elements in the process, method, article, or apparatus that includes the element. 
     In the embodiments provided in the present application, it should be understood that the disclosed device and method may be implemented in another manner. The device embodiment described above is merely exemplary; for example, division of the units is merely logic function division, and there may be additional division manners in actual implementation; for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be ignored or not performed. In addition, mutual coupling, direct coupling, or communication connections between various displayed or discussed components may be through some interfaces, and indirect coupling or communication connections between devices or units may be electrical, mechanical, or in other forms. 
     The foregoing units described as separate components may or may not be physically separate, the components displayed as units may or may not be physical units, and they may be located at one place or distributed on a plurality of network units; and some or all of the units may be selected according to actual requirements to achieve objectives of solutions of the embodiments. 
     In addition, respective functional units in respective embodiments of the present disclosure may be integrated into one processing unit, or a respective unit may act as one unit independently, respectively, or two or more than two units may be integrated into one unit; and the foregoing integrated unit may be implemented in a form of hardware or in a form of hardware in combination with a software functional unit. 
     It could be understood by those ordinarily skilled in the art that all or some steps implementing the foregoing method embodiment may be implemented by hardware related to program instructions. The foregoing program may be stored in a computer-readable storage medium, and, when the program is executed, steps including the foregoing method embodiments are performed; and the foregoing storage medium includes various media that can store a program code such as a removable storage device, a read-only memory (ROM), a magnetic disk, an optical disc, or the like. 
     Alternatively, when being implemented in a form of a software functional module and sold or used as an independent product, the foregoing integrated units in the present disclosure may be stored in a computer-readable storage medium. Based on such an understanding, the essence or the part contributing to the prior art of the technical solutions in the embodiments of the present disclosure may be embodied in a form of a software product, and the computer software product is stored in a storage medium and includes several instructions configured to cause a computer device (which may be a personal computer, a server, a network device, or the like) to perform all or a part of the method described in the embodiments of the present disclosure. The foregoing storage medium includes various media that can store a program code such as a removable storage device, a ROM, a magnetic disk, an optical disc, or the like. 
     The foregoing descriptions are merely specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Variations or replacements that may be easily contemplated by those ordinarily skilled in the art within the scope of technology disclosed in the present disclosure shall fall within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims. 
     Another objective of the embodiments of the present disclosure is to provide a remote support method, and a device for remotely supporting a computer apparatus, which are easy to apply without damaging safety.