Patent Description:
Data centers provide computing facilities that service operational processing needs of a wide variety of local and global customers. As such, data centers implement a vast number of computer processing systems, i.e., processing servers and associated electronic equipment that, depending on the scale of customers, can range from hundreds to thousands of the same. Typically, for maintenance accessibility reasons, data centers organize the computer processing systems into a series of spaced rows of racks arranged in parallel that are separated by an aisle space disposed in between two consecutive rows of racks.

In addition, the racks may be housed in a space have a large surface area of more than thousands of square meters. Due to such a substantially large size of the data centers, the amount of time and expense to perform daily operations by human operators of the data centers may be considerable. Notably, taking inventory of the computer processing systems contained in a data center is a valuable task for proper management of the data center. In conventional data centers, inventory of physical locations and hardware characteristics of the computer processing systems is manually performed. Besides, said inventory may have to be updated on a regular basis due to potential replacement, modifications of locations and/ or addition of new computer processing systems.

Therefore, there exists some interest in an autonomous system that takes inventory of a plurality of objects within a structure such as computer processing systems in a datacenter.

<CIT> discloses robots that locates and maps item locations in a distribution site using a set of markers that the robots can scan without cessation of movement. The optimal position of the robots is determined by affine transform computation or feature mapping. The robot first aligns itself according to the expected item position as indicated by one or more of the markers. The alignment is determined based on size and orientation of the markers in images obtained using the robot's camera. The robot then aligns itself according to the actual item position. Here, the repositioning is determined based on size and orientation of the actual item in images obtained using the robot's camera. Using the robot cameras and feature mapping, robots traverse the shelves to identify and map item location therein.

<CIT> discloses an automated datacenter imaging system including an automated guided vehicle having a housing. The system includes an optical imaging system coupled to the housing comprising a plurality of cameras each configured to have a respective field of view, the fields of view being at least partially non-overlapping with one another. The system includes a laser imaging system coupled to the housing and scans the datacenter to obtain a plurality of distances between the housing and a plurality of locations within the datacenter. The system includes an image processor configured to correlate a plurality of images taken by the cameras with the plurality of distances taken by the laser imaging system into a single mosaic map. The image processor locates the plurality of images and the plurality of distances relative to a known coordinate system of the datacenter.

<CIT> discloses a method for a multiple camera sensor suite mounted on an autonomous robot to detect and recognize shelf labels using color Haar classifiers.

It is an object of the present technology to ameliorate at least some of the inconveniences present in the prior art. The object of the invention is solved by a method according to claim <NUM> and an autonomous mobile robot according to claim <NUM>. Preferred embodiments are presented in the dependent claims.

Unless otherwise indicated, it should be noted that the figures may not be drawn to scale.

The examples and conditional language recited herein are principally intended to aid the reader in understanding the principles of the present technology and not to limit its scope to such specifically recited examples and conditions. It will be appreciated that those skilled in the art may devise various arrangements that, although not explicitly described or shown herein, nonetheless embody the principles of the present technology.

Moreover, all statements herein reciting principles, aspects, and implementations of the present technology, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof, whether they are currently known or developed in the future. Similarly, it will be appreciated that any flowcharts, flow diagrams, state transition diagrams, pseudo-code, and the like represent various processes that may be substantially represented in non-transitory computer-readable media and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.

The functions of the various elements shown in the figures, including any functional block labeled as a "processor" or "processing unit", may be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. In some implementations of the present technology, the processor may be a general-purpose processor, such as a central processing unit (CPU) or a processor dedicated to a specific purpose, such as a digital signal processor (DSP). Moreover, explicit use of the term a "processor" should not be construed to refer exclusively to hardware capable of executing software, and may implicitly include, without limitation, application specific integrated circuit (ASIC), field programmable gate array (FPGA), read-only memory (ROM) for storing software, random access memory (RAM), and non-volatile storage.

Moreover, it should be understood that module may include for example, but without being limitative, computer program logic, computer program instructions, software, stack, firmware, hardware circuitry or a combination thereof which provides the required capabilities.

In the present description, various terms relating to spatial orientation such as "front", "rear", "top", "bottom", "left", "right", "upward", "downward", etc. are described relative forward, up-right motion of the robot of the present technology according to standard operation. However, it is understood that these terms are merely used to improve the clarity of the description and in no way are meant to be limiting in regard to orientation or form thereof.

In the context of the present specification, a "server" is a computer program that is running on appropriate hardware and is capable of receiving requests (e.g., from client devices) over a network, and carrying out those requests, or causing those requests to be carried out. The hardware may be one physical computer or one physical computer system, but neither is required to be the case with respect to the present technology. In the present context, the use of the expression a "server" is not intended to mean that every task (e.g., received instructions or requests) or any particular task will have been received, carried out, or caused to be carried out, by the same server (i.e., the same software and/or hardware); it is intended to mean that any number of software elements or hardware devices may be involved in receiving/sending, carrying out or causing to be carried out any task or request, or the consequences of any task or request; and all of this soft-ware and hardware may be one server or multiple servers, both of which are included within the expression "at least one server".

In the context of the present specification, "user device" is any computer hardware that is capable of running software appropriate to the relevant task at hand. Thus, some (non-limiting) examples of user devices include personal computers (desktops, laptops, netbooks, etc.), smartphones, and tablets, as well as network equipment such as routers, switches, and gateways. It should be noted that a device acting as a user device in the present context is not precluded from acting as a server to other user devices. The use of the expression "a user device" does not preclude multiple user devices being used in receiving/sending, carrying out or causing to be carried out any task or request, or the consequences of any task or request, or steps of any method described herein.

In the context of the present specification, the expression "dataset" includes information of any nature or kind whatsoever capable of being stored in a database. Thus information includes, but is not limited to audiovisual works (images, movies, sound records, presentations etc.), data (location data, numerical data, etc.), text (opinions, comments, questions, messages, etc.), documents, spreadsheets, lists of words, etc..

In the context of the present specification, the expression "component" is meant to include software (appropriate to a particular hardware context) that is both necessary and sufficient to achieve the specific function(s) being referenced.

In the context of the present specification, the expression "computer usable information storage medium" is intended to include media of any nature and kind whatsoever, including RAM, ROM, disks (CD-ROMs, DVDs, floppy disks, hard drivers, etc.), USB keys, solid state-drives, tape drives, etc..

In the context of the present specification, unless expressly provided otherwise, an "indication" of an information element may be the information element itself or a pointer, reference, link, or other indirect mechanism enabling the recipient of the indication to locate a network, memory, database, or other computer-readable medium location from which the information element may be retrieved. For example, an indication of a document could include the document itself (i.e. its contents), or it could be a unique document descriptor identifying a data object with respect to a particular object storage device, or some other means of directing the recipient of the indication to a network location, memory address, database table, or other location where the data object may be accessed. As one skilled in the art would recognize, the degree of precision required in such an indication depends on the extent of any prior understanding about the interpretation to be given to information being exchanged as between the sender and the recipient of the indication. For example, if it is understood prior to a communication between a sender and a recipient that an indication of an information element will take the form of a database key for an entry in a particular table of a predetermined database containing the information element, then the sending of the database key is all that is required to effectively convey the information element to the recipient, even though the information element itself was not transmitted as between the sender and the recipient of the indication.

An autonomous mobile robot <NUM> according to the present technology is presented for taking inventory of a plurality of objects within a structure. According to one non-limiting example of the present technology, the autonomous mobile robot <NUM> (details of which are described below) is configured and arranged for taking inventory of servers within a datacenter <NUM>, illustrated schematically in <FIG>. Broadly, the datacenter <NUM> includes a plurality of rows <NUM> of server racks <NUM>. While ten server rack rows <NUM> are illustrated with six server racks <NUM> each, this is simply meant to be a non-limiting example. Each pair of rows <NUM> is grouped to form a warm alley therebetween, with cold alleys formed between pairs of rows <NUM>. Each server rack <NUM> supports a plurality of servers (not shown). The cold alleys allow for a robot transport path <NUM> along which the autonomous mobile robot <NUM> can travel during operation of methods described hereinbelow. The datacenter <NUM> further includes a robot charging station <NUM> for charging the autonomous mobile robot <NUM>. Specifics of the charging station <NUM> could vary in different examples.

With reference to <FIG>, the autonomous mobile robot <NUM> according to and performing at least some non-limiting examples of the present technology is illustrated.

The autonomous mobile robot <NUM>, also referred to herein as the autonomous mobile robot <NUM>, includes a mobile robot base <NUM>, also referred to as the base <NUM>, for providing movement of the autonomous mobile robot <NUM>. The base <NUM> provides a support structure for components of the autonomous mobile robot <NUM>, as well as the movement of the autonomous mobile robot <NUM>. The base <NUM> includes a main body <NUM>. In the illustrated example, the main body <NUM> is cylinder, with a diameter greater than its height for stability. It is contemplated that the body <NUM> could be implemented in different forms.

The base <NUM> includes a charging module <NUM> (shown schematically) disposed in the body <NUM>. The charging module <NUM> includes battery and charging-related electronic components (not separately indicated) for providing power to the components of the autonomous mobile robot <NUM>. The charging module <NUM> is configured and arranged to connect to the robot charging station <NUM> to recharge the battery as required.

The robot base <NUM> also includes electric motors <NUM> (shown schematically) operatively connected to the charging module <NUM> and a plurality of ground-engaging elements <NUM>, specifically wheels <NUM>. The wheels <NUM> are operatively connected to the motors <NUM> to be driven thereby. The exact number of motors <NUM> and wheels <NUM> may vary between different examples and embodiments and it not meant to be limited to the illustrated arrangement.

The robot base <NUM> includes a navigation and mapping controller <NUM> communicatively connected to the motors <NUM> for directing and driving the autonomous mobile robot <NUM> through the datacenter <NUM>. The robot base <NUM> further includes a plurality of sensors communicatively connected to the controller <NUM> to provide navigational and environmental information to the controller <NUM>.

Among the sensors for managing movement of the autonomous mobile robot <NUM>, the base <NUM> includes an inertial measurement unit (IMU) <NUM> (shown schematically) communicatively connected to the controller <NUM> and disposed within the base body <NUM>. In the present example, the IMU <NUM> is formed from a printed circuit board (PCB), MEMS-based gyroscope and three-axis accelerometer (not separately identified), although the specific implementation of the IMU <NUM> could vary. The IMU <NUM> measures acceleration, angular speed, and the orientation of the autonomous mobile robot <NUM> in space. The IMU <NUM> generally includes a signal processing circuit communicatively connected to the controller <NUM> for providing raw or treated spatial or kinetic data to the controller <NUM>.

The base <NUM> includes two LIDAR assemblies <NUM> communicatively connected to the controller <NUM> for detecting objects and obstacles surrounding the autonomous mobile robot <NUM> in order to map the surroundings for navigation. In the illustrated example, one LIDAR assembly <NUM> is connected to and disposed in an upper portion of the base body <NUM> and the other LIDAR assembly <NUM> is connected to and disposed below the base body <NUM>, but different placement of one or both assemblies <NUM> are contemplated in different examples. Each LIDAR assembly <NUM> has an imaging region <NUM> with a <NUM>° lateral field of view and a range of approximately <NUM> meters, as is illustrated in <FIG>. Depending on specific choice of LIDAR assembly, the exact imaging range and/or field of view could vary in different examples.

In addition to the LIDAR assemblies <NUM>, the base <NUM> also includes five ultrasonic sensors <NUM> communicatively connected to the controller <NUM>. The ultrasonic sensors <NUM> are integrated into the exterior walls of the base body <NUM>, with the ultrasonic sensors <NUM> being arranged to provide a sensing region <NUM> with a <NUM>° field of view around the autonomous mobile robot <NUM>. As is also illustrated in <FIG>, the sensing region <NUM> has a sensing range of approximately <NUM> meters from the base <NUM>. The ultrasonic sensors <NUM> thus provide obstacle detection around the autonomous mobile robot <NUM>, although information from the sensors <NUM> could also be integrated into mapping by the autonomous mobile robot <NUM>. Depending on the example, it is contemplated that the number and capabilities of the ultrasonic sensors <NUM> could vary.

The base <NUM> further includes wheel encoders <NUM> (shown schematically) for measuring movement of the wheels <NUM> in order to monitor distance traveled by the autonomous mobile robot <NUM>. Each wheel encoder <NUM> measures the rotations of a corresponding wheel <NUM>, which provide information on both a distance traveled by each wheel <NUM> (with the wheel radius being known), as well as the relative movement between wheels <NUM>. The wheel encoders <NUM> are communicatively connected to the controller <NUM> and disposed in the base body <NUM>. Depending on the example, it is contemplated that the wheel encoders <NUM> could be omitted in some cases.

The base <NUM> also includes four infrared sensors <NUM> communicatively connected to the controller <NUM>, via the charging module <NUM>. The infrared sensors <NUM> are disposed on a forward side of the base <NUM> in order to sense a small region in front of the autonomous mobile robot <NUM> (when moving in a generally forward direction). As is illustrated in <FIG>, an infrared sensing region <NUM> extends a short distance generally forward from the infrared sensors <NUM>. The infrared sensors <NUM> serve to detect the charging base <NUM>, when in close proximity to the base <NUM>, and to properly position the autonomous mobile robot <NUM> relative to the charging base <NUM> in order to autonomously connect the autonomous mobile robot <NUM> to the charging base <NUM> for charging. In at least some examples, the infrared sensors <NUM> could be omitted, disposed on a backward side of the base <NUM> in order to sense a small region behind the autonomous mobile robot <NUM> (when moving in a generally forward direction) or exchanged for different sensing technology for positioning the robot <NUM> relative to the charging base <NUM>.

The base <NUM> further includes a camera <NUM>, also referred to as a depth camera <NUM>, disposed on a top surface of the base body <NUM>. As will be described further below, the camera <NUM> provides a live and/or recorded view from a front side of the autonomous mobile robot <NUM> to an operator of the autonomous mobile robot <NUM>.

The autonomous mobile robot <NUM> includes a rigid member <NUM>, also referred to as the post <NUM>, connected to and extending generally vertically upward from the base <NUM>. The post <NUM> supports a variety of peripheral components of the autonomous mobile robot <NUM>, as will be described in more detail below. In the illustrated example, the post <NUM> is generally rectangular and defines therein a series of slots for receiving and/or connecting components. It is contemplated that the post <NUM> could be implemented in a variety of forms, including but not limited to a cylindrical form. It is also contemplated that more than one post could be included in the autonomous mobile robot <NUM>, for example for supporting different peripheral components on different posts.

The autonomous mobile robot <NUM> further includes a system manager <NUM>, also referred to as an embedded computer <NUM> or a controller <NUM>, for managing operation of the autonomous mobile robot <NUM>, the controller <NUM> being communicably connected to the navigation and mapping controller <NUM>. The computer <NUM> has a processor unit (not shown) for carrying out executable code, and hard disk storage <NUM>, also referred to as a memory unit <NUM>, operatively connected to the controller <NUM> that stores, inter alia, the executable code in a non-transitory medium (not shown) included in the storage <NUM>. The processor unit includes one or more processors for performing processing operations that implement functionality of the controller <NUM>. The processor unit may be a general-purpose processor or may be a specific-purpose processor comprising one or more preprogrammed hardware or firmware elements (e.g., application-specific integrated circuits (ASICs), electrically erasable programmable read-only memories (EEPROMs), etc.) or other related elements. The non-transitory medium of the storage <NUM> may be a semiconductor memory (e.g., read-only memory (ROM) and/or random-access memory (RAM)), a magnetic storage medium, an optical storage medium, and/or any other suitable type of memory. Management of the autonomous mobile robot <NUM> according to methods of the present technology will be described in more detail below.

The autonomous mobile robot <NUM> includes a Wi-Fi antenna/router <NUM> communicatively and operatively connected to the controller <NUM>. The Wi-Fi antenna/router <NUM> provides wireless communication between the autonomous mobile robot <NUM> (and/or the controller <NUM> of the autonomous mobile robot <NUM>) and related systems, such as an external controller for an operator. Depending on the example, it is contemplated that the autonomous mobile robot <NUM> could communicate with a central controller via the Wi-Fi antenna/router <NUM>.

In order to receive commands and provide information to a user in the data center <NUM>, the autonomous mobile robot <NUM> includes a human-machine interface (HMI) <NUM>. The HMI <NUM> is disposed on a top portion of the post <NUM>. While the HMI <NUM> is arranged in a position to facilitate interaction by the user, it is contemplated that the HMI <NUM> could be differently placed on the post <NUM>. The HMI <NUM> includes a touch screen <NUM> for presenting information to the user and for receiving touch commands from the user. In some examples, it is contemplated that the autonomous mobile robot <NUM> could receive commands only via wireless communication, including but not limited to the Wi-Fi antenna/router <NUM>. In some such cases, the HMI <NUM> and/or the touchscreen <NUM>.

<FIG> illustrates one non-limiting example of a control graphic user interface (GUI) <NUM> on the touch screen <NUM>. The control GUI <NUM> is one non-limiting example of a software-implemented program for providing information to the user and providing an interface for receiving user commands. In the illustrated example, the control GUI <NUM> presents a computer-generated map <NUM> of the structure surveyed by the autonomous mobile robot <NUM>, in this case the datacenter <NUM>. The map <NUM> could be presented in a variety of forms, including but not limited to: an architectural plan and/or a theoretical map saved to the storage <NUM>, and a map created and/or updated by the autonomous mobile robot <NUM> during exploration of the structure. The control GUI <NUM> also presents an image from the camera <NUM> (shown schematically) in order to illustrate the region in front of the autonomous mobile robot <NUM> to the user.

The control GUI <NUM>, in the illustrated example, also includes a variety of control functions selectable via a plurality of icons <NUM>. The control functions could vary depending on the particular example of the autonomous mobile robot <NUM> or needs of the structure surveyed thereby. The control functions could include, but are not limited to: a structure mapping function, a robot navigation function, an inventory activation function, and a structure monitoring function.

According to non-limiting examples, the controller <NUM>, the control GUI <NUM>, and components of the autonomous mobile robot <NUM> are operated using a robot operating system (ROS) according to a publisher/subscriber model. The controller <NUM> is communicatively connected to a plurality of nodes via the Wi-Fi antenna/router <NUM>. In at least some examples, the controller <NUM> is implemented as a remote Next Unit of Computing module (NUC) communicably connected to the base <NUM>. For example, the controller <NUM> may be remotely connected to the base <NUM> and components thereof using the Wi-Fi antenna/router <NUM> for transmitting signals. The controller <NUM> may also be communicably connected to the base <NUM> and components thereof using Ethernet protocol for transmitting signals. In different examples, different control programming is contemplated.

In this example, the autonomous mobile robot <NUM> further includes at least two cameras and an inventory module that, upon being executed by the controller <NUM>, causes autonomous mobile robot <NUM> to take inventory of objects in the data center <NUM> based on data provided by the at least two cameras. The at least two cameras include a first camera <NUM> and a second camera <NUM>. The cameras <NUM>, <NUM> may be Red-Green-Blue (RGB) cameras, Red-Green-Blue-Depth (RGBD) cameras, monochromatic cameras, or any other types of cameras that may provide data suitable for being used by the inventoring module <NUM>. The first camera <NUM> has a lower image resolution than the second camera <NUM> such that data provided by the first camera <NUM> may be manipulated by any data processing system (e.g. the controller <NUM>) with less computing resources. Processing of the data provided by the first camera <NUM> may thus be substantially faster than processing of the data provided by the second camera <NUM>. The data provided by the second camera <NUM> include more information about entities imaged by the second camera <NUM>, given that an image resolution of the second camera <NUM> is substantially higher. For example and without limitation, the first image resolution is 640x480 (i.e. <NUM> pixels across a width of a picture capture by the first camera <NUM>, and <NUM> pixels along a height of said picture) and the second image resolution is 4032x3040. The second image resolution is thus more than six times higher than the first image resolution. Execution of the inventoring module <NUM> is described in greater details here further below with respect to <FIG>.

The following description describes operation of the autonomous mobile robot <NUM> with a single first camera <NUM> and a single second camera <NUM>. It should be noted that, in some examples, the autonomous mobile robot <NUM> may comprise a plurality of first cameras <NUM> and a plurality of second cameras <NUM>. A plurality of point of views may thus be provided by the pluralities of cameras <NUM>, <NUM>. The plurality of first cameras <NUM> operate similarly to the described single first camera <NUM> and in a simultaneous manner with one another. The plurality of second cameras <NUM> operate similarly to the described single second camera <NUM> and in a simultaneous manner with one another. In some other examples, functions of the first and second cameras <NUM>, <NUM> are performed by a unique camera that may selectively capture images at the first and second image resolutions. In an example, said unique camera is a RASPBERRY™ PI HQ camera having a <NUM>° horizontal field of view (HFOV) lens.

Without specifically limiting the current technology, one example operating scheme for the autonomous mobile robot <NUM> is described herein. In one aspect, the autonomous mobile robot <NUM> takes inventory of a plurality of objects within a structure. In this example, the structure is the datacenter <NUM> and the plurality of objects is a plurality of servers housed in the racks <NUM> of the datacenter <NUM>. It is contemplated that the structure and the plurality of objects may differ from the datacenter <NUM> and the servers thereof respectively in alternative examples. As such, any system variation configured to distribute perform any inventory of a plurality of object in a structure can be adapted to execute examples of the present technology, once teachings presented herein are appreciated.

<FIG> illustrates a plurality of objects of a datacenter to be observed by the present technology. More specifically, <FIG> depicts eight (<NUM>) electronic equipment modules <NUM> hosted in one of the racks <NUM> of the data center <NUM>. It should be understood that the rack <NUM> may host a different number of electronic equipment modules <NUM> or other computer processing systems.

In this example, each electronic equipment module <NUM> includes a support unit <NUM> and electronic equipment such as a server or other electronic equipment (e.g., networking equipment) supported by the support unit <NUM>. The support unit <NUM> may provide connecting means for connecting the electronic equipment module <NUM> to the rack <NUM>. For example, the rack <NUM> may include vertical walls <NUM> laterally spaced from one another so as to define a housing section therebetween in which the electronic equipment modules <NUM> can be housed. The support unit <NUM> is connected to and/or supported by an equipment support <NUM>. The equipment supports <NUM> are connected to each vertical wall <NUM> and extend perpendicular thereto. A mounting of the electronic equipment modules <NUM> within the rack <NUM> is not limitative, other means for connecting the electronic equipment modules <NUM> to the rack <NUM> are contemplated in alternative examples. In this example, the rack <NUM> may for example be a <NUM>-inch, standard-size rack having dimensions as defined in the EIA/ECA-<NUM>-<NUM> E "Cabinets, Racks, Panels, And Associated Equipment" standard.

In this example, the rack <NUM> is a server rack and the electronic equipment modules <NUM> housed thereby are server modules or related functional modules (e.g., networking or power supply modules). Broadly speaking, the electronic equipment modules <NUM> are disposed in the rack <NUM> in a same manner for each electronic equipment module <NUM> to form a column of similarly looking electronic equipment modules <NUM>.

In this example, the support unit <NUM> of each electronic equipment module <NUM> includes a front panel <NUM>. An indicator <NUM> is disposed on the front panel <NUM>, which may be an electronic equipment module light (e.g. an Light-Emitting Diode (LED)) for example, for indicating, when lit, that electronic components of the corresponding electronic equipment module <NUM> are electrically powered. Broadly speaking, the indicator <NUM> may be used to indicate presence and/or position of the electronic equipment module <NUM> in the rack <NUM>. An inventory label <NUM> is also disposed on the front panel <NUM>. The inventory label <NUM> may include texts, pictures, bar codes, QR-codes, or any other two-dimensional or three-dimensional indications comprising information about hardware characteristics, software characteristics and/or an identifier of the corresponding electronic equipment module <NUM>. As such, any system suitable to read or scan the inventory label <NUM> may access said information about the corresponding electronic equipment module <NUM>.

<FIG> schematically depicts the autonomous mobile robot <NUM> taking inventory of the plurality of electronic equipment modules <NUM>. As depicted on <FIG>, the autonomous mobile robot <NUM> moves in a robot transport path <NUM> (e.g. an aisle of the data center <NUM>) along a row of racks <NUM>.

In this example, the controller <NUM> may use a navigation planner such as "move_base" algorithm from Robot Operating System (ROS) software library to provide global and local planners to the autonomous mobile robot <NUM>, coupled with "teb_local_planner" from the ROS software library that may optimize a trajectory of the autonomous mobile robot <NUM> with respect to a trajectory execution time, separation from detected obstacles and compliance with kinodynamic constraints at runtime. The navigation planner may thus plan navigation and further cause the autonomous mobile robot <NUM> to navigate along a generated path in the data center <NUM>. During navigation, the autonomous mobile robot <NUM> uses sensors such as the inertial measurement unit (IMU) <NUM>, the LIDAR assemblies <NUM>, the ultrasonic sensors <NUM>, the infrared sensors <NUM> and the camera <NUM> to receive environmental information about an environment of the autonomous mobile robot <NUM>. For example, the LIDAR assemblies <NUM> may detect presence of the row of racks <NUM> and provide the controller <NUM> with data including information about a distance between said row of racks <NUM> and the autonomous mobile robot <NUM> such that the controller <NUM> may cause the autonomous mobile robot <NUM> to navigate at a given distance of the row of racks <NUM>. This may increase a relative safety level of the autonomous mobile robot <NUM> and the racks <NUM> by reducing a probability of collision between the two. As another example, data provided by the camera <NUM> may be inputted in a object-detection algorithm to detect presence of a human entity (e,g, an operator of the data center), an open door of a rack <NUM>, a forklift, or any other object that may be suitably detected by a given object-detection algorithm. Broadly speaking, the controller <NUM> is provided with the environmental information including information about entities present in a vicinity of the autonomous mobile robot <NUM> such that the controller <NUM> may cause and/or adjust a navigation thereof.

In this example, the controller <NUM> accesses a plan of the data center <NUM> and causes the autonomous mobile robot <NUM> to displace through the structure, displacements of the autonomous mobile robot <NUM> being based at least in part on the plan of the structure.

During navigation of the autonomous mobile robot <NUM>, the controller <NUM> executes the inventoring module <NUM>. More specifically, the controller <NUM> causes the first camera <NUM> to capture positioning images at the first image resolution. For example, the first camera <NUM> may continuously capture images (e.g. with an imaging rate of five images per second), the first camera <NUM> being oriented toward the racks <NUM>. In some examples, the first camera provides a stream of images at the first image resolution to the controller <NUM>. Images captured by the first camera <NUM> are indicative of a position of the autonomous mobile robot <NUM> relatively to the racks <NUM>, as may thus be referred to as "positioning images". For example and without limitations, the first camera <NUM> may be fixedly attached to the post <NUM>, such that the first camera <NUM> is oriented toward the racks <NUM> when the autonomous mobile robot <NUM> navigates along the racks <NUM> on the robot transport path <NUM>.

Upon receiving the positioning images, the controller <NUM> processes the positioning images and determine presence of a predetermined landmark, such as the indicator <NUM>. The controller <NUM> may use a form-identifying image treatment algorithm to detect presence of the predetermined landmark in the positioning image. The image of the predetermined landmark may be, for example and without limitation, a shape, a contoured shape, a pattern, a color (e.g. RGB value, HSL value), a variation of color (e.g. gradient of color), a brightness and/or any visual characteristics or combination thereof of a pixel or a group of pixels of the captured positioning image.

In an example, the first camera <NUM> is an RGB camera and the controller <NUM> detects presence of the image of the predetermined landmark upon detecting a circular form having an RGB value within an RGB value range and a brightness value above a predetermined threshold. In this example, detection of the image of the predetermined landmark is indicative of an imaging, by the first camera <NUM>, of one of the indicators <NUM> of the electronic equipment modules <NUM>. Upon detecting the image of the predetermined landmark, the controller <NUM> identifies the positioning image containing the predetermined landmark as a landmarked positioning image.

With reference to <FIG>, the autonomous mobile robot <NUM> is in a position denoted P1 while the first camera <NUM> captures images of the racks <NUM>. For example, the controller <NUM> may identify a landmarked positioning image while the autonomous mobile robot <NUM> is positioned in P1.

In this example, the controller <NUM> uses the form-identifying image treatment algorithm to detect an image of a predetermined repeating landmark and identifies a positioning image as the landmarked positioning image in response to the predetermined repeating landmark being detected in said positioning image. For example, the indicators <NUM> of the electronic equipment module <NUM> of a same rack <NUM> form a column <NUM> of indicators, which can be referred to as a predetermined repeating landmark <NUM> given that it includes a plurality of the aforementioned predetermined landmarks. In other words, the form-identifying image treatment algorithm is used by the controller <NUM> to search for a predetermined repeating form pattern in the given positioning image. The positioning image is further identified as a landmarked positioning image in response to successfully finding the predetermined repeating form pattern in the given image.

In the same example, the first camera <NUM> is a RGB camera and the controller <NUM> detects presence of the image of the predetermined repeating landmark upon detecting a column of circular forms having an RGB value within an RGB value range and a brightness above a predetermined threshold. In other words, the predetermined repeating form pattern is a plurality of occurrences of circular forms having a RGB value within a RGB value range and a brightness value above a predetermined threshold in the positioning image, the circular forms being aligned along a same direction that is, in this example, a generally vertical line. In this example, detection of the image of the predetermined landmark is indicative of an imaging, by the first camera <NUM>, of a column of the indicators <NUM> extending substantially vertically of the electronic equipment modules <NUM> of a same rack <NUM>. Upon detecting the image of the predetermined repeating landmark, the controller <NUM> identifies the positioning image containing the predetermined repeating landmark as a landmarked positioning image.

Upon identifying a landmarked positioning image, the controller <NUM> may cause the autonomous mobile robot <NUM> to stop and determines a position of the autonomous mobile robot <NUM> relatively to the predetermined landmark. To do so, the controller <NUM> may, for example, use data provided by the IMU <NUM>, the LIDAR assemblies <NUM>, the ultrasonic sensors <NUM>, the infrared sensors <NUM> and the camera <NUM>. It is to be understood that in this context, the relative positions of the autonomous mobile robot <NUM> and the predetermined landmark include information about a location and an orientation of the autonomous mobile robot <NUM> relatively to the predetermined landmark and the rack <NUM>.

The controller <NUM> further determines a data collection position relatively to the predetermined landmark or the predetermined repeating landmark. The data collection position and determination thereof are based on the position of the autonomous mobile robot <NUM> during acquisition of the landmarked positioning image (i.e. P1 in this example). Information about the data collection position relatively to the predetermined landmark or the predetermined repeating landmark may be accessed by the controller <NUM> such that the controller <NUM> may cause the autonomous mobile robot <NUM> to reach said data collection position starting from P1.

For example, the controller <NUM> may determine that P1 is <NUM> meters away in a first direction from the predetermined landmark. The controller <NUM> may access information about the data collection position indicative that a location associated with the data collection position is <NUM> meter away from the predetermined landmark, and an orientation of the autonomous mobile robot <NUM> associated with the data collection position is a second predetermined direction. The controller <NUM> may thus determine the location and orientation of the data collection position relatively to P1 and cause the autonomous mobile robot <NUM> to reach said data collection position. It is to be understood that in this context, the autonomous mobile robot <NUM> is said to navigate to a predetermined data collection position in cases where a location and an orientation relative to the predetermine landmark are adjusted according to a predetermined location and angular orientation such that the autonomous mobile robot <NUM> is in the data collection position. In other word, upon identifying the landmarked positioning image and relative positions of the autonomous mobile robot <NUM> and the predetermined landmark or the predetermined repeating landmark, the controller <NUM> cause the autonomous mobile robot <NUM> to adjust a position (i.e. location and orientation of the autonomous mobile robot <NUM>) by navigating to the data collection position. With reference to <FIG>, the autonomous mobile robot <NUM> is in the data collection position denoted P2 and may have adjusted a location and/or an orientation of autonomous mobile robot <NUM> relatively to the predetermined landmark. In this example, the controller <NUM> causes the autonomous mobile robot <NUM> to stop once the autonomous mobile robot <NUM> is in the data collection position.

Once the autonomous mobile robot <NUM> is in the data collection position, the controller <NUM> cause the second camera <NUM> to acquire at least one inventory image at the second image resolution. In this example, the data collection position P2 is defined such that a field of view of the second camera <NUM> include the inventory labels <NUM> when the autonomous mobile robot <NUM> is in the data collection image P2. As shown on <FIG>, the inventory labels <NUM> of the <NUM> of a same rack <NUM> form a label column <NUM>. In this example, the field of view of the second camera <NUM> is set such that the inventory image includes the label column <NUM>. The second camera <NUM> may be fixedly attached to the post <NUM> and adjacent to the first camera <NUM>.

The controller <NUM> further extracts, from the inventory image, the plurality of inventory labels <NUM>. The controller <NUM> may transmit the extracted inventory labels or data red or scanned therefrom to an operator device of an operator of the data center <NUM>. The controller <NUM> may also access a database (e.g. using the Wi-Fi antenna/router <NUM>) to access information about the electronic equipment modules <NUM> such as hardware characteristics, software characteristics and/or identifiers of the corresponding electronic equipment modules <NUM> based on the extracted inventory labels <NUM>. Inventory of the electronic equipment modules <NUM> of a rack <NUM> may be performed. The autonomous mobile robot <NUM> may further proceed with navigating along the robot transport path <NUM> and take inventory of another rack <NUM>.

It should be noted that the autonomous mobile robot <NUM> may successively take inventory of the electronic equipment modules <NUM> of consecutive racks <NUM> along the robot transport path <NUM>. The autonomous mobile robot <NUM> may also receive instructions to navigate to a specific rack <NUM> within the data center <NUM> and take inventory of the electronic equipment modules <NUM> therefrom.

It should be noted that <FIG> illustrates the autonomous mobile robot <NUM> operating in the data center <NUM> for taking inventory of electronic equipment modules <NUM> and more specifically, of servers within the data center <NUM>. As such, any system variation configured to taking inventory of a plurality of objects within a structure can be adapted to execute examples of the present technology, once teachings presented herein are appreciated. Examples and embodiments of the present technology can be equally applied to other types of the structure and other types of objects to take inventory for. For example, the structure may be a warehouse, a mall, a shipping platform, a library, or any other structure where taking inventory may be suitably performed by the autonomous mobile robot <NUM>. For example, the objects may be labelled items, shops, parcels, books, or any other objects that may be detected by the autonomous mobile robot <NUM>.

<FIG> is a flow diagram of a method <NUM> for taking inventory of a plurality of objects within a structure according to some examples of the present technology. In one or more aspects, the method <NUM> or one or more steps thereof may be performed by an autonomous mobile robot, in the present example by the autonomous mobile robot <NUM>. The method <NUM> or one or more steps thereof may be, for example and without limitation, executed by the controller <NUM> or a remote controller communicably connected with the autonomous mobile robot <NUM>. The method <NUM> or one or more steps thereof may be embodied in computer-executable instructions that are stored in a computer-readable medium, such as a non-transitory mass storage device, loaded into memory and executed by a CPU. In this example, the steps of the method <NUM> are executed by a controller of the autonomous mobile robot, such as controller <NUM> of the autonomous mobile robot <NUM>. Some steps or portions of steps in the flow diagram may be omitted or changed in order.

The method <NUM> begins, at step <NUM>, with causing the autonomous mobile robot to navigate through at least a portion of the structure. In an example, the structure is a data center such as the data center <NUM>.

In one example, the controller of the autonomous mobile robot accesses a map or a plan of the structure and further causes the autonomous mobile robot to displace through the structure. The displacements of the autonomous mobile robot may be based at least in part on the plan of the structure. For example, the controller may use a navigation planner to determine the displacements of the autonomous mobile robot such as the aforementioned"move_base" and "teb_local_planner" algorithms. The controller may also use the "map_server" algorithm from the ROS software library to access and/or retrieve a map of the structure from a server or a database communicably connected to the controller.

In one example, the controller of the autonomous mobile robot may acquire environmental information using at least one sensor of the autonomous mobile robot. For example, the controller may communicate with sensors of the autonomous mobile robot such as the inertial measurement unit (IMU) <NUM>, LIDAR assemblies <NUM>, ultrasonic sensors <NUM>, infrared sensors <NUM> and camera <NUM>, and receive data therefrom. Said data may include navigational and environmental information. The controller may further base displacements of the autonomous mobile robot at least in part on the environmental information.

The method <NUM> continues, at step <NUM>, with causing, while navigating through the portion of the structure, at least one camera of the autonomous mobile robot to acquire a plurality of positioning images at a first image resolution. The at least one camera may be the first camera <NUM>. The first image resolution may be, for example and without limitation, 640x480. In some examples, the at least one camera images at least one rack of the datacenter, such as the rack <NUM> to acquire the positioning images.

In an example, the autonomous mobile robot includes a plurality of cameras that may be identical to the first camera <NUM>. The controller of the autonomous mobile robot causes the plurality of cameras to acquire a plurality of sets of positioning images. In this example, the plurality of cameras simultaneously image different areas of the structure. For example, the cameras may simultaneously image a front side of the rack <NUM> depicted on <FIG>.

The method <NUM> further continues, at step <NUM>, with determining that at least one positioning image of the plurality of positioning images contains an image of a predetermined landmark. The controller of the autonomous mobile robot may process a given image using a form-identifying image treatment algorithm to search for an image of the predetermined landmark for each positioning image. The form-identifying image treatment algorithm may be, for example and without limitation, YOLOv5 available on GitHub™ platform. Other algorithms may be used in addition to or instead of the YOLOv5 algorithm. For example, the controller may detect presence of the image of the predetermined landmark upon detecting a circular form having a RGB value within an RGB value range and a brightness value above a predetermined threshold in a given positioning image. The positioning image containing the image of the predetermined landmark may be referred to as a landmarked positioning image.

In this example, the controller may identify a positioning image as a landmarked positioning image upon determining presence of a predetermined repeating landmark in said positioning image. The controller of the autonomous mobile robot may use the form-identifying image treatment algorithm to search for a predetermined repeating form pattern in the given image for each of the plurality of positioning images. For example, the predetermined repeating form pattern may be a column of circular forms having a RGB value within a RGB value range and a brightness value above a predetermined threshold. In this example, the column of circular forms may correspond to the column <NUM> of indicators such as electronic equipment module light (e.g. LEDs) of the electronic equipment modules <NUM> of a same rack <NUM>. In other words, the circular forms are expected to correspond to the indicators <NUM> (e.g. LEDs) as described in <FIG>. In response to successfully finding the predetermined repeating form pattern, the controller identifies the given image as containing the image of the predetermined repeating landmark.

The method <NUM> further continues, at step <NUM>, with causing, in response to determining that the at least one positioning image of the plurality of positioning images contains the image of the predetermined landmark, the autonomous mobile robot to reach a predetermined data collection position. More specifically, the controller may access information about a location and an orientation that the autonomous mobile robot should reach relatively to the predetermined landmark. Determination of the location of the predetermined data collection position relatively to a current location of the autonomous mobile robot is based at least in part on a position of the autonomous mobile robot during acquisition of the landmarked positioning image. In an example, the controller causes the autonomous mobile robot to come to a stop in an imaging position relative to the predetermined landmark.

The method <NUM> further continues, at step <NUM>, with causing the at least one camera of the autonomous mobile robot to acquire at least one inventory image at a second image resolution, the second image resolution being greater than the first image resolution. The second image resolution may be, for example and without limitation, 4032x3040.

It should be noted that, in this example, the autonomous mobile robot includes a first camera (e.g. the first camera <NUM>) to acquire images at the first image resolution and a second camera (e.g. the second camera <NUM>) to acquire images at the second image resolution. Data provided by the first camera may thus be manipulated by any data processing system (e.g. the controller <NUM>) with less computing resources compared to data provided by the second camera. Processing of the data provided by the first camera may thus be substantially faster than processing of the data provided by the second camera. On the other hand, data provided by the second camera include more information due to the relative higher image resolution.

In this example, acquiring the inventory images includes imaging inventory labels, such as the inventory labels <NUM>, disposed on a front surface of the electronic equipment modules <NUM>. More specifically, it is expected that the inventory labels are included in a field of view of the second camera when the autonomous mobile robot is in the data collection position.

In an example, the controller of the autonomous mobile robot causes the at least one camera to acquire the plurality of positioning images at the first image resolution and the at least one inventory image at the second image resolution at a same distance to the plurality of objects.

Claim 1:
A method (<NUM>) for taking inventory of a plurality of objects within a structure, the method comprising:
causing an autonomous mobile robot (<NUM>) to navigate through at least a portion of the structure;
while navigating through the portion of the structure, causing at least one camera (<NUM>, <NUM>) of the autonomous mobile robot (<NUM>) to acquire a plurality of positioning images at a first image resolution;
determining that at least one positioning image of the plurality of positioning images contains an image of a predetermined landmark;
in response to determining that the at least one positioning image of the plurality of positioning images contains the image of the predetermined landmark, causing the autonomous mobile robot (<NUM>) to navigate to a predetermined data collection position, the predetermined data collection position being based at least in part on a position of the autonomous mobile robot (<NUM>) during acquisition of the at least one positioning image;
the method being characterized in that the method further comprises:
causing the at least one camera (<NUM>, <NUM>) of the autonomous mobile robot (<NUM>) to acquire at least one inventory image at a second image resolution, the second image resolution being greater than the first image resolution; and
extracting, from the at least one inventory image, a plurality of inventory labels (<NUM>), each inventory label (<NUM>) relating to a corresponding object of the plurality of objects within the structure.