Patent ID: 12233533

DETAILED DESCRIPTION

Overview

Robots configured to deliver mail, packages, food, groceries, concierge or room service, and/or other goods or services are often deployed in populated areas (e.g., urban environments, inside buildings). Use of such robots may reduce traffic and emissions while also lowering overhead (e.g., fuel and/or other costs) when compared to human, automobile, or truck-based delivery. As commercial and private operators cause these robots to appear more frequently in populated areas, there may be little oversight for ensuring robots operate in an efficient, safe, and unobtrusive manner among traffic and pedestrians. For example, robots may be free to operate according to their operators' commercial or private interests regardless of potential negative effects on traffic, pedestrian safety, or public opinion in the adoption of robots in populated areas.

Furthermore, without oversight, these robots often struggle to deal with real-world scenarios, such as overcrowded streets, elevators, hallways with pedestrians or other objects, handicapped people traveling on the sidewalk, other robots, and/or other vehicles in bike lanes and cross intersections. For example, when two robots are on a head-on collision scenario, they may simply stop within some proximity of each other and wait for the other to move. Not only do the goods fail to get delivered, but the robots can make the traffic situations they are supposed to help worse.

In some examples, the techniques and systems herein enable robot management in populated areas. Specifically, a robot management system receives, from a robot system, a requested route and time for the robot system to travel. A dynamic map that includes dynamic and temporal information about an area of the requested route is then used to determine whether the requested route and time are approved, approved with a modified route or time, or disapproved. An approval message, a modified approval message, or a disapproval message is then generated and sent to the robot system. By using the described techniques, municipalities or businesses can monitor, manage, and regulate robots across various disparate robot operators. Doing so may minimize densities of robots along various routes, disruptions to the services of such robots, and/or disruptions to surrounding pedestrians and vehicles.

EXAMPLE ENVIRONMENT

FIG.1illustrates an example environment100where robot management in populated areas may be used. The example environment100includes many aspects of a typical urban environment or city (e.g., roads, buildings, cars, objects, construction zones) and robots102(e.g., robots102A and102B) with respective robot modules104(e.g.,104A and104B), which may be implemented in software, hardware, or a combination thereof. Although not shown, the environment100may also contain building interiors (e.g., insides, hallways, and/or elevators of malls, hotels, and/or other buildings) that the robots102may operate within. The robots102may comprise any number of autonomous vehicles and/or devices. For example, the robots102may be LMD robots, drones, or other robots. Although only two robots102are shown, any number of robots102may exist within the urban environment.

The robots102may be owned, leased, rented, operated, and/or hired by one or more operators, which may be individual, private, commercial, or government entities. For example, robots102may both be associated with a single delivery company, or they may be associated with separate delivery companies. Furthermore, the robot modules104(or portions thereof) may be remote to the robots102. For example, a single robot module104may exist for multiple robots102. In this way, each operator may have a robot module104in addition to, or instead of, each robot102(performing different functionalities of the robot module104). Regardless of how it is implemented, the robot module104is configured to request routes and times for at least one robot102to travel along and to receive approval messages, modified approval messages, and disapproval messages corresponding to those requested routes and times. Functionality of the robot module104is discussed further below.

The example environment100also includes a robot manager106with a robot management module108, which may be implemented in software, hardware, or some combination thereof. The robot manager106may be any system configured to communicate with the robots102and/or the robot modules104(e.g., private or public server or cloud system). The robot management module108is configured to manage operation of the robots102(e.g., manage routes, approve/disapprove routes, maintain a dynamic map, monitor robots102as they travel along routes). Functionality of the robot management module108is discussed further below.

The example environment100further includes a vehicle110with a vehicle perception module112, which may be implemented in software, hardware, or some combination thereof and an infrastructure114with an infrastructure perception module116, which may be implemented in software, hardware, or some combination thereof. The vehicle110may be any automobile, truck, bicycle, or other vehicle with sensor systems able detect attributes of the robots102. The infrastructure114may be any device capable of detecting attributes of the robots102(e.g., streetlight, smart device, cell phone, interior or exterior camera, cell tower).

The vehicle perception module112and the infrastructure perception module116are configured to identify attributes of robots102as they progress around the environment and communicate the attributes to the robot manager106. Functionality of the vehicle perception module112and the infrastructure perception module116is discussed further below.

EXAMPLE SYSTEMS

FIG.2illustrates an example robot system200configured to be disposed in a robot102and configured to implement robot management in populated areas. Components of the robot system200may be arranged anywhere within or on the robot102. The robot system200may include at least one processor202, computer-readable storage media204(e.g., media, medium, mediums), and a communication system206. The components are operatively and/or communicatively coupled via a link208(e.g., wired or wireless).

The processor202(e.g., application processor, microprocessor, digital-signal processor (DSP), controller) is coupled to the computer-readable storage media204via the link208and executes instructions (e.g., code) stored within the computer-readable storage media204(e.g., non-transitory storage device such as a hard drive, solid-state drive (SSD), flash memory, read-only memory (ROM)) to implement or otherwise cause the robot module104(or a portion thereof) to perform the techniques described herein. Although shown as being within the computer-readable storage media204, the robot module104may be a stand-alone component (e.g., having dedicated computer-readable storage media comprising instructions and/or executed on dedicated hardware, such as a dedicated processor, pre-programmed field-programmable-gate-array (FPGA), system on chip (SOC), and the like). Furthermore, portions of the robot module104may exist outside the robot102acting on behalf of the robot102(e.g., in a centralized computer system of an operator). The processor202and the computer-readable storage media204may be any number of components, comprise multiple components distributed throughout the robot102, located remote to the robot102, dedicated or shared with other components, modules, or systems of the robot102, and/or configured differently than illustrated without departing from the scope of this disclosure.

The computer-readable storage media204also contains sensor data210generated by one or more sensors or types of sensors (not shown) that may be local or remote to the robot system200. The sensor data210indicates or otherwise enables the determination of information usable to perform the techniques described herein. In some implementations, the sensor data210may come from a remote source (e.g., via communication system206).

The communication system206may be any number of systems or components configured to communicate with remote systems. For example, the communication system206may be configured to communicate with other robots102, the robot manager106, a central operator system in communication with the robot manager106, the vehicle110, and/or the infrastructure114. The communication system206may communicate via any wired or wireless connection and/or communication protocol (e.g., ethernet, fiberoptic, Wi-Fi, Bluetooth, everything to everything (X2X), dedicated short-range communications (DSRC), internet-of-things (IoT), cellular).

FIG.3illustrates an example robot manager system300configured to be disposed in the robot manager106and configured to implement robot management in populated areas. Components of the robot manager system300may be arranged anywhere within the robot manager106. The robot manager system300may include at least one processor302, computer-readable storage media304(e.g., media, medium, mediums), and a communication system306. The components are operatively and/or communicatively coupled via a link308(e.g., wired or wireless).

The processor302(e.g., application processor, microprocessor, digital-signal processor (DSP), controller) is coupled to the computer-readable storage media304via the link308and executes instructions (e.g., code) stored within the computer-readable storage media304(e.g., non-transitory storage device such as a hard drive, solid-state drive (SSD), flash memory, read-only memory (ROM)) to implement or otherwise cause the robot management module108(or a portion thereof) to perform the techniques described herein. Although shown as being within the computer-readable storage media304, the robot management module108may be a stand-alone component (e.g., having dedicated computer-readable storage media comprising instructions and/or executed on dedicated hardware, such as a dedicated processor, pre-programmed field-programmable-gate-array (FPGA), system on chip (SOC), and the like). The processor302and the computer-readable storage media304may be any number of components, comprise multiple components distributed throughout the robot manager106, located remote to the robot manager106, dedicated or shared with other components, modules, or systems of the robot manager106, and/or configured differently than illustrated without departing from the scope of this disclosure.

The computer-readable storage media304also contains sensor data310generated by one or more sensors or types of sensors (not shown) that may be local or remote to the robot manager system300. The sensor data310indicates or otherwise enables the determination of information usable to perform the techniques described herein. In some implementations, the sensor data310may come from a remote source (e.g., via communication system306).

The communication system306may be any number of systems or components configured to communicate with remote systems. For example, the communication system306may be configured to communicate with robots102, a central operator system in communication with the robots102, the vehicle110, and/or the infrastructure114. The communication system306may communicate via any wired or wireless connection and/or communication protocol (e.g., ethernet, fiberoptic, Wi-Fi, Bluetooth, X2X, dedicated short-range communications (DSRC), internet-of-things (IoT), cellular).

FIG.4illustrates an example vehicle system400configured to be disposed in the vehicle110and configured to implement robot management in populated areas. Components of the vehicle system400may be arranged anywhere within the vehicle110. The vehicle system400may include at least one processor402, computer-readable storage media404(e.g., media, medium, mediums), and a communication system406. The components are operatively and/or communicatively coupled via a link408(e.g., wired or wireless).

The processor402(e.g., application processor, microprocessor, digital-signal processor (DSP), controller) is coupled to the computer-readable storage media404via the link408and executes instructions (e.g., code) stored within the computer-readable storage media404(e.g., non-transitory storage device such as a hard drive, solid-state drive (SSD), flash memory, read-only memory (ROM)) to implement or otherwise cause the vehicle perception module112(or a portion thereof) to perform the techniques described herein. Although shown as being within the computer-readable storage media404, the vehicle perception module112may be a stand-alone component (e.g., having dedicated computer-readable storage media comprising instructions and/or executed on dedicated hardware, such as a dedicated processor, pre-programmed field-programmable-gate-array (FPGA), system on chip (SOC), and the like). The processor402and the computer-readable storage media404may be any number of components, comprise multiple components distributed throughout the vehicle110, located remote to the vehicle110, dedicated or shared with other components, modules, or systems of the vehicle110, and/or configured differently than illustrated without departing from the scope of this disclosure.

The computer-readable storage media404also contains sensor data410generated by one or more sensors or types of sensors (not shown) that may be local or remote to the vehicle system400. The sensor data410indicates or otherwise enables the determination of information usable to perform the techniques described herein. In some implementations, the sensor data410may come from a remote source (e.g., via communication system406).

The communication system406may be any number of systems or components configured to communicate with remote systems. For example, the communication system406may be configured to communicate with robots102, a central operator system in communication with the robots102, the robot manager106, other vehicles110, and/or the infrastructure114. The communication system406may communicate via any wired or wireless connection and/or communication protocol (e.g., ethernet, fiberoptic, Wi-Fi, Bluetooth, X2X, dedicated short-range communications (DSRC), internet-of-things (IoT), cellular).

FIG.5illustrates an example infrastructure system500configured to be disposed in the infrastructure114and configured to implement robot management in populated areas. Components of the infrastructure system500may be arranged anywhere within the infrastructure114. The infrastructure system500may include at least one processor502, computer-readable storage media504(e.g., media, medium, mediums), and a communication system506. The components are operatively and/or communicatively coupled via a link508(e.g., wired or wireless).

The processor502(e.g., application processor, microprocessor, digital-signal processor (DSP), controller) is coupled to the computer-readable storage media504via the link508and executes instructions (e.g., code) stored within the computer-readable storage media504(e.g., non-transitory storage device such as a hard drive, solid-state drive (SSD), flash memory, read-only memory (ROM)) to implement or otherwise cause the infrastructure perception module116(or a portion thereof) to perform the techniques described herein. Although shown as being within the computer-readable storage media504, the infrastructure perception module116may be a stand-alone component (e.g., having dedicated computer-readable storage media comprising instructions and/or executed on dedicated hardware, such as a dedicated processor, pre-programmed field-programmable-gate-array (FPGA), system on chip (SOC), and the like). The processor502and the computer-readable storage media504may be any number of components, comprise multiple components distributed throughout the infrastructure114, located remote to the infrastructure114, dedicated or shared with other components, modules, or systems of the infrastructure114, and/or configured differently than illustrated without departing from the scope of this disclosure.

The computer-readable storage media504also contains sensor data510generated by one or more sensors or types of sensors (not shown) that may be local or remote to the infrastructure system500. The sensor data510indicates or otherwise enables the determination of information usable to perform the techniques described herein. In some implementations, the sensor data510may come from a remote source (e.g., via communication system506).

The communication system506may be any number of systems or components configured to communicate with remote systems. For example, the communication system506may be configured to communicate with robots102, a central operator system in communication with the robots102, the robot manager106, the vehicle110, and/or other pieces of infrastructure114. The communication system506may communicate via any wired or wireless connection and/or communication protocol (e.g., ethernet, fiberoptic, Wi-Fi, Bluetooth, X2X, dedicated short-range communications (DSRC), internet-of-things (IoT), cellular).

EXAMPLE ROUTE MANAGEMENT

FIG.6illustrates, at600, an example flow of route management for robot management in populated areas. The example flow contains two robot modules104that may be associated with two robots or respective operators (acting on behalf of one or more robots). Each robot module104contains a route request module602that is configured to generate a requested route/time604. The requested route/time604may contain an identifier of an associated robot (e.g., serial number (alphanumeric), registration number (alphanumeric), or other identity or identifier usable to differentiate robots102), a requested path, and a time of day (and, in some implementations, a date) the associated robot would like to travel.

Although two requested route/times604are shown, the system may function with any number of requested route/times604. Continuing the previous example, the requested route/time604A may be associated with robot102A and requested route/time604B may be associated with robot102B. In the case of the robot102A and the robot102B being from a single operator, there may only be a single robot module104and a single route request module602generating both the requested route/time604A and the requested route/time604B.

The robot modules104are in communication with the robot management module108and transmit the requested route/times604to the robot management module108. The robot modules104(or a single robot module104) may send the requested route/times604concurrently or separately. In the case of a single route request module602for a plurality of robots, requested route/times604may be batched and transmitted for receipt by the robot management module108.

Each of the requested routes/times604may be received by a route management module606that generates an approval608, a modified approval610, or a disapproval612of the respective requested route/time604. The approval608is indicative that the associated robot may travel along the requested route/time604. The modified approval610is indicative that the associated robot may travel along a modification of the requested route/time604(e.g., a different path and/or a different time). The disapproval612is indicative that the associated robot may not travel along the requested route/time604.

To determine whether to issue the approval608, the modified approval610, or the disapproval612, the route management module606may first determine, at decision614, whether the associated robot is registered. The route management module606may compare the identifier of the associated robot (e.g., received as part of the requested route/time604) to a plurality of registered robots616. If the associated robot is not one of the registered robots616(e.g., a “no” out of decision614), the route management module606may issue the disapproval612.

If the associated robot is one of the registered robots616(e.g., a “yes” out of decision614), the route management module606may determine, at decision618, whether the requested route/time604are ok. If the requested route/time604is ok (e.g., a “yes” out of decision618), the route management module606may issue the approval608.

If the requested route/time604are not ok (e.g., a “no” out of decision620), the route management module606may determine, at decision620, whether a modification of the requested route/time604are ok. If a modification of the requested route/time604is ok (e.g., a “yes” out of decision620), the route management module606may issue the modified approval610(e.g., with a modified path and/or time). If a modification of the requested route/time604cannot be determined within certain parameters such as a time frame or distance variance (e.g., a “no” out of decision620), the route management module606may issue the disapproval612. Attributes of the environment surrounding the requested route/time604are used to make the decision618and the decision620, as discussed further below.

The approval608, the modified approval610, or the disapproval612is then transmitted back to the associated route request module602(e.g., via a transmission of an approval message, a modified approval message, or a disapproval message, respectively). In the case of the approval608and the modified approval610, the associated robot may then progress along the corresponding path and at the corresponding time (e.g., along the approved route).

The robot module104may be configured to transmit periodic messages/updates as the robot102travels along the approved route. For example, the robot module104may transit messages when a route is started, at certain intervals or segments along the route, and when the route is completed. The robot module104may also transmit messages about locations or other information about the robot102when the robot102is idle.

In some implementations, the decision620may not occur. For example, if the route management module606is not configured to determine alternatives or modifications to the requested route/time604, then a “no” out of decision618may result in the disapproval612. In such cases (similar to receiving the disapproval612resulting from the “no” out of decision620), the route request module602of the robot module104may transmit another requested route/time604. In some implementations, reasons why the requested route/time604was disapproved may be given (e.g., too many robots, too much pedestrian traffic, construction zone, another area is less crowded) such that the route request module602may better determine the other requested route/time604. Also, in some implementations, decision614may not occur (e.g., it may be omitted such that the first, and possibly, the only, decision is decision618).

In order to make the decision618and the decision620(if used), the route management module606may utilize a dynamic map622. The dynamic map622is indicative of any map or database that contains dynamic information about the urban environment that the robots travel within. For example, the dynamic map622may contain information about pedestrian densities, traffic densities, construction areas, blockages, weather, events, accidents, emergency vehicles or emergency situations, other robots, “no-go” zones, or any other information that may affect how the associated robot progresses/moves along the route (e.g., they are likely to get stuck, take too long avoiding obstacles, have a run in with another robot). If there is a high density of pedestrian traffic along the requested route, for example, then the route management module606may issue the disapproval612or the modified approval610with a route that has less pedestrian traffic. The information may be real-time or based on historical data. Furthermore, certain aspects may have associated times, e.g., an event is happening on a certain day at a certain time in a certain area.

In some implementations, certain operators may be allowed more robots in an area than others. Similar to how the Federal Communications Commission distributes spectrum for cellular communications to various providers, the robot management module108may license certain densities of robots to respective operators. The allowed densities may then be used to determine whether requested routes/times604are approved.

In some implementations, the size, shape, weight, or other aspects of the robot may be used to determine whether the approval608, the modified approval610, or the disapproval612are issued. For example, the route management module606may determine whether the robot102is too big for the requested route/time604(e.g., it will overflow into a traffic lane or take up too much of a sidewalk).

Other aspects such as “no-cell” zones or businesses that do not want robots102on or near the premises may be included in the dynamic map622. The route management module606may then ensure that robots102avoid such “no-go” areas.

The information is compiled by a map update module624that is configured to maintain the dynamic map622. As part of maintaining the dynamic map622, approved routes626are added to the dynamic map622. The approved routes626are indicative of requested routes/times604that receive the approval608or modifications of the routes/times604as part of the modified approval610. Thus, the map update module624maintains the dynamic map622such that the robot management module108is aware of approved routes and approved times that the registered robots616are traveling.

FIG.7illustrates, at700, an example of route modification for robot management in populated areas. The illustrated example includes robot102A and robot102B. Assume that robot102B will travel along a previously approved route (not shown) at a certain time. Now assume that robot102A generates a requested route/time604A from an origin702to a destination704. The route management module606may determine, based on the dynamic map622, that robot102A will intersect with robot102B along the requested route/time604A or that there are too many approved routes626in an area surrounding the requested route/time604A at the time of the requested route/time604A (e.g., robot density will be over a certain pre-selected value).

Either way, the route management module606determines that the robot102A should not be allowed along the requested route/time604A. In one implementation, the route management module606may issue the disapproval612, such that the route request module602A can request an alternate route/time (e.g., that, if approved, becomes the approved route322A). In another implementation, the route management module606may determine the alternative route/time and issue the modified approval610with the approved route322A.

There may be factors other than other robots102or robot densities usable to manage requested routes/times604(e.g., those discussed above). For example, consider that there may be a large amount of pedestrian traffic coming from a pier at location706at a certain time but not a lot of people near a mall at location708. Since the dynamic map622may contain such information, the route management module606(or the route request module602A with help from the route management module606) may determine that the approved route322A avoids the congestion and, thus, is better.

FIG.8illustrates an example method800of route management for robot management in populated areas. The example method800may be implemented in any of the previously described environments, by any of the previously described systems or components, and by utilizing any of the previously described flows or techniques. The example method800may also be implemented in other environments, by other systems or components, and utilizing other flows or techniques. Example method800may be implemented by one or more entities (e.g., the robot management module108). The order in which the operations are shown and/or described is not intended to be construed as a limitation, and the order may be rearranged without departing from the scope of this disclosure. Furthermore, any number of the operations can be combined with any other number of the operations to implement the example process flow or an alternate process flow.

At step802, a requested route for a robot system to travel along and a requested time for the robot system to travel along the requested route is received from the robot system. For example, the route management module606may receive a requested route/time604from the robot system200associated with a robot102.

At step804, a dynamic map containing information about the requested route and surrounding areas of a city at the requested time and other times is received. For example, the route management module606may receive the dynamic map622. The times and other times are represented by the temporal aspects (e.g., time domain) of the dynamic map622.

At step806, it is determined, based on the dynamic map, whether the requested route and the requested time are approved. For example, the route management module606may determine an outcome to decision618.

At step808, it is determined, based on the dynamic map, whether the requested route and the requested time are approved with at least one of a modified route or a modified time. For example, the route management module606may determine an outcome to decision620.

At step810, it is determined, based on the dynamic map, whether the requested route and the requested time are disapproved. For example, the route management module606may determine an outcome to decision614and/or decision620.

At step812, an approval message, a modified approval message, or a disapproval message is generated based on whether the requested route and the requested time are approved, approved with a modification, or disapproved, respectively. For example, the route management module606may determine whether to issue the approval608based on decision618, whether to issue the modified approval610based on decision620, and whether to issue the disapproval612based on decision614and/or decision620.

At step814, the approval message, the modified approval message, or the disapproval message is sent to the robot system. For example, the route management module606may send the approval608, the modified approval610, or the disapproval612to the robot system200associated with the robot102.

Although described in relation to an outside environment, the techniques and systems described above may be easily adapted for internal environments. For example, the route management module606may determine that certain hallways or areas of an interior of a building are occupied (e.g., based on the dynamic map622having information about the interior of the building) and reroute robots102to avoid the congestion.

EXAMPLE MAP UPDATES

FIG.9illustrates, at900, an example flow of updating a dynamic map for robot management in populated areas. As discussed above, the map update module624may update the dynamic map622to have information about the approved routes626. In order to get information about other attributes of the urban environment (e.g., traffic, densities, weather, accidents, blockages), the map update module624may pool information from any number of remote sources.

Along with traditional databases (e.g., GIS data, police data, internet data), the map update module624may obtain information from the vehicle perception module112, the infrastructure perception module116, and/or a robot perception module902of the robot module104(collectively referred to as perception modules). The perception modules may be able to determine respective environment attributes904and communicate them with the map update module624.

The perception modules may use data from any number of sensors to determine any number of attributes about environments surrounding the vehicle110, the infrastructure114, and/or the robot102, respectively. The attributes may comprise aspects such as vehicle density, pedestrian density, street widths, sidewalk widths, bike lane widths, crosswalk availabilities, road closures, construction work, broken down vehicles, accidents, emergency vehicles, weather, road/sidewalk conditions, objects (lights, signs, etc.), events, and so on. For example, the perception modules may use radar, lidar, and/or camera data (or a combination thereof) to determine densities of vehicles and pedestrians proximate their locations.

The perception modules may form the environment attributes904based on the attributes they determine from the sensor data and communicate the environment attributes904to the map update module624via their respective communication systems (e.g., via environment messages). In this way, the dynamic map622contains temporal information (e.g., real-time and/or predictive based on data analytics) about aspects usable by the route management module606to determine whether to approve requested routes or not, as discussed above.

EXAMPLE ROUTE VERIFICATION

FIG.10illustrates, at1000, an example flow of route verification for robot management in populated areas. If robots102are only operating under the techniques discussed above (e.g., traveling along approved routes), there may not be any problems with their use. There is, however, a possibility that robots102may travel along unapproved routes, spoof their locations, and/or travel unregistered. To mitigate such situations, the robot management module108may leverage one or more of the perception modules to monitor their activity.

The perception modules may determine, based on sensor data, that a robot102is within a vicinity and determine robot attributes1002of the robot102. The robot attributes1002may comprise a registration or serial number, an operator or company, a location, a heading, a speed, an acceleration, a size, or a shape of the associated robot102.

For example, the vehicle perception module112may use the sensor data410to determine that a robot102is crossing in front of the vehicle110. Various techniques may be used to identify the robot102from other objects. The vehicle perception module112may then determine a serial number of the robot (e.g., based on optical or other sensor data or a message received from the robot102). The vehicle perception module112may then estimate a location of the robot102and send the location along with the serial number (or other identifying information if a serial number is not determined) to a route verification module1004of the robot management module108.

The route verification module1004may receive the robot attributes1002and compare them to the approved routes626, at decision1006, to determine if the robot attributes1002are expected (e.g., a registered robot on an approved route626). If the robot attributes1002are not expected (e.g., a “no” out of decision1006), the route verification module1004may issue a violation indication1008.

The violation indication1008may be used to fine the operator of the robot102. Also, the violation indication1008may be sent to local law enforcement for other disciplinary actions (e.g., robot confiscation). By using these techniques, the robot management module108may better ensure that robots102only travel along approved routes626(or within some proximity of them).

FIG.11illustrates, at1100, an example of generating robot attributes1002for use in route verification. In the illustrated example, a robot102is within a sensor range of the vehicle110. The vehicle perception module112uses the sensor data410(e.g., radar, lidar, cameras, some combination thereof) to detect the robot102and determine the robot attributes1002of the robot102.

To do so, an object classification module1102may receive the sensor data410and identify the robot102from the sensor data410(e.g., identified robot1104). The object classification module (or another module) may determine the robot attributes1002of the identified robot1104from the sensor data410. For example, the object classification module1102may determine a location, heading, and/or speed of the identified robot1104relative to a vehicle coordinate system and, from that, determine an approximate global location, heading, and speed of the identified robot1104. Furthermore, the object classification module1102may determine a serial number, barcode, symbol, identity, and/or operator of the identified robot1104from the sensor data. To do so, the object classification module1102may determine alphanumeric characters or a barcode on the identified robot1104.

The vehicle perception module112may then generate a robot message1106that contains the robot attributes1002(including an identifier of the identified robot1104). The robot message1106may be configured for transmission according to various protocols (e.g., X2X, V2X, DSRC, HTTP, internet-of-things (IoT), cellular). Furthermore, the vehicle perception module112may package the robot attributes within the robot message1106such that route verification module1004can determine if the identified robot1104is on an approved route626and/or operating as expected. For example, various fields may exist for various attributes. The vehicle perception module112may then send the robot message1106to the robot management module108for processing by the route verification module1004.

The illustrated example also depicts the infrastructure114(e.g., a streetlight). The infrastructure perception module116(not shown) may act similarly to the vehicle perception module112in determining robot attributes1002and sending a robot message1106to the robot management module108.

By using these techniques, the vehicle perception module112may contribute to robot management. Specifically, by using the robot messages1106from the vehicle,110, other vehicles, and the infrastructure114, the vehicle perception module112may leverage vast amounts of sensors in an environment to monitor robot activity. Doing so may ensure that robots are operating as directed/managed and following certain rules.

FIG.12illustrates an example method1200of generating robot attributes1002. The example method1200may be implemented in any of the previously described environments, by any of the previously described systems or components, and by utilizing any of the previously described flows or techniques. The example method1200may also be implemented in other environments, by other systems or components, and utilizing other flows or techniques. Example method1200may be implemented by one or more entities (e.g., the vehicle perception module112). The order in which the operations are shown and/or described is not intended to be construed as a limitation, and the order may be rearranged without departing from the scope of this disclosure. Furthermore, any number of the operations can be combined with any other number of the operations to implement the example process flow or an alternate process flow.

At step1202, sensor data is received indicating aspects of an environment proximate a vehicle that comprises the vehicle system. For example, the object classification module1102of the vehicle system400may receive the sensor data410about the environment proximate the vehicle110.

At step1204, it is determined whether robots are in the environment proximate the vehicle based on the sensor data. For example, the object classification module1102may use any known technique to ascertain that the identified robot1104(corresponding to robot102) is proximate the vehicle110.

At step1206, responsive to determining that a robot is in the environment proximate the vehicle, attributes of the robot are determined based on the sensor data. For example, the object classification module1102may determine the robot attributes1002, including an identifier of the robot102.

At step1208, a robot message including indications of the attributes of the robot is generated based on the attributes of the robot. For example, the vehicle perception module112may generate the robot message1106with indications of the robot attributes1002.

At step1210, the robot message is transmitted for receipt by a robot management system. For example, the robot message1106may be transmitted to the robot management module108of the robot management system300via the communication system406(X2X, cellular, DSRC, V2X, etc.).

EXAMPLES

Example 1a: A method performed by a robot management system, the method comprising: receiving, from a robot system corresponding to a robot, a requested route for the robot to travel along and a requested time for the robot to travel along the requested route; receiving a dynamic map containing information about the requested route and surrounding areas of a city at the requested time and other times; determining, based on the dynamic map, whether the requested route and the requested time are approved; determining, based on the dynamic map, whether the requested route and the requested time are approved with at least one of a modified route or a modified time; determining, based on the dynamic map, whether the requested route and the requested time are disapproved; generating an approval message, a modified approval message, or a disapproval message based on whether the requested route and the requested time are approved, approved with a modification, or disapproved, respectively; and sending the approval message, the modified approval message, or the disapproval message to the robot system.

Example 2a: The method of example 1a, further comprising determining whether the robot is one of a plurality of registered robots, wherein generating the approval message, the modified approval message, or the disapproval message is based further on whether the robot is one of the registered robots.

Example 3a: The method of any preceding example, further comprising, responsive to sending the approval message or the modified approval message, updating the dynamic map with the requested route and the requested time, the modified route and the requested time, the requested route and the modified time, or the modified route and the modified time.

Example 4a: The method of any preceding example, wherein the dynamic map comprises information about approved routes and approved times of other robots.

Example 5a: The method of example 4a, wherein the dynamic map further comprises information about at least one of: vehicle traffic, pedestrian traffic, events, closures, emergency situations, emergency vehicles, sidewalk widths, bike lane widths, streets, or street widths.

Example 6a: The method of any preceding example, wherein receiving the requested route and the requested time and sending the approval message, the modified approval message, or the disapproval message to the robot system are performed via a wireless connection between the robot system and the robot management system.

Example 7a: The method of example 6a, wherein the wireless connection comprises a vehicle-to-everything (V2X) connection, an everything-to-everything (X2X) connection, an internet-of-things (IoT) connection, a dedicated short-range communications (DSRC) connection, a Wi-Fi connection, or a cellular connection.

Example 8a: The method of any preceding example, wherein determining whether the requested route and the requested time are approved, approved with a modification, or disapproved is based further on minimizing a density of robots along the requested route and at the requested time.

Example 9a: The method of any preceding example, further comprising receiving periodic messages from the robot system as the robot progresses along the approved route or the modified route.

Example 10a: The method of any preceding example, further comprising receiving a robot message from a vehicle system or an infrastructure system, the robot message describing attributes of the robot as the robot moves throughout the city.

Example 11a: A robot management system comprising: at least one processor configured to: receive, from a robot system associated with a robot, a requested route for the robot to travel along and a requested time for the robot to travel along the requested route; receive a dynamic map containing information about the requested route and surrounding areas of a city at the requested time and other times; determine, based on the dynamic map, whether the requested route and the requested time are approved; determine, based on the dynamic map, whether the requested route and the requested time are approved with at least one of a modified route or a modified time; determine, based on the dynamic map, whether the requested route and the requested time are disapproved; generate an approval message, a modified approval message, or a disapproval message based on whether the requested route and the requested time are approved, approved with a modification, or disapproved, respectively; and send the approval message, the modified approval message, or the disapproval message to the robot system.

Example 12a: The system of example 11a, wherein: the processor is further configured to determine whether the robot is one of a plurality of registered robots; and the generation of the approval message, the modified approval message, or the disapproval message is based further on whether the robot is one of the registered robots.

Example 13a: The system of example 11a or 12a, wherein the processor is further configured to, responsive to sending the approval message or the modified approval message, update the dynamic map with the requested route and the requested time, the modified route and the requested time, the requested route and the modified time, or the modified route and the modified time.

Example 14a: The system of any of examples 11a-13a, wherein the dynamic map comprises information about approved routes and approved times of other robots and at least one of: vehicle traffic, pedestrian traffic, events, closures, emergency situations, emergency vehicles, sidewalk widths, bike lane widths, streets, or street widths.

Example 15a: The system of any of examples 11a-14a, wherein the receipt of the requested route and the requested time and the transmission of the approval message, the modified approval message, or the disapproval message to the robot system are performed via a wireless connection between the robot system and the robot management system.

Example 16a: The system of example 15a, wherein the wireless connection comprises a vehicle-to-everything (V2X) connection, an everything-to-everything (X2X) connection, an internet-of-things (IoT) connection, a dedicated short-range communications (DSRC) connection, a Wi-Fi connection, or a cellular connection.

Example 17a: The system of any of examples 11a-16a, wherein the determination of whether the requested route and the requested time are approved, approved with a modification, or disapproved is based further on minimizing a density of robots along the requested route and at the requested time.

Example 18a: The system of any of examples 11a-17a, wherein the processor is further configured to receive periodic messages from the robot system as the robot progresses along the approved route or the modified route.

Example 19a: The system of any of examples 11a-18a, wherein the processor is further configured to receive a robot message from a vehicle system or an infrastructure system describing attributes of the robot as the robot moves throughout the city.

Example 20a: Computer-readable storage media comprising instructions that, when executed, cause at least one processor to: receive, from a robot system associated with a robot, a requested route for the robot to travel along and a requested time for the robot to travel along the requested route; receive a dynamic map containing information about the requested route and surrounding areas of a city at the requested time and other times; determine, based on the dynamic map, whether the requested route and the requested time are approved; determine, based on the dynamic map, whether the requested route and the requested time are approved with at least one of a modified route or a modified time; determine, based on the dynamic map, whether the requested route and the requested time are disapproved; generate an approval message, a modified approval message, or a disapproval message based on whether the requested route and the requested time are approved, approved with a modification, or disapproved, respectively; and send the approval message, the modified approval message, or the disapproval message to the robot system.

Example 1b: A method performed by a vehicle system, the method comprising:

receiving sensor data indicating aspects of an environment proximate a vehicle that comprises the vehicle system; determining, based on the sensor data, whether robots are in the environment proximate the vehicle; responsive to determining that a robot is in the environment proximate the vehicle, determining, based on the sensor data, attributes of the robot; generating, based on the attributes of the robot, a robot message including indications of the attributes of the robot; transmitting, for receipt by a robot management system, the robot message.

Example 2b: The method of example 1b, wherein transmitting the robot message comprises transmitting the robot message via a vehicle to everything (V2X) or dedicated short range communications (DSRC) protocol.

Example 3b: The method of example 1b or 2b, wherein: the attributes of the robot comprise an identity of the robot that is specific to the robot and usable to differentiate the robot from other robots; and the robot message includes a field indicating the identity of the robot.

Example 4b: The method of example 3b, wherein determining the attributes of the robot comprises determining the identity of the robot based on optical sensor data.

Example 5b: The method of example 3b or 4b, wherein determining the identity comprises determining a serial number of the robot from detected alphanumeric characters or a barcode that is disposed on the robot.

Example 6b: The method of example 3b, wherein determining the identity comprises receiving a serial number of the robot from the robot.

Example 7b: The method of example 6b, wherein receiving the serial number of the robot comprises receiving the serial number via a vehicle to everything (V2X) or dedicated short range communications (DSRC) protocol.

Example 8b: The method of any of examples 1b-7b, wherein: the attributes of the robot comprise at least one of: a location of the robot, a heading of the robot, a speed of the robot, an acceleration of the robot, or a size of the robot; and the robot message includes respective fields that indicate the attributes of the robot.

Example 9b: The method of examples 1b-8b, further comprising: determining, based on the sensor data, attributes of the environment proximate the vehicle; generating, based on the attributes of the environment proximate the vehicle, an environment message including indications of the attributes of the environment proximate the vehicle; and transmitting, for receipt by the robot management system, the environment message.

Example 10b: The method of example 9b, wherein: the attributes of the environment comprise at least one of: an amount of traffic proximate the vehicle, an amount of pedestrians proximate the vehicle, weather proximate the vehicle, an event proximate the vehicle, a closure proximate the vehicle, or an emergency vehicle proximate the vehicle; and the environment message includes respective fields that indicate the attributes of the environment.

Example 11b: A vehicle system configured to be disposed in a vehicle, the vehicle system comprising: at least one processor configured to: receive sensor data from one or more sensors indicating an environment proximate the vehicle; determine, based on the sensor data, whether robots are in the environment proximate the vehicle; responsive to determining that a robot is in the environment proximate the vehicle, determining, based on the sensor data, attributes of the robot; generate, based on the attributes of the robot, a robot message indicating the attributes of the robot; and transmit, for receipt by a robot management system, the robot message.

Example 12b: The vehicle system of example 11b, wherein the processor is further configured to transmit the robot message via a vehicle to everything (V2X) or dedicated short range communications (DSRC) protocol.

Example 13b: The vehicle system of example 11b or 12b, wherein: the attributes of the robot comprise an identity of the robot that is specific to the robot and usable to differentiate the robot from other robots; and the robot message includes a field indicating the identity of the robot.

Example 14b: The vehicle system of example 13b, wherein the processor is further configured to determine the identity of the robot based on optical sensor data.

Example 15b: The vehicle system of example 13b or 14b, wherein the processor is further configured to determine a serial number of the robot from detected alphanumeric characters or a barcode that is disposed on the robot.

Example 16b: The vehicle system of examples 11b-15b, wherein the processor is further configured to receive a serial number of the robot from the robot.

Example 17b: The vehicle system of example 16b, wherein the processor is further configured to receive the serial number via a vehicle to everything (V2X) or dedicated short range communications (DSRC) protocol.

Example 18b: The vehicle system of examples 11b-17b, wherein: the attributes of the robot comprise at least one of: a location of the robot, a heading of the robot, a speed of the robot, an acceleration of the robot, or a size of the robot; and the robot message includes respective fields that indicate the attributes of the robot.

Example 19b: The vehicle system of examples 11b-18b, wherein the processor is further configured to: determine, based on the sensor data, attributes of the environment proximate the vehicle, the attributes of the environment proximate the vehicle comprising at least one of: an amount of vehicle traffic proximate the vehicle, an amount of pedestrians proximate the vehicle, weather proximate the vehicle, an event proximate the vehicle, a closure proximate the vehicle, or an emergency vehicle proximate the vehicle; generate, based on the attributes of the environment proximate the vehicle, an environment message including indications of the attributes of the environment proximate the vehicle; and transmit, for receipt by the robot management system, the environment message.

Example 20b: Computer-readable storage media comprising instructions that, when executed, cause at least one processor to: receive sensor data from one or more sensors indicating an environment proximate the vehicle; determine, based on the sensor data, whether robots are in the environment proximate the vehicle; responsive to determining that a robot is in the environment proximate the vehicle, determining, based on the sensor data, attributes of the robot; generate, based on the attributes of the robot, a robot message indicating the attributes of the robot; and transmit, for receipt by a robot management system, the robot message.

CONCLUSION

While various embodiments of the disclosure are described in the foregoing description and shown in the drawings, it is to be understood that this disclosure is not limited thereto but may be variously embodied to practice within the scope of the following claims. From the foregoing description, it will be apparent that various changes may be made without departing from the spirit and scope of the disclosure as defined by the following claims.

The use of “or” and grammatically related terms indicates non-exclusive alternatives without limitation unless the context clearly dictates otherwise. As used herein, a phrase referring to “at least one of” a list of items refers to any combination of those items, including single members. As an example, “at least one of: a, b, or c” is intended to cover a, b, c, a-b, a-c, b-c, and a-b-c, as well as any combination with multiples of the same element (e.g., a-a, a-a-a, a-a-b, a-a-c, a-b-b, a-c-c, b-b, b-b-b, b-b-c, c-c, and c-c-c or any other ordering of a, b, and c).