Systems and methods for autonomous vehicle operation

Disclosed herein are systems and methods for autonomous vehicle operation. A computing system can include a communication device configured to receive a plurality of event signals from at least a first autonomous vehicle that is traversing a path, and a processor in electrical communication with the communication device and configured to determine whether the event signals are indicative of an obstacle in a portion of the path. The communication device can be configured to receive, from at least a second autonomous vehicle, at least one characteristic of the obstacle captured by at least one sensor of the second autonomous vehicle, and transmit, to at least a third autonomous vehicle, at least one task to clear the obstacle from the portion of the path. The processor can be configured to determine, based on the characteristic of the obstacle, the at least one task to be transmitted by the communication device.

TECHNICAL FIELD

The following disclosure is directed to systems and methods for operating autonomous vehicles and, more specifically, systems and methods for mitigating obstacles in autonomous vehicle operation.

BACKGROUND

Warehouses or stores for stocking items are typically organized in rows of storage shelves. The rows are separated by aisles to allow people, carts, vehicles, etc. to travel between the rows to access the shelves. In many instances, aisles may be wide enough for one-way or two-way foot and/or vehicle traffic. Due to the relatively fixed nature of the storage shelves and the limited space within an aisle, an obstacle in an aisle can be a significant impediment to the flow of people and/or vehicles.

SUMMARY

Described herein are exemplary systems and methods for mitigating obstacles in autonomous vehicle operation.

In one aspect, the disclosure features a computing system for autonomous vehicle operation. The computing system can include a communication device configured to receive a plurality of event signals from at least a first autonomous vehicle that is traversing a path, and a processor in electrical communication with the communication device and configured to determine whether the event signals are indicative of an obstacle in a portion of the path. The communication device can be further configured to receive, from at least a second autonomous vehicle, at least one characteristic of the obstacle captured by at least one sensor of the second autonomous vehicle, and transmit, to at least a third autonomous vehicle, at least one task to clear the obstacle from the portion of the path. The processor can be further configured to determine, based on the characteristic of the obstacle, the at least one task to be transmitted by the communication device.

Various embodiments of the computing system can include one or more of the following features.

The communication device can be further configured to transmit the task to a controller of the third autonomous vehicle, in which the controller can be configured to navigate, in response to receiving the task, the third autonomous vehicle to the portion of the path to clear the obstacle according to the task. The communication device can be further configured to receive a signal that the obstacle is cleared from the path. The plurality of event signals can indicate at least one of: (i) decreased speed of the first autonomous vehicle while traversing the path; (ii) increased congestion of vehicles or humans in the path; (iii) deviation of the first autonomous vehicle from the path; or (iv) collision with the obstacle. The at least one sensor can include at least one of a camera, a LiDAR sensor, or a depth sensor. The at least one characteristic of the obstacle can include at least one of a size of the obstacle, a shape of the obstacle, a weight of the obstacle, or a type of the obstacle.

At least two of the group consisting of the first autonomous vehicle, the second autonomous vehicle, and the third autonomous vehicle are a same autonomous vehicle. The at least one task can be included in a task list for the third autonomous vehicle. The processor, in determining whether the event signals are indicative of an obstacle, can be further configured to determine whether the event signals are indicative of vehicle events not within a set of defined events associated with traversal of the path. The processor, in determining the at least one task to be transmitted by the communication device, can be further configured to compare the characteristic of the obstacle to a characteristic of a known obstacle.

In another aspect, the disclosure features a computer-implemented method for autonomous vehicle operation. The method can include receiving, by a computing system, a plurality of event signals from at least a first autonomous vehicle that is traversing a path and determining, by the computing system, whether the event signals are indicative of an obstacle in a portion of the path. The method can include receiving, by the computing system from at least a second autonomous vehicle, at least one characteristic of the obstacle captured by at least one sensor of the second autonomous vehicle; determining, by the computing system and based on the characteristic of the obstacle, at least one task to clear the obstacle from the portion of the path; and transmitting, by the computing system, the task to at least a third autonomous vehicle.

Various embodiments of the method can include one or more of the following features.

The task can be transmitted to a controller of the third autonomous vehicle, and can further include navigating, by the controller in response to receiving the task, the third autonomous vehicle to the portion of the path to clear the obstacle according to the task. The method can include receiving, by the computing system, a signal that the obstacle is cleared from the path. The plurality of event signals can indicate at least one of: (i) decreased speed of the first autonomous vehicle while traversing the path; (ii) increased congestion of vehicles or humans in the path; (iii) deviation of the first autonomous vehicle from the path; or (iv) collision with the obstacle. The at least one sensor can include at least one of a camera, a LiDAR sensor, or a depth sensor. The at least one characteristic of the obstacle can include at least one of a size of the obstacle, a shape of the obstacle, a weight of the obstacle, or a type of the obstacle. At least two of the group consisting of the first autonomous vehicle, the second autonomous vehicle, and the third autonomous vehicle are a same autonomous vehicle. The at least one task is included in a task list for the third autonomous vehicle. Determining whether the event signals are indicative of an obstacle can include determining whether the event signals are indicative of vehicle events not within a set of defined events associated with traversal of the path.

In another aspect, the disclosure features a non-transitory computer-readable medium having instructions stored thereon that, when executed by one or more computer processors, cause the computer processors to perform operations including receiving a plurality of event signals from at least a first autonomous vehicle that is traversing a path, and determining whether the event signals are indicative of an obstacle in a portion of the path. The operations can further include receiving, from at least a second autonomous vehicle, at least one characteristic of the obstacle captured by at least one sensor of the second autonomous vehicle; determining, based on the characteristic of the obstacle, at least one task to clear the obstacle from the portion of the path; and transmitting the task to at least a third autonomous vehicle.

DETAILED DESCRIPTION

Obstacles present in the paths of autonomous vehicles can be detrimental to the productive and efficient operation of an automated warehouse (or a storage space in which items are organized for picking and/or delivery, e.g., a retail store, a grocery store, a hospital, a school, an office, etc.). Autonomous vehicles and/or computing systems can be configured to infer the existence of obstacles in these paths and take action for mitigating (e.g., moving, removing, etc.) the obstacles.

In various embodiments, one or more autonomous vehicles can be routed from one location in a warehouse to another for picking and/or stocking. To reach a destination location, vehicles may be configured to travel routes through aisles that are prescribed or determined on-the-fly. As described further below, vehicles may have predetermined speeds and/or expected operational efficiency which can be hampered by obstacles in their paths.

The technology described herein may be employed in mobile carts of the type described in, for example, U.S. Pat. No. 9,834,380, issued Dec. 5, 2017 and titled “Warehouse Automation Systems and Methods,” the entirety of which is incorporated herein by reference and described in part below.

Exemplary Application to Autonomous Warehouse Carts

FIG.1Adepicts an enhanced cart system100including an enhanced cart102(e.g., an autonomous vehicle). As illustrated, one or more enhanced carts, often referred to in the industry as picking carts, can work alongside one or more warehouse workers104(also referred to as associates) to move inventory items around a warehouse. The enhanced carts102are intended to assist in most warehouse tasks, such as picking, re-stocking, moving, sorting, counting, or verifying items (e.g., products). These carts102can display information to the associate104through the use of a user interface (e.g., screen)106and/or onboard visual and/or audible indicators that improve the performance of the associates104. The cart102can be propelled by a motor (e.g., an electric motor) that is coupled to a power source (e.g., a battery, a supercapacitor, etc.), such that the cart102moves autonomously and does not require being pushed or pulled by a human or other force. The cart102may travel to a charging area to charge its battery or batteries.

Referring still toFIG.1A, the enhanced carts102may be configured to carry one or many similar or distinct storage containers108, often in the form of totes or boxes, that can be used to hold one or more different products. These storage containers108may be removable from the enhanced cart102. In some cases, each container108can be used as a separate picking location (i.e., one container108is a single order). In other cases, the containers108can be used for batch picking (i.e., each container108can contain multiple complete or partial orders). Each container108may be assigned to one or many different stations for post-pick sortation and processing. In one embodiment, one or more of the containers108are dedicated to batch picking of multiple types of products and another one or more containers108are dedicated to picking multiple quantities of a single product (e.g., for orders that only have one item). This singleton picking allows the warehouse to skip secondary sortation and deliver products directly to a packaging station. In another embodiment, one or more of the containers108are assigned to order picking (e.g., for potentially time sensitive orders) and one or more of the containers108are assigned to batch picking (e.g., for lower cost or less time sensitive orders). In yet another embodiment, one or more of the containers108carry product that will be used to re-stock product into storage locations. Another option is for the enhanced cart102to move product and/or shipments throughout the warehouse as needed between different stations, such as packing and shipping stations. In yet another implementation, one or more of the containers108is left empty to assist in counting product into and then back out of the container108as part of a cycle count task regularly carried out in warehouses for inventory management. The tasks may be completed in a mode dedicated to one task type or interleaved across different task types. For example, an associate104may be picking products into container “one” on the enhanced cart102and then be told to grab products from container “two” on the enhanced cart102and put them away in the same aisle.

FIG.1Bis an alternative embodiment of the enhanced cart102, and is shown (for ease of understanding) without the storage containers108being present. As before, the enhanced cart102includes the screen106and lighting indicators110,112. In operation, the storage containers108may be present on the enhanced cart102depicted inFIG.1B. With reference to bothFIGS.1A and1B, the enhanced cart102may include first and second platforms150,154for supporting a plurality of containers108capable of receiving products. At least one support158may support the first platform150above the second platform154. The at least one support158may be substantially centrally-located along respective lengths162,166of the first and second platforms150,154between front and back ends170,174thereof and may support the first and second platforms150,154at locations disposed within interior portions of the first and second platforms150,154. As illustrated inFIG.1B, the front end170of the cart102may define a cutout156. There may be one or more sensors (e.g., light detecting and ranging (LiDAR) sensors) housed within the cutout156. The cutout156permits the sensor(s) to view and detect objects in front of and to the side of (e.g., more than 180° around) the cart102.

The following discussion focuses on the use of autonomous vehicles, such as the enhanced cart102, in a warehouse environment, for example, in guiding workers around the floor of a warehouse and carrying inventory or customer orders for shipping. However, autonomous vehicles of any type can be used in many different settings and for various purposes, including but not limited to: guiding shoppers or stocking inventory in a retail store, driving passengers on roadways, delivering food and medicine in hospitals, carrying cargo in shipping ports, cleaning up waste, etc. The autonomous vehicles can be employed in a warehouse-like environment open to the public (e.g., big box stores or wholesalers). This disclosure, including but not limited to the technology, systems, and methods described herein, is equally applicable to any such type of autonomous vehicle.

Obstacles in Vehicle Paths

In a warehouse setting (or in a retail store, a grocery store, a hospital ward, etc.), a computing system (e.g., a computing system internal or external to an autonomous vehicle102) can determine a path for the autonomous vehicle, thereby enabling the vehicle to collect items located throughout the warehouse according to a picklist (for a customer order) or a task list (e.g., for re-stocking items, moving items, clearing obstacles, etc.). A controller can navigate the vehicle through an optimized sequence of locations within the warehouse such that a worker (also referred to as an associate or picker) or a mechanical device (e.g., a robotic arm coupled to the autonomous vehicle) can physically place an item into a container (also referred to as a tote) for the vehicle to carry. The controller may be a central controller (e.g., part of a remote computing system), a vehicle controller on the autonomous vehicle, or may include two or more controllers (e.g., part of a remote computing system and/or autonomous vehicle) configured to operate together (e.g., via a communication link). For example, a central controller may send instructions to a vehicle controller to navigate the autonomous vehicle about a warehouse to restock items on shelves or collect items for a customer order. In another example, the vehicle controller can be configured to navigate the autonomous vehicle to move items around a warehouse. As discussed above, the warehouse can be organized into a series of aisles, in which each aisle has enough space for one or more vehicles to traverse while collecting items on shelves or racks on one or more sides of the aisle. Accordingly, aisles are typically kept clear of significant obstacles to allow for quick and easy movement of people and/or vehicles.

In some instances, a particular location in the path of the autonomous vehicle (e.g., within an aisle) can be blocked by an obstacle. Obstacles can be of varying size, weight, shape, or material. For example, obstacles in a warehouse environment may include debris, packing material (e.g., cardboard, tape, packing foam, etc.), liquid spills (e.g., from cleaning supplies, shelved products, a roof leak, etc.), containers108, autonomous vehicle parts, etc. In another example, obstacles in a retail environment may include paper signage, receipts, shopping carts, shopping baskets, dropped items, etc.

Some obstacles, e.g., a wooden pallet or a liquid spill, may be large enough or simply impassible so as to block the path of autonomous vehicles102in a given aisle. In such cases, one or more sensors of the vehicle102may detect the obstacle and stop some distance away. Some obstacles, e.g., wire or tape, may be small such that they can be driven over or picked up by a worker accompanying the vehicle. However, even small obstacles can impair the efficient operation of the warehouse by preventing or slowing autonomous vehicles from collecting items and/or restocking inventory along their paths. For example, some small objects may even become stuck to the underside of the vehicle102, requiring intervention by a human to remove.

Computing Systems for Autonomous Vehicle Operation

FIG.2illustrates a system200configured to mitigate obstacles in autonomous vehicle operation. The system200may include a remote computing system202configured to be coupled directly or indirectly to one or more autonomous vehicles102a,102b,102c(collectively referred to as102). For instance, the remote computing system202may communicate directly with the computing system206of an autonomous vehicle102(e.g., via communication channel208). Additionally or alternatively, the remote computing system202can communicate with one or more autonomous vehicles102via a network device of network210. In some embodiments, the remote computing system202may communicate with a first autonomous vehicle (e.g., vehicle102a) via a second autonomous vehicle (e.g., vehicle102b).

The exemplary remote computing system202may include a processor212coupled to a communication device214configured to receive and transmit messages and/or instructions. The exemplary vehicle computing system206may include a processor216coupled to a communication device218and a controller220. The vehicle communication device218may be coupled to the remote communication device214. The vehicle processor216may be configured to process signals from the remote communication device214and/or vehicle communication device218. The controller220may be configured to send control signals to a navigation system and/or other components of the vehicle102, as described further herein.

As discussed herein and unless otherwise specified, the term “computing system” may refer to the remote computing system202and/or the vehicle computing system206. The computing system(s) may receive and/or obtain information about a customer order (e.g., from another computing system or via a network), including the list of items, the priority of the order relative to other orders, the target shipping date, whether the order can be shipped incomplete (without all of the ordered items) and/or in multiple shipments, etc. A processor (e.g., of system202and/or of system206) may process the customer order to determine an optimal path for one or more autonomous vehicles102to collect items in a “picklist” for the order. For example, a picklist of items may be assigned to a single vehicle or to two or more vehicles102.

The determined path may be transmitted to the controller220of the vehicle102. The controller220may navigate the vehicle102in an optimized sequence of stops (also referred to as a trip) within the warehouse to collect the items. At a given stop, a worker near the vehicle102may physically place the item into a container108for the vehicle102to carry. Alternatively or additionally, the autonomous vehicle102may include an apparatus (e.g., a robotic arm) configured to collect items into a container108.

Systems and Methods for Mitigating Obstacles

FIG.3is a flowchart of an exemplary method300for mitigating obstacles in autonomous vehicle operation.FIG.4illustrates an exemplary warehouse configuration400in which one or more autonomous vehicles102are employed to mitigate obstacles.FIG.5illustrates a workflow for mitigating obstacles in autonomous vehicle operation. For the sake of clarity and conciseness,FIGS.3-5are discussed together herein.

One or more systems enabling the automated warehouse can be configured to detect and/or mitigate obstacles. These systems can include (i) a remote computing system configured to manage one or more components of the warehouse, (ii) autonomous vehicles configured to collect items for customer orders or restock inventory, and/or (iii) specialized autonomous vehicles, as described further below. The remote computing system can be configured to communicate with the respective computing systems of the autonomous vehicles, e.g., for transmitting paths for the vehicles to navigate, tasks for completion, etc.

In step302of method, a computing system (e.g., the remote computing system202and/or the vehicle computing system206) can receive signals from one or more vehicles102in a particular path402in an aisle404aor404bof the warehouse400. These signals can be associated with events occurring in the particular path402or aisle404athat may indicate the presence of an obstacle406a(workflow process502). One or more of the following types of event signals may be generated by a vehicle102and/or received by the computing system.

In some embodiments, the event signals can include data from one or more sensors of the vehicle. For example, a vehicle sensor may be a camera, a LiDAR sensor, a depth sensor, etc. In some embodiments, the event signals can include one or more images, video, audio, depth information, etc. In some embodiments, the vehicle102may be equipped with a stereo camera configured to capture visual and depth information for the vehicle's field of view (e.g., via the cutouts156).

The event signals may indicate a vehicle speed slower than expected or prescribed in the path402. For example, the computing system may assign speed limits to one or more paths and/or aisles. A vehicle102may be configured to operate at or below the speed limits according to the path402or aisle404athe vehicle is traveling. In some embodiments, the computing system (e.g.,202or206) may compare the actual speed to assigned speed limits of the path or aisle (or a portion thereof. For example, assigned speed limits may vary according to different portions of the same path or aisle, may depend on a turn radius required for the vehicle to move between paths or aisles, etc. If the actual speed is below an acceptable threshold below the assigned speed (e.g., 5% below, 10% below, etc. of the assigned speed), then an event signal may be generated accordingly.

In another example, the vehicle102may have an internal speed limit that may be the same or different from the path or aisle speed limits. This internal speed limit may be determined based on the particular operating condition (e.g., age, mechanical condition, charge level, weight of the load carried by the vehicle, etc.) of the vehicle102. In some embodiment, the maximum internal speed limit can be based on a distance between the vehicle and the nearest unexpected object detected by the sensor(s). The computing system206of vehicle102may be configured to compare the internal speed limit to the actual speed of the vehicle102to determine whether an event signal should be generated. In some embodiments, a communication device218of the vehicle102may be configured to report its internal speed limit to the remote computing system202periodically, intermittently, upon request, etc. Additionally, the vehicle102may report its actual speed to the remote computing system202continuously, intermittently, upon request, etc. For example, the device218may report the actual speed once per second, per minute, every 3 minutes, every 5 minutes, every 10 minutes, etc. The remote computing system202may compare an actual speed of the vehicle102to a report of the internal speed limit to determine whether an event signal should be generated. If the actual speed is below an acceptable threshold below the internal speed limit (e.g., 5% below, 10% below, etc. of the internal speed limit), then an event signal may be generated accordingly.

In some instances, the event signals may indicate inefficient collecting or restocking of items by vehicles102. This may occur if vehicles102are slowing down to go around or travel alternative routes due to the presence of an obstacle406a. For example, an average level of efficiency for picking or restocking items in a warehouse400by a particular vehicle102may be determined and stored by a computing system. Alternatively or additionally, a predetermined threshold of efficiency may be determined for vehicles102(e.g., of a vehicle type, for designated paths402, in a particular warehouse configuration400, etc.). A vehicle's level of efficiency on a particular day may be compared to the average level or the predetermined threshold to determine whether the particular level is less efficient. Accordingly, the event signal may include the efficiency comparison and/or the particular level of efficiency.

In some instances, the event signals may indicate increased congestion in vehicles and/or human workers. Increased congestion may indicate that vehicles are backed up due to the presence of an obstacle406a. For example, an average congestion level for a particular path402or aisle404amay be determined and stored by a computing system. Alternatively or additionally, a predetermined threshold of congestion level may be determined for the particular path402or aisle404a. The congestion level of a particular time or day may be compared to the average congestion level or the predetermined threshold. In some embodiments, congestion level may be determined by a comparison of the positions of two or more vehicles102in the path402or aisle404a. In some embodiments, congestion level may be determined based on visual information from images or video captured by one or more cameras and/or depth information from one or more depth sensors of vehicles102. For example, a first vehicle may capture visual or depth information to a second vehicle (or human) in the same path402or aisle404a. The visual or depth information may be processed to determine a distance between the first and second vehicles.

In some instances, the event signals may indicate a deviation of a vehicle102from the path402. The deviation may indicate that the presence of an obstacle in the path of the vehicle. For example, the vehicle102may be on a prescribed path402and may deviate from the path402to go around an obstacle406avia deviated path408. Alternatively, a deviation may include the vehicle102turning around to take a different path410.

In some embodiments, the event signals may indicate human intervention in the vehicle operation. For example, a human may stop, decrease the speed, or change the route of the vehicle102upon seeing an obstacle by interacting with the user interface106.

In some embodiments, one event signal type may be a trigger for at least one other event signal. For example, a first event signal may indicate that the vehicle102is slowing down in a particular path402. The transmission of the first event signal may trigger a process in the computing system206of the vehicle102to capture sensor data (e.g., by a camera sensor, by a LiDAR sensor, by a depth sensor, etc.) of the vehicle's surroundings. For example, a first event signal may indicate increased congestion, which can trigger the recording of sensor data for a certain duration (e.g., 0.5, 1, 3, 5, 10 minutes, or more) or distance travelled (e.g., 20 feet, 50 feet, 100 feet, or more) by the vehicle102. This sensor data may be transmitted (e.g., by communication device218) as a second event signal to the computing system (e.g., computing system202).

In step304, the computing system can process one or more received event signals to determine whether there is an obstacle in the path (e.g., at or near a particular location in the path). For example, the processor (e.g., processor212and/or216) may process an event signal including visual information using image processing techniques to determine the presence of an obstacle. In another example, the processor may process an event signal indicating the decreased speed of a vehicle102to determine whether the vehicle102is slowing or stopping for an obstacle.

In some cases, two or more instances of the same event signal type can be indicative of an obstacle. For example, an obstacle may be determined if an autonomous vehicle slows down each time along the same portion of the path. Each time, the vehicle102may transmit an event signal indicating the slower speed. In some embodiments, the processor may compare the event signals to at least one other event signal to categorize or group the event signal type for a particular path402and/or aisle404a. If there are a number of the same or similar event signals above a threshold, then the processor may determine the presence of an obstacle. For example, referring toFIG.5, if vehicle102travels path402four times in a given duration and, each time, deviates from the path402(e.g., via path408or path410) upon approaching obstacle406a, then four event signals indicating the deviated path are transmitted to the computing system. A processor of the computing system may compare the received four event signals to a threshold (e.g., three, four, five event signals, etc.) to determine whether an obstacle is present.

In other cases, two or more autonomous vehicles102reporting the same or similar event can indicate an obstacle. For example, an obstacle may be determined if multiple autonomous vehicles slow down at the same portion of the path402. In each scenario, the speed of the autonomous vehicles can be received and/or monitored by the computing system and compared to historical speeds for the particular vehicles and/or for the particular location on the path. For example, paths within the warehouse may be assigned “speed limits” or ranges of speed to which vehicles are configured to adhere. If the vehicle is determined to be traveling below the limit or range, the computing system may infer that an obstacle exists in the path.

In some embodiments, the receiving of the event signals can trigger the collecting and/or recording of data by one or more vehicles on the particular path. In step306, an autonomous vehicle can employ one or more on-board sensors (e.g., camera(s), LiDAR, etc.) to capture data of an obstacle. For example, the vehicle camera(s) can collect one or more images along the particular portion of the path. In another example, a depth sensor of the vehicle can collect data along the particular portion of the path. The sensor data can be processed to determine characteristics of the obstacle (e.g., its size, shape, weight, type, material, etc.) (workflow process504). For example, a processor may employ various techniques (e.g., machine learning techniques, subtractive geometry techniques, etc.) to determine characteristics of the obstacle.

In step308, based on the captured characteristics of the obstacle, the computing system can determine one or more tasks to mitigate or clear the obstacle from the path. For example, the system may determine whether the obstacle can be safely cleared by a human worker (workflow process506). In some embodiments, to determine whether the obstacle can be safely handled by a worker104, the computing system can compare an image of the obstacle to images of known obstacles (e.g., cart, basket, product, etc.). Each of these known obstacles can be labelled as safe or unsafe for normal handling. In some embodiments, the system can determine whether the obstacle is safe for handling based on the obstacle size determined from the image or video.

If the obstacle can be cleared (e.g., the obstacle is small enough, light enough, non-toxic, etc.) by a human worker in the vicinity of the obstacle (e.g., a picker, a stocker, etc.), the system may determine whether a container is required for transporting and/or clearing the obstacle (workflow process508). In step310, the computing system may send a signal with instructions to a user interface106(e.g., of an autonomous vehicle or other networked system) (workflow process510). If a container108is needed for clearing the obstacle, the computing system may transmit a signal to a controller of an autonomous vehicle in the vicinity of the obstacle to navigate to the obstacle location (workflow process512). In some cases, the user interface106may be of the same autonomous vehicle that transmitted event signals on the path and/or collected data of the obstacle characteristics. In some instances, the task may also be sent to one or more additional user interfaces to involve at least one additional human worker in clearing the obstacle. The user interface106may be configured to receive one or more of an input from the worker indicating that the obstacle was successfully cleared, an input indicating that the obstacle was not cleared, an input requesting special assistance with the obstacle, etc.

In some cases, the obstacle may not be able to be easily cleared due to its size, shape, weight, type, toxicity, etc. If, for example, the obstacle cannot be cleared by a nearby human worker, the task(s) may be transmitted to a user interface of another autonomous vehicle and/or the user interface of a remote computing system (e.g., system202) (workflow process514). In some cases, the task may be transmitted to a controller of another autonomous vehicle for navigating to the obstacle location (workflow process516). In some embodiments, the task may be transmitted to an autonomous vehicle that is within a certain travel distance from the obstacle, a vehicle that is navigating in a direction toward the obstacle, a vehicle with sufficient space aboard for the obstacle (e.g., based on the obstacle characteristics) and/or a vehicle with lower priority tasks. In some embodiments, if a vehicle102meeting the criteria is identified and the vehicle is in the process of picking items for a customer order, the computing system can send the vehicle the clean-up task, remove the order corresponding to the empty container on the vehicle, and integrate that order back into the main pool of orders for inducting onto future available vehicles for customer picking. In some embodiments, the processor can conduct a cost analysis to determine whether a vehicle102should prioritize an obstacle-clearing task over a customer order. For instance, because some obstacles can inefficiency for or completely block multiple vehicles (and therefore each of their customer orders), such obstacles can be prioritized over customer orders. Other obstacle types may be lower priority depending on the cost analysis.

For example, the other autonomous vehicle may be a specialized or designated vehicle for clearing obstacles, a larger vehicle, a vehicle with a larger container, a vehicle with a particular container (e.g., for liquids, etc.). In some embodiments, the obstacle may be too large or shaped such that a full shelf or level (e.g., platform150or154) of a vehicle102can be used to transport the obstacle. In another example, the task may be transmitted to a user interface that is accessed by workers other than pickers or stockers, e.g., workers that are assigned clean-up or maintenance activities. For example, these workers may specialize in clearing obstacles and/or handling other maintenance activities. These workers may accompany other types of autonomous vehicles or robots configured to clear obstacles.

In some embodiments, the obstacle-clearing task can be integrated into a regular work task list for an autonomous vehicle (workflow process518). For example, the location of the obstacle can be included as a stop for the vehicle among its several stops to pick items for a customer order and/or restock items within the warehouse. The obstacle's location can be used to determine an optimal path for the vehicle102, similar to determining an optimal path in collecting items for a customer order. The relative priority of the obstacle clean-up task as compared with customer orders or other work tasks in the system can be configured based on the size of the obstacle and/or the level of poor performance of the vehicles near the obstacle as initially measured. In some embodiments, the computing system can generate a specific task for clearing the obstacle. Such an obstacle-clearing task may include information including at least of the type of worker required, the type of vehicle required, the size or shape of container required, the type of the obstacle, one or more characteristics of the obstacles, images of the obstacle, etc.

Note that, in some instances, the vehicles102may be navigated away from a specified area around the obstacle for a particular time window to enable safe removal and/or to increase efficiency in other tasks. In some embodiments, the vehicle102itself can be configured to clear the obstacle, e.g., by a robotic arm or by pushing the obstacle with the front portion170or back portion174of the vehicle102.

In some examples, some or all of the processing described above can be carried out on a personal computing device, on one or more centralized computing devices, or via cloud-based processing by one or more servers. In some examples, some types of processing occur on one device and other types of processing occur on another device. In some examples, some or all of the data described above can be stored on a personal computing device, in data storage hosted on one or more centralized computing devices, or via cloud-based storage. In some examples, some data are stored in one location and other data are stored in another location. In some examples, quantum computing can be used. In some examples, functional programming languages can be used. In some examples, electrical memory, such as flash-based memory, can be used.

FIG.6is a block diagram of an example computer system600that may be used in implementing the systems and methods described herein. General-purpose computers, network appliances, mobile devices, or other electronic systems may also include at least portions of the system600. The system600includes a processor610, a memory620, a storage device630, and an input/output device640. Each of the components610,620,630, and640may be interconnected, for example, using a system bus650. The processor610is capable of processing instructions for execution within the system600. In some implementations, the processor610is a single-threaded processor. In some implementations, the processor610is a multi-threaded processor. The processor610is capable of processing instructions stored in the memory620or on the storage device630.

The memory620stores information within the system600. In some implementations, the memory620is a non-transitory computer-readable medium. In some implementations, the memory620is a volatile memory unit. In some implementations, the memory620is a nonvolatile memory unit.

The storage device630is capable of providing mass storage for the system600. In some implementations, the storage device630is a non-transitory computer-readable medium. In various different implementations, the storage device630may include, for example, a hard disk device, an optical disk device, a solid-date drive, a flash drive, or some other large capacity storage device. For example, the storage device may store long-term data (e.g., database data, file system data, etc.). The input/output device640provides input/output operations for the system600. In some implementations, the input/output device640may include one or more of a network interface devices, e.g., an Ethernet card, a serial communication device, e.g., an RS-232 port, and/or a wireless interface device, e.g., an 802.11 card, a 3G wireless modem, or a 4G wireless modem. In some implementations, the input/output device may include driver devices configured to receive input data and send output data to other input/output devices, e.g., keyboard, printer and display devices660. In some examples, mobile computing devices, mobile communication devices, and other devices may be used.

In some implementations, at least a portion of the approaches described above may be realized by instructions that upon execution cause one or more processing devices to carry out the processes and functions described above. Such instructions may include, for example, interpreted instructions such as script instructions, or executable code, or other instructions stored in a non-transitory computer readable medium. The storage device630may be implemented in a distributed way over a network, such as a server farm or a set of widely distributed servers, or may be implemented in a single computing device.

The term “system” may encompass all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. A processing system may include special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). A processing system may include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

Terminology