SYSTEMS AND METHODS FOR MATERIAL FLOW AUTOMATION

A system and method are provided for a material flow automation process. In some embodiments, the system and/or method comprise: receiving a first input including a plurality of core material flow elements; receiving a second input including a variable parameter that includes a status of each of the core material flow elements; applying the parameter to the plurality of core material flow elements; determining a plurality of composable material flow logic patterns from the application of the variable parameter to the plurality of core material flow elements; and applying the composable material flow logic patterns for managing an automation of movement of a vehicle.

FIELD OF INTEREST

The present inventive concepts relate to the field of robotics and material flow planning that includes the use of autonomous mobile robots (AMRs) for material handling. In particular, the inventive concepts may be related to systems and methods that implement composable patterns of material flow logic for the automation of movement in a complex environment to maximize speed and quality of application development.

BACKGROUND

Within increasing numbers and types of environments autonomous vehicles may travel through areas and/or along pathways that are shared with other vehicles and/or pedestrians. Such other vehicles can include other autonomous vehicles, semi-autonomous vehicles, and/or manually operated vehicles. The autonomous vehicles can take a variety of forms and can be referred to using various terms, such as mobile robots, robotic vehicles, automated guided vehicles, and/or autonomous mobile robots (AMRs). In some cases, these vehicles can be configured for operation in an autonomous mode where they self-navigate or in a manual mode where a human directs the vehicle's navigation. Herein, vehicles that are configured for autonomous navigation are referred to as AMRs.

Multiple AMRs may have access to an environment and both the state of the environment and the state of an AMR are constantly changing. The environment can be within, for example, a warehouse or large storage space or facility and the AMRs can include, but are not limited to, pallet lifts, pallet trucks, and tuggers.

Industrial AMRs need to use industrial controllers, that is, programmable logic controllers (PLCs), to achieve a higher level of automation. In order to fully leverage PLCs in industrial automation, they need to be integrated with a fleet management software. When enabling the integration, the integration can be done directly and specifically, or more generally. To enable more industrial automation use cases, a generalized approach is required to abstract integration between industrial controllers and AMRs.

Conventional material flow planning for indoor operations treats each indoor facility, e.g., warehouse, etc., as having a unique space where the material flow such as the storage, packaging and movement of goods has distinct problems and requires a unique plan. However, in order to implement a unique or bespoke flow solution, considerable time and resources are required and expensive to implement. In addition, conventional bespoke material flow automation designs cannot be replicated, and therefore reduce efficiencies. After each solution is designed by application developers, a series of complex decision-based proprietary rules are created for the individual application. Accordingly, material flow automation solutions are formed on a case-by-case and non-repeatable basis.

SUMMARY

In accordance with various aspects of the inventive concepts, provided is a method for material flow automation process, comprising: receiving a first input including a plurality of core material flow elements; receiving a second input including a variable parameter that includes a status of each of the core material flow elements; applying the parameter to the plurality of core material flow elements; determining a plurality of composable material flow logic patterns from the application of the variable parameter to the plurality of core material flow elements; and applying the composable material flow logic patterns for managing an automation of movement of a vehicle.

In various embodiments, the core material flow elements include data regarding a pick, drop, location, and route of the vehicle.

In various embodiments, the vehicle is an autonomous mobile robot (AMR).

In various embodiments the key variable includes a status of whether the core material flow elements are known or unknown.

In various embodiments, the core material flow elements and the variable parameter are arranged as a pattern language for determining the composable material flow logic patterns, and the method further comprises modeling a material flow for repeatable patterns of movement by the vehicle according to the pattern language.

In various embodiments, the pattern language is based on at least one indoor flow pattern of a factory or warehouse.

In various embodiments, the pattern language includes a collection of workflow templates for material movement which are used for determining a material workflow based on one or more combinations of the core material flow elements.

In various embodiments, applying the composable material flow logic patterns includes dynamically selecting one of a plurality of possible routes when a route is unknown, the one of the possible routes including a combination of the plurality of core material flow elements.

In accordance with various aspects of the inventive concepts, provided is a computer readable medium having computer executable instructions for a material flow planning system that when executed by a processor performs the following steps comprising: receiving at first input of the material flow planning system including a plurality of core material flow elements; receiving a second input of the material flow planning system including a variable parameter that includes a status of each of the core material flow elements; applying the parameter to the plurality of core material flow elements; determining a plurality of composable material flow logic patterns from the application of the variable parameter to the plurality of core material flow elements; and applying the composable material flow logic patterns for managing an automation of movement of a vehicle.

In various embodiments, the core material flow elements include data regarding a pick, drop, location, and route of the vehicle.

In various embodiments, the vehicle is an autonomous mobile robot (AMR).

In various embodiments the key variable includes a status of whether the core material flow elements are known or unknown.

In various embodiments, the core material flow elements and the variable parameter are arranged as a pattern language for determining the composable material flow logic patterns, and the method further comprises modeling a material flow for repeatable patterns of movement by the vehicle according to the pattern language.

In various embodiments, the pattern language is based on at least one indoor flow pattern of a factory or warehouse.

In various embodiments, the pattern language includes a collection of workflow templates for material movement which are used for determining a material workflow based on one or more combinations of the core material flow elements.

In various embodiments, applying the composable material flow logic patterns includes dynamically selecting one of a plurality of possible routes when a route is unknown, the one of the possible routes including a combination of the plurality of core material flow elements.

In accordance with various aspects of the inventive concepts, provided is a pattern language for use in modeling a material flow, comprising: four core material flow elements, including pick data, drop data, location data, and route data of a material flow machine; and a variable parameter including a status of at least one of the four core material flow elements.

In various embodiments, the pattern language determines one or more composable material flow logic patterns, and a material flow for repeatable patterns of movement by a vehicle is determined according to the pattern language.

In various embodiments, the pattern language is based on at least one indoor flow pattern of a factory or warehouse.

In various embodiments, the pattern language includes a collection of workflow templates for material movement which are used for determining a material workflow based on one or more combinations of the core material flow elements.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Various aspects of the inventive concepts will be described more fully hereinafter with reference to the accompanying drawings, in which some exemplary embodiments are shown. The present inventive concept may, however, be embodied in many different forms and should not be construed as limited to the exemplary embodiments set forth herein.

It will be understood that, although the terms first, second, etc. are be used herein to describe various elements, these elements should not be limited by these terms. These terms are used to distinguish one element from another, but not to imply a required sequence of elements. For example, a first element can be termed a second element, and, similarly, a second element can be termed a first element, without departing from the scope of the present invention. As used herein, the term “and/or” includes any and all combinations of one or more of the associated listed items.

To the extent that functional features, operations, and/or steps are described herein, or otherwise understood to be included within various embodiments of the inventive concept, such functional features, operations, and/or steps can be embodied in functional blocks, units, modules, operations and/or methods. And to the extent that such functional blocks, units, modules, operations and/or methods include computer program code, such computer program code can be stored in a computer readable medium, e.g., such as non-transitory memory and media, that is executable by at least one computer processor.

In accordance with aspect of the inventive concepts, to enable a flexible system for implementing composable and repeatable patterns of material flow logic for a plurality of material flow automation solutions, a system and method are provided that leverage a pattern language comprising a combination of a set of core material flow elements, namely, pick, drop, location, and route and a key variable based on a known and unknown status on the elements. For example, details on destination locations and path plans may not be known in advance, for example, before a robot such as an AMR performs a pick or drop operation. Although an AMR may be trained to operate along a given route, multiple originating and/or destination locations may be available so an operator may desire for the AMR to dynamically determine a route. A plurality of repeatable patterns of a material flow can be established from the pattern language. The core material elements existing in known and unknown states allows a special-purpose computer to perform route planning and simulation, modeling, analytics, and so on and to accommodate a considerable number, e.g., thousands, of likely material flow scenarios from a user interface.

In order for the AMR to perform a pick or drop operation, in some embodiments, an AMR may interface with an industrial infrastructure to pick and drop pallets. In order for an AMR to accomplish this, its perception and manipulation systems in accordance with principles of inventive concepts may maintain a model for what a pallet is, as well as models for all the types of infrastructure for which it will place the pallet (e.g., tables, carts, racks, conveyors, etc.). These models are software components that are parameterized in a way to influence the algorithmic logic of the computation.

In example embodiments a route network may be constructed by an operator through training-by-demonstration, wherein an operator leads the AMR through a training route and inputs behaviors (for example, picks or places) along the route. A build procedure employs information gathered during training (for example, odometry, grid information including localization information, and operator input regarding behaviors) into a route network. An AMR may then employ the route network to autonomously follow during normal operation. The route network may be modeled, or viewed, as a graph of nodes and edges, with stations as nodes and trained segments as edges. Behaviors may be trained within segments. Behaviors may include “point behaviors” such as picks and drops or “zone behaviors” such as intersections. In example embodiments an AMR's repetition during normal operations of a trained route may be referred to as a “follow.” Anything, other than the follow itself, the AMR does during the follow may be viewed as a behavior. Zones such as intersections may include behaviors that are performed before, during, and/or after the zone. For intersections, the AMR requests access to the intersection from a supervisory system, also referred to herein as a supervisor or supervisory processor, (for example, Supervisor™ described elsewhere herein), e.g., shown inFIG.2, prior to reaching the area covered by the intersection zone. When the AMR exits the zone, it releases that access to the supervisory system.

Referring toFIGS.1and2, shown is an example of a self-driving or robotic vehicle in the form of an AMR lift truck100that is equipped and configured to drop off and pick up objects, such as palletized loads or other loads, in accordance with aspects of the inventive concepts. Although the robotic vehicle can take the form of an AMR lift truck100, the inventive concepts could be embodied in any of a variety of other types of robotic vehicles and AMRs, including, but not limited to, forklifts, tow tractors, tuggers, and the like.

In this embodiment, AMR100includes a payload area102configured to transport any of a variety of types of objects that can be lifted and carried by a pair of forks110. Such objects can include a pallet104loaded with goods106, collectively a “palletized load,” or a cage or other container with fork pockets, as examples. Outriggers108extend from the robotic vehicle100in the direction of forks110to stabilize the AMR, particularly when carrying palletized load104,106.

Forks110may be supported by one or more robotically controlled actuators coupled to a carriage that enable AMR100to raise and lower, side-shift, and extend and retract to pick up and drop off objects in the form of payloads, e.g., palletized loads104or other loads to be transported by the AMR. In various embodiments, the AMR may be configured to robotically control the yaw, pitch, and/or roll of forks110to pick a palletized load in view of the pose of the load and/or horizontal surface that supports the load. In various embodiments, the AMR may be configured to robotically control the yaw, pitch, and/or roll of forks110to pick a palletized load in view of the pose of the horizontal surface that is to receive the load.

The AMR100may include a plurality of sensors150that provide various forms of sensor data that enable the AMR to safely navigate throughout an environment, engage with objects to be transported, and avoid obstructions. In various embodiments, the sensor data from one or more of sensors150can be used for path navigation and obstruction detection and avoidance, including avoidance of detected objects, hazards, humans, other robotic vehicles, and/or congestion during navigation.

One or more of sensors150can form part of a two-dimensional (2D) or three-dimensional (3D) high-resolution imaging system used for navigation and/or object detection. In some embodiments, one or more of the sensors can be used to collect sensor data used to represent the environment and objects therein using point clouds to form a 3D evidence grid of the space, each point in the point cloud representing a probability of occupancy of a real-world object at that point in 3D space.

In computer vision and robotic vehicles, a typical task is to identify specific objects in a 3D model and to determine each object's position and orientation relative to a coordinate system. This information, which is a form of sensor data, can then be used, for example, to allow a robotic vehicle to manipulate an object or to avoid moving into the object. The combination of position and orientation is referred to as the “pose” of an object. The image data from which the pose of an object is determined can be either a single image, a stereo image pair, or an image sequence where, typically, the camera as a sensor150is moving with a known velocity as part of the robotic vehicle.

Sensors150can include one or more stereo cameras152and/or other volumetric sensors, sonar sensors, radars, and/or LiDAR scanners or sensors154a,154bpositioned about AMR100, as examples. Inventive concepts are not limited to particular types of sensors, nor the types, configurations, and placement of the AMR sensors inFIGS.1and2. In some embodiments, object movement techniques (i.e., dropping an object in the zone, removing an object from a zone) described herein are performed with respect to one or more of sensors150, in particular, a combination of object detection sensors and load presence sensors. The object detection sensor(s) is/(are) configured to locate a position of an object within the zone. An object detection sensor can be or include at least one camera, LiDAR, electromechanical, and so on. The load presence sensor(s) is/(are) configured to determine whether AMR100is carrying an object.

In the embodiment shown inFIG.1, at least one of LiDAR devices154a,bcan be a 2D or 3D LiDAR device for performing safety-rated forward obstruction sensing functions. In alternative embodiments, a different number of 2D or 3D LiDAR devices are positioned near the top of AMR100. Also, in this embodiment a LiDAR157is located at the top of the mast. In some embodiments LiDAR157is a 2D LiDAR used for localization or odometry-related operations.

The object detection and load presence sensors can be used in combination with others of the sensors, e.g., stereo camera head152. Examples of stereo cameras arranged to provide 3-dimensional vision systems for a vehicle, which may operate at any of a variety of wavelengths, are described, for example, in U.S. Pat. No. 7,446,766, entitled Multidimensional Evidence Grids and System and Methods for Applying Same and U.S. Pat. No. 8,427,472, entitled Multi-Dimensional Evidence Grids, which are hereby incorporated by reference in their entirety. LiDAR systems arranged to provide light curtains, and their operation in vehicular applications, are described, for example, in U.S. Pat. No. 8,169,596, entitled System and Method Using a Multi-Plane Curtain, which is hereby incorporated by reference in its entirety.

FIG.3is a block diagram of components of an embodiment of AMR100ofFIG.1, incorporating technology for moving and/or transporting objects (e.g., loads or pallets) to/from a predefined zone, in accordance with principles of inventive concepts. In the example embodiment shown inFIGS.1and2, AMR100is a warehouse robotic vehicle, which can interface and exchange information with one or more external systems, including a supervisor system, fleet management system, and/or warehouse management system (collectively “supervisor200”). In various embodiments, supervisor200could be configured to perform, for example, fleet management and monitoring for a plurality of vehicles (e.g., AMRs) and, optionally, other assets within the environment. Supervisor200can be local or remote to the environment, or some combination thereof.

In various embodiments, supervisor200can be configured to provide instructions and data to AMR100, and to monitor the navigation and activity of the AMR and, optionally, other AMRs. The AMR can include a communication module160configured to enable communications with supervisor200and/or any other external systems. Communication module160can include hardware, software, firmware, receivers, and transmitters that enable communication with supervisor200and any other external systems over any now known or hereafter developed communication technology, such as various types of wireless technology including, but not limited to, Wi-Fi, Bluetooth™, cellular, global positioning system (GPS), radio frequency (RF), and so on.

As an example, supervisor200could wirelessly communicate a path for AMR100to navigate for the vehicle to perform a task or series of tasks. The path can be a virtual line that the AMR is following during autonomous motion. The path can be relative to a map of the environment stored in memory and, optionally, updated from time-to-time, e.g., in real-time, from vehicle sensor data collected in real-time as AMR100navigates and/or performs its tasks. The sensor data can include sensor data from one or more sensors described with reference toFIG.1. As an example, in a warehouse setting the route could include a plurality of stops along a route for the picking and loading and/or the unloading of objects, e.g., payload of goods. The route can include a plurality of path segments, including a zone for the acquisition or deposition of objects. Supervisor200can also monitor AMR100, such as to determine the AMR's location within the environment, battery status and/or fuel level, and/or other operating, vehicle, performance, and/or load parameters.

As described above, when training an AMR100, a route may be developed. That is, an operator may guide AMR100through a travel path within the environment while the AMR, through a machine-learning process, learns and stores the route for use in task performance and builds and/or updates an electronic map of the environment as it navigates, with the route being defined relative to the electronic map. The route may be stored for future use and may be updated, for example, to include more, less, or various locations, or to otherwise revise the travel route and/or path segments, as examples.

As is shown inFIG.2, in example embodiments, AMR100includes various functional elements, e.g., components and/or modules, which can be housed within housing115. Such functional elements can include at least one processor10coupled to at least one memory12to cooperatively operate the vehicle and execute its functions or tasks. Memory12can include computer program instructions, e.g., in the form of a computer program product, executable by processor10. Memory12can also store various types of data and information. Such data and information can include route data, path data, path segment data, pick data, location data, environmental data, and/or sensor data, as examples, as well as the electronic map of the environment. In some embodiments, memory12stores relevant measurement data for use by a dynamic route determination module185. In some embodiments, the dynamic route determination module185is part of a controller, for example, industrial controller312described with respect toFIG.5. In some embodiments, the dynamic route determination module185includes a processor and memory for performing some or all of the material flow automation process20ofFIG.4.

In this embodiment, processor10and memory12are shown onboard AMR100ofFIG.1, but external (offboard) processors, memory, and/or computer program code could additionally or alternatively be provided. That is, in various embodiments, the processing and computer storage capabilities can be onboard, offboard, or some combination thereof. For example, some processor and/or memory functions could be distributed across the supervisor200, other vehicles, and/or other systems external to the robotic vehicle100.

The functional elements of AMR100can further include a navigation module170configured to access environmental data, such as the electronic map, and path information stored in memory12, as examples. Navigation module170can communicate instructions to a drive control subsystem120to cause AMR100to navigate its route by navigating a path within the environment. During vehicle travel, navigation module170may receive information from one or more sensors150, via a sensor interface (I/F)140, to control and adjust the navigation of the AMR. For example, sensors150may provide 2D and/or 3D sensor data to navigation module170and/or drive control subsystem120in response to sensed objects and/or conditions in the environment to control and/or alter the AMR's navigation. As examples, sensors150can be configured to collect sensor data related to objects, obstructions, equipment, goods to be picked, hazards, completion of a task, and/or presence of humans and/or other robotic vehicles. An object can be a pickable or non-pickable object within a zone used by the vehicle, such as a palletized load, a cage with slots for forks at the bottom, a container with slots for forks located near the bottom and at the center of gravity for the load. Other objects can include physical obstructions in a zone such as a traffic cone or pylon, a person, and so on.

A safety module130can also make use of sensor data from one or more of sensors150, in particular, LiDAR scanners154, to interrupt and/or take over control of drive control subsystem120in accordance with applicable safety standard and practices, such as those recommended or dictated by the United States Occupational Safety and Health Administration (OSHA) for certain safety ratings. For example, if safety sensors detect objects in the path as a safety hazard, such sensor data can be used to cause the drive control subsystem120to stop the vehicle to avoid the hazard.

As shown inFIGS.1and2, in various embodiments, the system can comprise a mobile robotics platform, such as an AMR, at least one sensor150configured to collect/acquire point cloud data, such as a LiDAR scanner or 3D camera; and at least one local processor10configured to process, interpret, and register the sensor data relative to a common coordinate frame. For example, scans from the sensor150, e.g., LiDAR scanner or 3D camera, are translated and rotated in all six degrees of freedom to align to one another and create a contiguous point cloud. To do this, a transform is applied to the data. The sensor data collected by sensors150can represent objects using the point clouds, where points in a point cloud represent discrete samples of the positions of the objects in 3-dimensional space. AMR100may respond in various ways depending upon whether a point cloud based on the sensor data includes one or more points impinging upon, falling within an envelope of, or coincident with the 3-dimensional path projection (or tunnel) of AMR100.

FIG.3illustrates an example of a warehouse environment in which embodiments of the present inventive concepts can be practiced. In example embodiments, a material flow system in accordance with principles of the inventive concepts may be implemented in a facility such as a manufacturing, processing, or warehouse facility, for example. For brevity and clarity of description the example embodiments described herein will generally be in reference to warehouse implementations, but inventive concepts are not limited thereto.

In the example embodiment ofFIG.3, items (not shown) can be stored in storage racks302distributed throughout a warehouse. Storage racks302may be divided into bays304and bays304may be further divided into shelves (not shown). Racks302may be configured to store items within bins, on any of a variety of pallets, or other materials handling storage units. The racks302may be single- or multi-level, for example, and may vary in width, length, and height. Staging areas (not shown) may be used to temporarily store items for shipping or receiving, respectively, to/from transportation means, such as truck or train for example, to external facilities. Rows306and aisles308provide access to storage racks302.

As shown, a plurality of vehicles such as AMRs100A-100D (generally,100) can be in communication with a fleet management system (FMS) and/or warehouse management system (WMS)302, in accordance with aspects of inventive concepts. One or more user interfaces, for example, user interface320shown inFIG.5, may be distributed throughout the warehouse. The user interfaces may be employed by an operator to interact with a system such as one described in the discussion related toFIG.2to direct a vehicle to pick an item from one location (a specific storage rack, for example) and to place it in another location (a staging area, for example). The user interfaces may be included within AMRs, may be in standalone screens or kiosks positioned throughout the warehouse, may be handheld electronic devices, or may be implemented as applications on smartphones or tablets, for example. One or more humans (not shown) may also work within the environment and communicate with the WMS301, for example, via a user interface. The humans and the AMRs100can also communicate directly, in some embodiments. In some embodiments, the humans can order pickers that load goods on AMRs at pick locations within the warehouse environment. The humans may employ handheld electronic devices through which they can communicate with the WMS and/or the AMRs.

The AMRs100can operate according to route, destination, and robotic actions determined by embodiments of the systems and methods herein. For example, an AMR100may travel along a first predetermined route, for example, according to the process described inFIG.4, and in doing so can use its cameras, sensors, processors, and autonomous technology, e.g., shown inFIGS.1and2, to collect information that can be used for a subsequent pick or drop, which may be unknown while a location of the subsequent pick or drop is known. A material flow planning system may be implemented in the WMS/FMS301or implemented as part of an automation system in communication with the WMS/FMS301, for example, implemented at supervisor200shown inFIG.2, to collect information from the AMR during the first predetermined route to produce a pattern language that may be used for modeling the material flow based on the information gathered in connection with the first predetermined route. The pattern language can be used to establish repeatable patterns of movement, i.e., a second and subsequent predetermined routes. The FMS and/or WMS, either one or both of which may be implemented on supervisory processor200, can wirelessly communicate with all of the AMRs100and monitor their status, assign a next task, and/or instruct navigation or a non-work location. Accordingly, a system controlling the AMRs100, for example, some or all of which may be implemented in a combination of the WMS/FMS301and AMRs100, may operate according to a pattern language generated for modeling a material flow to accommodate the varying system requirements. The pattern language may be used for modeling the material flow to increase speed, allow for replicability, and reduce cost in delivering the material flow automation solution regardless of the unique indoor environment.

FIG.4is a flow diagram of a material flow automation process20, in accordance with aspects of inventive concepts. An AMR100shown inFIGS.1-3may be programmed to travel along a predetermined route established by the process20and to perform operations of a material flow, for example, an indoor material flow. One example of an operation is where the AMR100places, or drops, objects on a pallet. Another example is where the AMR100picks an object from a pallet. The process20may include material flow230, path plan220, and information gathering210stages and in doing is constructed for generating a pattern language for establishing repeatable patterns of material flow. In some embodiments, one or more sensors150, in particular, navigation cameras and pallet detection system sensors, are used for at least the information gathering stage210.

As used herein, a pattern language describes a collection of templates of workflows for material movement. By creating a centralized collection of these templates, different material flow processes can be identified and executed using a predefined template rather than having to explain the detailed material flow steps each time. The templates may represent simplified real-world scenarios resulting from combining core elements, e.g., pick, drop, location, route in different combinations. The flow elements needed for a particular pattern or template can be derived directly from how the material is physically moved around in the facility. If a customer requires an AMR100to pick up an object at one location and drop it off at a different location, the details of this movement can be represented as material flow elements in the template.

In some embodiments, a pattern language is used to model a material flow in a simple manner so that an operator may ensure that his entries have been properly recorded by the system and that, as a result, his material flow jobs will be carried out as he envisions. The present inventive concept can refer to a given customer site as being an “X type of material flow site”. If the material flow in the site is novel and a process flow template, for example, used for robotic process automation or the like, has not yet been generated, then a new template can be created. A pattern language here can be a system for evaluating material flows and deriving common characteristics that are shared with other flows.

The process20can begin by the AMR100collecting data about a travel route from a current location to a new location. Here, the AMR100may not be preprogrammed and is configured to be expected to determine a route to the new location, for example, executing the dynamic route determination module185. The decision diamonds201,205,209indicative of a known and unknown status can be applied to the core elements of the material flow, e.g., pick, drop, location, and travel route201-209of the path plan220and material flow230stages, respectively, and in doing so may allow the process20to identify one or more repeatable patterns of movement. As described herein, a repeatable pattern of movement may be identified based on the material flow elements, e.g., pick, drop, location, route by modeling the status of all four elements, for example, according to a parameter of a known and unknown status of the elements. The process20can distinguish known states from unknown states. For example, the process20recognizes when there is uncertainty as to where the material flow occurs, and also recognizes when a certainty about a path or destination is known upfront, prior to a motion of the AMR100. It is well known that routes can be preprogrammed, for example, in cases where they are static and predefined. Here, if a robot route is static it is said to be known ahead of time. Here, the robot moves to the first location to pick up a pallet, the travels to a second location to drop off the pallet in a same manner.

However, as described above in other cases a route cannot be preprogrammed because the operator may require an AMR100to dynamically determine the route to an intended destination. Accordingly, the process20relies on a combination of known variables in advance as well as unknown variables which are determined as part of the process20. In contrast to the static route mentioned in the previous example, if the AMR100route is configured to operation in a dynamic manner, the AMR100may pick a pallet from one of five different locations and drop it off at another location of the five locations. The exact route is not known in advance because there is multiple (i.e.,25) possible combinations of pick up and drop off locations and corresponding routes to be dynamically determined or selected by the operator. A pattern language may describe a collection of templates that are stored in a data repository so that different material flow scenarios can be determined using a predefined template, which represents a scenario resulting from the various combinations. For example, location203may be different from location207and not the same as required in a programmed AMR for the same static location. The particular combination that is selected may depend on the state of the material that needs to be moved in the facility, or other factors.

If the new location, e.g., location207, is unknown, then the collected data can be processed to determine a travel route to the new location. At the new location, the AMR may perform a pick operation. The information about the pick can be collected, for example, by cameras and/or other sensors of the AMR100shown inFIG.1, and provided to a system illustrated inFIG.5, where it can be processed for planning a travel route. Modeling can be performed by a person evaluating a new site and is qualitative in nature. In doing so, the operator may inquire as to what kind of each of the core elements is happening and based on which and whether they are known or not, they match that to an existing pattern.

Thus, if an operator incorporates known and unknown information about the core elements for modeling, e.g., so that the core elements are matched to an existing pattern, the AMR knows how to get to every location that has been trained in the system. Thus, if an operator selects a location to send the AMR to, the robot can compute what paths to take to arrive there based on the trained path network it has in its memory. Although a location may not be unknown from the AMR's perspective with respect to being trained to arrive at the location. The location here is not known in advance with respect to the operator directing the AMR to the location for a given route in advance.

As described above, a pattern language comprising the core elements and key variables regarding the known and unknown status may be used to establish a plurality of repeatable patterns, for example, shown inFIG.4by the core elements of material flow, e.g., pick, drop, location, route, and a known/unknown status parameter on the core elements. As shown inFIG.5, a software tool implanting and executing these features may be displayed on a user interface320allowing a user to use the collected information regarding the elements and patterns for the rapid articulation of a material flow that controls an automation system. These features also provided for an informed backend computing development and material flow modeling environment.

Referring again toFIG.5, illustrated is a block diagram of a system for implementing patterns of material flow logic, in accordance with some embodiments. The system includes a material flow planning system310, a user interface320, and at least one AMR330or other mechanism that includes a computer processor or the like for executing instructions of the process ofFIG.3. The material flow planning system310may include an industrial controller312that communicates with the AMR330via an application programming interface (API) or the like to send instructions to the AMR330in response to the method ofFIG.4. For example, the controller312can send a signal that a path or location is determined according to a repeatable pattern of a material flow determined by a pattern language based on a combination of flow patterns determined from stored historical data and fundamental flow patterns based on public traffic systems or the like. The material flow planning system310may include a first input for receiving a plurality of core material flow elements, for example, from the WMS/FMS, AMS, and/or computer server and second input for receiving a parameter that includes a status of each of the core material flow elements, for example, from the WMS/FMS, AMS, and/or computer server. The first and second inputs can receive commands, information, etc. from the user interface320. The computer server may execute some or all of the process steps ofFIG.4and may reside in a cloud computing environment or in the indoor operation.

Repeatable patterns of movement can be identified by combining the core elements of material flow, e.g., pick, drop, location, route) and a known/unknown status parameter on the core elements, for example, a status indicating that there is uncertainty regarding a path plan or destination where a pick or drop operation is desired. The material flow planning system310can use this data to increase the speed of the AMR330and allow replicability of the movement of the AMR330and/or other apparatuses in the material flow.

The foregoing can be illustrated by way of the following example. A database table may be generated and stored that contains all the composable material flow logic patterns for a material flow automation environment, for example, shown inFIG.3. The table may be arranged to populate each row with a scenario name, each corresponding to a material flow scenario resulting from combining the core elements in different combinations. For each scenario name row, a plurality of columns may include relevant data. For example, a column for a scenario may include a description where the operator species the drop-off destination at a pickup area for an AMR and a column that include a path sequence, for example, a single pickup and a single drop-off, or multiple pickups and a single drop-off, and so on. The table may be presented from a graphical user interface (GUI) or other display that allows an operator to select various options, for example, whether the AMR has full route knowledge or is expected to dynamically determine a destination path. Another user-selectable field may include the vehicle type, i.e., a lift, tug, pallet jack, and so on. Another user-selectable field may include a material exchange method, i.e., fully autonomous, semi-autonomous, manual, and so on. These selections can be used to create new templates or modify existing templates, which in turn can be used for providing a repeatable pattern of a material flow determined by historical data and flow pattern data, even in cases where an AMR has not been to a particular location, and does not know in advance of a route to a location where an operator plans to send the AMR.

While the foregoing has described what are considered to be the best mode and/or other preferred embodiments, it is understood that various modifications can be made therein and that aspects of the inventive concepts herein may be implemented in various forms and embodiments, and that they may be applied in numerous applications, only some of which have been described herein. It is intended by the following claims to claim that which is literally described and all equivalents thereto, including all modifications and variations that fall within the scope of each claim.

For example, it will be appreciated that all of the features set out in any of the claims (whether independent or dependent) can be combined in any given way.

Below follows an itemized list of statements describing embodiments in accordance with the inventive concepts:

1. A method for material flow automation process, comprising:receiving a first input including a plurality of core material flow elements;receiving a second input including a variable parameter that includes a status of each of the core material flow elements;applying the parameter to the plurality of core material flow elements;determining a plurality of composable material flow logic patterns from the application of the variable parameter to the plurality of core material flow elements; andapplying the composable material flow logic patterns for managing an automation of movement of a vehicle.

2. The method of statement1, or any other statement or combinations of statements, wherein the core material flow elements include data regarding a pick, drop, location, and route of the vehicle.

3 The method of statement1, or any other statement or combinations of statements, wherein the vehicle is an autonomous mobile robot (AMR).

4. The method of statement1, or any other statement or combinations of statements, wherein the key variable includes a status of whether the core material flow elements are known or unknown.

5. The method statement1, or any other statement or combinations of statements, wherein the core material flow elements and the variable parameter are arranged as a pattern language for determining the composable material flow logic patterns, and wherein the method further comprises modeling a material flow for repeatable patterns of movement by the vehicle according to the pattern language.

6. The method of statement5, or any other statement or combinations of statements, wherein the pattern language is based on at least one indoor flow pattern of a factory or warehouse.

7. The method of statement5, or any other statement or combinations of statements, wherein the pattern language includes a collection of workflow templates for material movement which are used for determining a material workflow based on one or more combinations of the core material flow elements.

8. The method of statement1, or any other statement or combinations of statements, wherein applying the composable material flow logic patterns includes dynamically selecting one of a plurality of possible routes when a route is unknown, the one of the possible routes including a combination of the plurality of core material flow elements.

9. A computer readable medium having computer executable instructions for a material flow planning system that when executed by a processor performs the following steps comprising:receiving at first input of the material flow planning system including a plurality of core material flow elements;receiving a second input of the material flow planning system including a variable parameter that includes a status of each of the core material flow elements;applying the parameter to the plurality of core material flow elements;determining a plurality of composable material flow logic patterns from the application of the variable parameter to the plurality of core material flow elements; andapplying the composable material flow logic patterns for managing an automation of movement of a vehicle.

10. The computer readable medium of statement9, or any other statement or combinations of statements, wherein the core material flow elements include data regarding a pick, drop, location, and route of the vehicle.

11. The computer readable medium of statement9, or any other statement or combinations of statements, wherein the vehicle is an autonomous mobile robot (AMR).

12. The computer readable medium of statement9, or any other statement or combinations of statements, wherein the key variable includes a status of whether the core material flow elements are known or unknown.

13. The computer readable medium of statement9, or any other statement or combinations of statements, wherein the core material flow elements and the variable parameter are arranged as a pattern language for determining the composable material flow logic patterns, and wherein the method further comprises modeling a material flow for repeatable patterns of movement by the vehicle according to the pattern language.

14. The computer readable medium of statement13, or any other statement or combinations of statements, wherein the pattern language is based on at least one indoor flow pattern of a factory or warehouse.

15. The computer readable medium of statement13, or any other statement or combinations of statements, wherein the pattern language includes a collection of workflow templates for material movement which are used for determining a material workflow based on one or more combinations of the core material flow elements.

16. The computer readable medium of statement9, or any other statement or combinations of statements, wherein applying the composable material flow logic patterns includes dynamically selecting one of a plurality of possible routes when a route is unknown, the one of the possible routes including a combination of the plurality of core material flow elements.

17. A computer program product executable by at least one processor to model a material flow using a pattern language, comprising:four core material flow elements, including pick data, drop data, location data, and route data of a material flow machine; anda variable parameter including a status of at least one of the four core material flow elements.

18. The computer program product of statement17, or any other statement or combinations of statements, wherein the pattern language determines one or more composable material flow logic patterns, and a material flow for repeatable patterns of movement by a vehicle is determined according to the pattern language.

19. The computer program product of statement17, or any other statement or combinations of statements, wherein the pattern language is based on at least one indoor flow pattern of a factory or warehouse.

20. The computer program product of statement17, or any other statement or combinations of statements, wherein the pattern language includes a collection of workflow templates for material movement which are used for determining a material workflow based on one or more combinations of the core material flow elements.