Robotic system with collision avoidance mechanism and method of operation thereof

A robotic system includes a user interface configured to receive a jog command for manually operating a robotic unit; a control unit, coupled to the user interface, configured to: real-time parallel process the jog command including to: execute a collision check thread to determine whether the jog command results in a collision or results in an unobstructed status for the robotic unit within an operation environment based on an environment model and a robot model, execute a jog operation thread to determine whether the unobstructed status is provided within a collision check time limit; and execute the jog command by the robotic unit based on the unobstructed status provided before the collision check time limit.

TECHNICAL FIELD

The present technology is directed generally to robotic systems and, more specifically, to collision avoidance mechanism.

BACKGROUND

Modern robotics and automation are providing increasing levels of functionality to support in industrial settings, such as manufacturing facilities, receiving and distribution centers, and warehouses. Research and development in the existing technologies can take a myriad of different directions.

As users become more empowered with the growth of robotic systems, new and old paradigms begin to take advantage of this new technology space. There are many technological solutions to take advantage of these new capabilities to enhance or augment automation of robotic systems, such as the capability for the robotic systems to avoid collisions. However, manual operations are still required with the additional functionalities of robotic systems and avoidance of collisions are still required for manual operations.

Thus, a need still remains for a robotics system with a collision avoidance mechanism for manual operations. In view of the ever-increasing commercial competitive pressures, along with growing consumer expectations and the diminishing opportunities for meaningful product differentiation in the marketplace, it is increasingly critical that answers be found to these problems. Additionally, the need to reduce costs, improve efficiencies and performance, and meet competitive pressures adds an even greater urgency to the critical necessity for finding answers to these problems.

SUMMARY

An embodiment of the present invention provides a robotic system, including: a user interface configured to receive a jog command for manually operating a robotic unit; a control unit, coupled to the user interface, configured to: real-time parallel process the jog command including to: execute a collision check thread to determine whether the jog command results in a collision or results in an unobstructed status for the robotic unit within an operation environment based on an environment model and a robot model, execute a jog operation thread to determine whether the unobstructed status is provided within a collision check time limit; and execute the jog command by the robotic unit based on the unobstructed status provided before the collision check time limit.

An embodiment of the present invention provides a method of operation of a robotic system including: receiving a jog command for manually operating a robotic unit; real-time parallel processing the jog command including: executing a collision check thread to determine whether the jog command results in a collision or results in an unobstructed status for the robotic unit within an operation environment based on an environment model and a robot model, executing a jog operation thread to determine whether the unobstructed status is provided within a collision check time limit; executing the jog command by the robotic unit based on the unobstructed status provided before the collision check time limit.

An embodiment of the present invention provides a non-transitory computer readable medium including instructions for a robotic system, including: receiving a jog command for manually operating a robotic unit; real-time parallel processing the jog command including: executing a collision check thread to determine whether the jog command results in a collision or results in an unobstructed status for the robotic unit within an operation environment based on an environment model and a robot model, executing a jog operation thread to determine whether the unobstructed status is provided within a collision check time limit; executing the jog command by the robotic unit based on the unobstructed status provided before the collision check time limit.

DETAILED DESCRIPTION

In the following description, numerous specific details are set forth to provide a thorough understanding of the presently disclosed technology. In other embodiments, the techniques introduced here can be practiced without these specific details. In other instances, well-known features, such as specific functions or routines, are not described in detail in order to avoid unnecessarily obscuring the present disclosure. References in this description to “an embodiment,” “one embodiment,” or the like mean that a particular feature, structure, material, or characteristic being described is included in at least one embodiment of the present disclosure. The appearances of such phrases in this specification do not necessarily all refer to the same embodiment. On the other hand, such references are not necessarily mutually exclusive. Furthermore, the particular features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments.

It is to be understood that the various embodiments shown in the figures are merely illustrative representations. Further, the drawings showing embodiments of the system are semi-diagrammatic, and not to scale and, particularly, some of the dimensions are for the clarity of presentation and are shown exaggerated in the drawing figures. Similarly, although the views in the drawings for ease of description generally show similar orientations, this depiction in the figures is arbitrary for the most part. Generally, the invention can be operated in any orientation.

Several details describing structures or processes that are well known and often associated with robotic systems and subsystems, but that can unnecessarily obscure some significant aspects of the disclosed techniques, are not set forth in the following description for purposes of clarity. Moreover, although the following disclosure sets forth several embodiments of different aspects of the present technology, several other embodiments can have different configurations or different components than those described in this section. Accordingly, the disclosed techniques can have other embodiments with additional elements or without several of the elements described below.

Many embodiments or aspects of the present disclosure described below can take the form of computer-executable or controller-executable instructions, including routines executed by a programmable computer or controller. Those skilled in the relevant art will appreciate that the disclosed techniques can be practiced on computer or controller systems other than those shown and described below. The techniques described herein can be embodied in a special-purpose computer or data processor that is specifically programmed, configured, or constructed to execute one or more of the computer-executable instructions described below. Accordingly, the terms “computer” and “controller” as generally used herein refer to any data processor and can include Internet appliances and handheld devices, including palm-top computers, wearable computers, cellular or mobile phones, multi-processor systems, processor-based or programmable consumer electronics, network computers, mini computers, and the like. Information handled by these computers and controllers can be presented at any suitable display medium, including a liquid crystal display (LCD). Instructions for executing computer- or controller-executable tasks can be stored in or on any suitable computer-readable medium, including hardware, firmware, or a combination of hardware and firmware. Instructions can be contained in any suitable memory device, including, for example, a flash drive, USB device, and/or other suitable medium.

The terms “coupled” and “connected,” along with their derivatives, can be used herein to describe structural relationships between components. It should be understood that these terms are not intended as synonyms for each other. Rather, in particular embodiments, “connected” can be used to indicate that two or more elements are in direct contact with each other. Unless otherwise made apparent in the context, the term “coupled” can be used to indicate that two or more elements are in either direct or indirect (with other intervening elements between them) contact with each other, or that the two or more elements cooperate or interact with each other (e.g., as in a cause-and-effect relationship, such as for signal transmission/reception or for function calls), or both.

The term “module” or “unit” referred to herein can include software, hardware, mechanical mechanisms, or a combination thereof in an embodiment of the present invention, in accordance with the context in which the term is used. For example, the software can be machine code, firmware, embedded code, or application software. Also, for example, the hardware can be circuitry, a processor, a special purpose computer, an integrated circuit, integrated circuit cores, a pressure sensor, an inertial sensor, a microelectromechanical system (MEMS), a passive device, or a combination thereof. Furthermore, the mechanical mechanism can include actuators, motors, arms, joints, handles, end effectors, guides, mirrors, anchoring bases, vacuum lines, vacuum generators, liquid source lines, or stoppers. Further, if a “module” or “unit” is written in the system claims section below, the “module” or “unit” is deemed to include hardware circuitry for the purposes and the scope of the system claims.

The modules or units in the following description of the embodiments can be coupled or attached to one another as described or as shown. The coupling or attachment can be direct or indirect, without or with intervening items between coupled or attached modules or units. The coupling or attachment can be by physical contact or by communication between modules or units.

It is also understood that the nouns or elements in the embodiments can be described as a singular instance. It is understood that the usage of singular is not limited to singular but the singular usage can be applicable to multiple instance for any particular noun or element in the application. The numerous instances can be the same or similar or can be different.

Referring now toFIG.1, therein is shown an example environment for a robotic system100with a collision avoidance mechanism in an embodiment. The environment for the robotic system100can includes one or more structures, such as robots or robotic devices, configured to execute one or more tasks. Aspects of the object collision avoidance mechanism can be practiced or implemented by the various structures.

In the example illustrated inFIG.1, the robotic system100can include an unloading unit102, a transfer unit104, a transport unit106, a loading unit108, a robotic unit110, a controller112, or a combination thereof in a warehouse, a distribution center, or a shipping hub. The robotic system100or a portion of the robotic system100can be configured to execute one or more tasks.

The tasks can be combined in sequence to perform an operation that achieves a goal, for example, such as to unload a target object120from a vehicle, such as a truck, trailer, a van, or train car, for storage in a warehouse or to unload the target object120from storage locations and load the target object120onto a vehicle for shipping. The tasks are functions performed or executed by the robotic system100for the physical transformation upon the unloading unit102, the transfer unit104, the transport unit106, the loading unit108, the robotic unit110, or a combination thereof.

For example, the task can include moving the target object120from one location, such as a container, bin, cage, basket, shelf, platform, pallet, or conveyor belt, to another location. The robotic system100or a portion of the robotic system100can be configured to execute a sequence of actions, such as operating one or more components therein, to execute a task.

The target object120can represent one or more containers to be displaced or moved by the robotic system100. An example of the target object120can include bins, boxes, crates, enclosures, packages, or a combination thereof. The target object120will be further described later.

In this example, an operation environment124for the robotic system100is described as the area and items126in the area where the robotic unit110operates within. The operation environment124includes the robotic unit110itself. As a specific example, the operation environment124can also include the controller112, the transfer unit104, the target object120, or a combination thereof. The operation environment124can also include other of the items126shown or not shown inFIG.1that can affect the operations of the robotic unit110. Similarly, the operation environment124can also include or be described as the area and the items126in the area where the unloading unit102operates within. For brevity, the robotic system100will be described with respect to the robotic unit110but can be applicable to the unloading unit102or other parts of the robotic system100that performs articulation or movement associated with the robotic system100.

FIG.1illustrates examples of the possible functions and operations that can be performed by the various units of the robotic system100in handling the target object120and it is understood that the environment, including the operation environment124, and conditions can differ from those described hereinafter. For example, the unloading unit102can be a vehicle offloading robot configured to transfer the target object120from a location in a carrier, such as a truck, to a location on a conveyor belt.

Also, the transfer unit104, such as a palletizing robot, can be configured to transfer the target object120from a location on the conveyor belt to a location on the transport unit106, such as for loading the target object120on a pallet on the transport unit106. In another example, the transfer unit104can be a piece-picking robot configured to transfer the target object120. In completing the operation, the transport unit106can transfer the target object120from an area associated with the transfer unit104to an area associated with the loading unit108, and the loading unit108can transfer the target object120, such as by moving the pallet carrying the target object120, from the transfer unit104to a storage location, such as a location on the shelves.

For illustrative purposes, the robotic system100is described in the context of a shipping center; however, it is understood that the robotic system100can be configured to execute tasks in other environments or for other purposes, such as for manufacturing, assembly, packaging, healthcare, or other types of automation. It is also understood that the robotic system100can include other units, such as manipulators, service robots, modular robots, that are not shown inFIG.1. For example, in some embodiments, the robotic system100can include a depalletizing unit for transferring the objects from cages, carts, or pallets onto conveyors or other pallets, a container-switching unit for transferring the objects from one container to another, a packaging unit for wrapping the objects, a sorting unit for grouping objects according to one or more characteristics thereof, a piece-picking unit for manipulating the objects differently, such as sorting, grouping, and/or transferring, according to one or more characteristics thereof, or a combination thereof.

The controller112can provide the intelligence for the robotic system100or a portion of the robotic system100to perform the tasks. As an example, the controller112can control the operations of the robotic unit110and the motions of the robotic unit110to perform the tasks. The controller112can control operations of the robotic unit110to perform tasks associated with the target object120. The controller112can provide manual operation instructions to the robotic unit110as well as pre-programmed operations with a trajectory. Manual operations do not have pre-programmed trajectory for motion of at least a portion of the robotic unit110.

The controller112can provide parallel processing of executing instructions to perform a task. As an example, the controller112can parallel process instructions for collision checking to avoid a collision128by the robotic unit110as well as simultaneously executing other instructions related to the manual operation without waiting for the for the collision checking to be completed. The collision128is a physical contact by the robotic unit110with at least one of the items126in the operation environment124that is not included as part of the instruction of the operation of the robotic unit110. The collision128can also include physical contact by at least a portion of the robotic unit110with another portion of the robotic unit110that is not included as part of the instruction of the operation of the robotic unit110.

For illustrative purposes, the robotic system100is described with separate components, such as the robotic unit110and the controller112, although it is understood that the robotic system100can be organized differently. For example, the robotic system100can include the functions provided by the controller112distributed throughout the robotic system100and not as a separate enclosure for the controller112as shown inFIG.1. Also for example, the controller112can be included as a portion of the robotic unit110. Further for example, the controller112can be multiple enclosure each providing intelligences to different portions or units of the robotic system100.

Returning to the robotic unit110, the robotic unit110can include a gripper122. The robotic unit110can utilize the gripper122to move the target object120in the transfer unit104. As described earlier, the controller112can provide the intelligences for the robotic unit110. Similarly, the controller112can also provide the intelligence for the gripper122.

As an example, the intelligence from the controller112can be distributed with the robotic unit110. As a specific example, the gripper122can also provide some intelligence for the operation of the gripper122and can interact with the intelligence from the controller112or the distributed intelligence as part of the robotic unit110.

Referring now toFIG.2, therein is shown an example of a block diagram of the robotic system100. The example shown inFIG.2can be for the robotic system100shown inFIG.1. In one embodiment, the robotic system100can include a control unit202, a storage unit206, a communication unit212, a user interface216, an actuation unit220, and a sensor unit230. In one embodiment, one or more of these components can be combined in the controller112as depicted by a dashed box.

The controller112can house a portion of the robotic system100. For example, the controller112can be a case, a chassis, a box, a console, a computer tower, or a computer motherboard. As a specific example, the operation environment124ofFIG.1can include the controller112. Also as a specific example, the robotic unit110ofFIG.1can also include the controller112. Further as a specific example, the controller112can be distributed in the robotic system100. Yet further as a specific example, there can be a plurality of the controller112within, external to, or a combination thereof with respect to the robotic system100.

Continuing with the example, the control unit202, the storage unit206, the communication unit212, or a combination thereof can be housed and included in the controller112. Also for example, the control unit202, the storage unit206, the communication unit212, or a combination thereof can be housed and included in the controller112while the user interface216, can be accessible external to the controller112.

While one or more portions of the robotic system100can be housed in or on the controller112, other portions of the robotic system100can be external to the controller112. For example, the user interface216, the actuation unit220, the sensor unit230, or a combination thereof can be external to the controller112while the control unit202, the storage unit206, and the communication unit212, are housed and included in the controller112. Other combinations of portions of the robotic system100or the robotic unit110can be housed in the controller112or together.

Further, for example, the operation environment124can include the robotic unit110, the controller112, the sensor unit230, the user interface216, the communication unit212, the storage unit206, the control unit202, or a combination thereof. As a specific example, the robotic unit110can include the sensor unit230, the communication unit212, the storage unit206, the control unit202, or a combination thereof.

The control unit202can execute a software210to provide the instructions and intelligence of the robotic system100. The control unit202can also execute the software210for the other functions of the robotic system100, the robotic unit110, the controller112, or a combination thereof. The control unit202can be implemented in a number of different ways. For example, the control unit202can be a processor, an application specific integrated circuit (ASIC), an embedded processor, a microprocessor, a hardware control logic, a hardware finite state machine (FSM), a digital signal processor (DSP), or a combination thereof.

For illustrative purposes, the control unit202is shown as a single element, although it is understood that the control unit202can represent a number of devices and a distribution of compute resources. For example, the control unit202can be a distribution of compute resources throughout and external to the robotic system100. Also for example, the control unit202can be distributed between the controller112, the robotic unit110, the sensor unit230, the gripper122ofFIG.1, or a combination thereof. The software210can also be distributed between the controller112, the robotic unit110, the sensor unit230, the gripper122, or a combination thereof.

The control unit202can perform parallel processing of instructions of the software210and can be implemented in a number of ways. For example, the control unit202can include multiple processing cores each capable of executing instructions from the software210. Also for example, the control unit202can include a pipeline architecture capable of executing multiple instructions of the software210before completion of the previous instruction. Further, for example, the control unit202can execute multiple instructions of the software210with distributed computing in addition to processing cores and pipeline architecture.

The control unit202can include a control interface204. The control interface204can be used for communication between the control unit202and other functional units of the robotic system100, the robotic unit110, the sensor unit230, or a combination thereof. The control interface204can also be used for communication that is external to the robotic system100. The control interface204can receive information from the other functional units or from external sources, or can transmit information to the other functional units or to external destinations. The external sources and the external destinations refer to sources and destinations external to the robotic system100.

The control interface204can be implemented in different ways and can include different implementations depending on which functional units or external units are being interfaced with the control interface204. For example, the control interface204can be implemented with a pressure sensor, an inertial sensor, a microelectromechanical system (MEMS), optical circuitry, waveguides, wireless circuitry, wireline circuitry, an application programming interface, or a combination thereof.

The storage unit206can store the software210. For illustrative purposes, the storage unit206is shown as a single element, although it is understood that the storage unit206can represent a number of devices and a distribution of storage elements. Also for illustrative purposes, the robotic system100is shown with the storage unit206as a single hierarchy storage system, although it is understood that the robotic system100can have the storage unit206in a different configuration. For example, the storage unit206can be formed with different storage technologies forming a memory hierarchal system including different levels of caching, main memory, rotating media, or off-line storage. Also for example, the storage unit206can be distributed between the controller112, the robotic unit110, the gripper122, or a combination thereof. The software210can also be distributed between the controller112, the robotic unit110, the gripper122, or a combination thereof.

The storage unit206can include a storage interface208. The storage interface208can be used for communication between the storage unit206and other functional units of the robotic system100. The storage interface208can also be used for communication external to the robotic system100. The storage interface208can receive information from the other functional units of the robotic system100or from external sources, or can transmit information to the other functional units of the robotic system100or to external destinations. The external sources and the external destinations refer to sources and destinations external to the robotic system100.

The storage interface208can include different implementations depending on which functional units or external units are being interfaced with the storage unit206. The storage interface208can be implemented with technologies and techniques similar to the implementation of the control interface204.

The communication unit212can enable communication to and from the robotic system100, including communication between portions of the robotic system100, external devices, or a combination thereof. For example, the communication unit212can permit the robotic system100to communicate with an external device, such as an external computer, an external database, an external machine, an external peripheral device, or a combination thereof through a communication path238.

The communication path238can span and represent a variety of networks and network topologies. For example, the communication path238can include wireless communication, wired communication, optical communication, ultrasonic communication, or the combination thereof. For example, satellite communication, cellular communication, Bluetooth, Infrared Data Association standard (IrDA), wireless fidelity (WiFi), and worldwide interoperability for microwave access (WiMAX) are examples of wireless communication that can be included in the communication path238. Cable, Ethernet, digital subscriber line (DSL), fiber optic lines, fiber to the home (FTTH), and plain old telephone service (POTS) are examples of wired communication that can be included in the communication path238.

Further, the communication path238can traverse a number of network topologies and distances. For example, the communication path238can include direct connection, personal area network (PAN), local area network (LAN), metropolitan area network (MAN), wide area network (WAN), or a combination thereof. Also for example, the communication path238can also include information exchange between elements within the robotic system100. As a specific example, the communication path238can be between the parts of the robotic unit110, with the sensor unit230, with the controller112, or a combination thereof. The control unit202can further execute the software210for interaction with the communication path238via the communication unit212.

The communication unit212can also function as a communication hub allowing the robotic system100to function as part of the communication path238and not be limited to be an end point or terminal unit to the communication path238. The communication unit212can include active and passive components, such as microelectronics or an antenna, for interaction with the communication path238.

The communication unit212can include a communication interface214. The communication interface214can be used for communication between the communication unit212and other functional units of the robotic system100. The communication interface214can receive information from the other functional units of the robotic system100or from external sources, or can transmit information to the other functional units of the robotic system100or to external destinations. The communication interface214can include different implementations depending on which functional units are being interfaced with the communication unit212. The communication interface214can be implemented with technologies and techniques similar to the implementation of the control interface204.

The control unit202can operate the user interface216to present or receive information generated by the robotic system100. The user interface216can include an input device and an output device. Examples of the input device of the user interface216can include a keypad, a touchpad, soft-keys, a keyboard, a microphone, sensors for receiving remote signals, a camera for receiving motion commands, or any combination thereof to provide data and communication inputs. Examples of the output device can include a display interface218and an audio interface232.

The display interface218can be any graphical user interface such as a display, a projector, a video screen, or any combination thereof. The audio interface232can include speakers, microphones, headphones, subwoofers, sound components, transducers, or any combination thereof. The display interface218and the audio interface232allow a user of the robotic system100to interact with the robotic system100. The display interface218and the audio interface232can be optional.

The robotic system100can also include the actuation unit220. The actuation unit220can include devices, for example, motors, springs, gears, pulleys, chains, rails, wires, artificial muscles, electroactive polymers, or a combination thereof, configured to drive, manipulate, displace, orient, re-orient, or a combination thereof, the structural members or mechanical components of the robotic system100about or at a corresponding mechanical joint. The control unit202can operate the actuation unit220, to control or manipulate the actuation unit220.

For illustrative purposes, the actuation unit220is shown as a single element, although it is understood that the actuation unit220can represent a number of devices and be a distribution of actuators. For example, the actuation unit220can be distributed throughout the robotic system100. Also for example, the actuation unit220can be distributed throughout the robotic unit110, the gripper122, or a combination thereof.

The actuation unit220can include an actuation interface222. The actuation interface222can be used for communication between the actuation unit220and other functional units of the robotic system100, the robotic unit110, the gripper122, or a combination thereof. The actuation interface222can also be used for communication that is external to the robotic system100. The actuation interface222can receive information from the other functional units of the robotic system100or from external sources, or can transmit information to the other functional units or to external destinations. The actuation interface222can function as a source for the actuation process, such as gas lines.

The actuation interface222can include different implementations depending on which functional units of the robotic system100or external units are being interfaced with the actuation unit220. The actuation interface222can be implemented with technologies and techniques similar to the implementation of the control interface204. The actuation interface222can also be implemented with pneumatic or gas devices.

The robotic system100can include the sensor unit230configured to obtain sensor readings246used to execute the tasks and operations, such as for manipulating the structural members of the robotic system100, the robotic unit110, the gripper122, or a combination thereof. The sensor unit230can also be configured to obtain the sensor readings246for portions of the robotic system100. For example, the sensor unit230can obtain the sensor readings246for the robotic unit110, the gripper122, or a combination thereof. Also for example, the sensor unit230can obtain the sensor readings246for the items126ofFIG.1operated upon by the robotic system100, the robotic unit110, the gripper122, or a combination thereof. As a specific example, the sensor unit230can provide the sensor readings246for the target object120ofFIG.1.

The sensor readings246can include information or data from the sensor unit230to detect events or changes in the environment of the robotic system100and to send the information to portions of the robotic system100, external devices, or a combination thereof to facilitate the tasks. Examples for the sensor readings246can include image readings, optical readings, pressure reading, distance reading, or a combination thereof. Also as examples, the sensor readings246can represent the distance from an object that can cause collision or impact to the robotic unit110or a portion of the robotic unit110.

For illustrative purposes, the sensor unit230is shown as a single element, although it is understood that the sensor unit230can represent a number of devices. For example, the actuation unit220can be distributed throughout the robotic system100. Also for example, the actuation unit220can be distributed throughout the robotic unit110, the gripper122, or a combination thereof.

The sensor unit230can include a sensor interface224. The sensor interface224can be used for communication between the sensor unit230and other portions of the robotic system100, the robotic unit110, or a combination thereof. The sensor interface224can also be used for communication that is external to the robotic system100, the robotic unit110, or a combination thereof. The sensor interface224can receive information from the other portions of the robotic system100or from external sources, or can transmit information to the other portions of the robotic system100or to external destinations. As a specific example, the sensor interface224can provide communication with and between the robotic unit110, the gripper122, or a combination thereof as well as with the other portions of the robotic system100.

The sensor interface224can include different implementations depending on which functional units of the robotic system100or external units are being interfaced with the sensor unit230. The sensor interface224can be implemented with technologies and techniques similar to the implementation of the control interface204.

Referring now toFIG.3, therein is shown another example of a robotic system300in a further embodiment. The robotic system300can includes one or more structures, such as robots or robotic devices, configured to execute one or more tasks. Aspects of the object collision avoidance mechanism can be practiced or implemented by the various structures. The similarities to the robotic system100are not all repeated for the robotic system300. Additional information or distinctions from the robotic system100can be described inFIG.3. The block diagram inFIG.2is also applicable to the robotic system300and not repeated for brevity.

In the example and embodiment inFIG.3, the robotic system300can include a transport unit306, a robotic unit310, a controller312, or a combination thereof. The robotic system300or a portion of the robotic system300can be configured to execute one or more tasks.

For the embodiment shown inFIG.3and as an example, the transport unit306can transfer the target object320for the task of processing by the robotic unit310. The transport unit306can include shelves334or bins loaded with the target object320and rolled to the place to be accessed by the robotic unit310. The transport unit306can also include a door336between the transport unit306and a portion of an operation environment324for the robotic unit310to operate.

The door336can provide separation of the interior of the transport unit306and the portion of the operation environment324where the robotic unit310is located, in this example. The door336can be in an open position or a closed position. The open position allows the robotic unit310access into the interior of the transport unit306. The closed position prevents the robotic unit310access to the interior of the transport unit306.

Continuing the example ofFIG.3, the robotic unit310can operate on the target object320. As a specific example, the robotic unit310can grab the target object320from the transport unit306. The robotic unit310can secure and move the target object320for and during processing by the robotic system300. The user interface216can invoke the tasks to secure, move, process, or a combination thereof the target object320by the robotic unit310, the robotic system300, or a combination. The user interface216can provide manual commands to the robotic system300, the robotic unit310, or a combination thereof. The user interface216can be part of or operates with the controller312.

For example, the manual commands can include a jog command338. The jog command338instructs at least a portion of the robotic unit310to move by manual operation. As an example, the jog command338does not include a pre-planned trajectory of motion but instead includes commands to direct the robotic unit310by the manual operation.

Similar to the controller112ofFIG.1, the controller312can provide the instruction, intelligence, or operational control for the task for the robotic system300, the robotic unit310, or a combination thereof. The controller312can also provide the collision avoidance with the operation environment324of the robotic unit310within the robotic system300, the transport unit306, or a combination thereof. The tasks are functions performed or executed by the robotic system300to effectuate the physical transformation with the transport unit306, the robotic unit310, the target object320, or a combination thereof.

The operation environment324for the robotic system300is similar as described for the robotic system100. In this example, the operation environment324for the robotic system300is described including the physical space and the items126in the physical space in which the robotic unit310operates. The operation environment324includes the robotic unit310itself.

Continuing this example, the operation environment324can provide information about the robotic system300, the robotic unit310, or a combination thereof. Returning to the example of the transport unit306, the operation environment324can include a chamber where the robotic unit310is housed within. The operation environment324represent the physical environment, the information for the physical environment, or a combination thereof. The operation environment324when representing the physical environment can also include information about the door336, such as location, whether in the open position, the closed position, the location and information of the shelves334, the location and information of the target object320, or a combination thereof.

Further continuing this example, the operation environment324can also other types of the items126including tooling as well as one or more mechanisms within the chamber for executing the task for the processing the target object320. The operation environment324can also include a further instance of the robotic unit310or other robotic devices capable of movement similar to, that is the same as, or different to the robotic unit310. The other tooling, mechanism(s), structures, or a combination thereof can be stationary and at a fixed position or capable of being moved for processing the target object320. The operation environment324can also include other instances of the items126shown or not shown inFIG.3that can affect the operations of the robotic unit310.

FIG.3illustrates examples of the possible functions and operations that can be performed by the various units of the robotic system300in handling the target object320and it is understood that the environment, including the operation environment324, and conditions associated with the environment can differ from those described. For example, the unloading unit102ofFIG.2can be a vehicle offloading robot configured to transfer the target object320from a location in a carrier, such as a truck, to a location on a conveyor belt. Also as an example, the transfer unit104ofFIG.1can be configured to transfer the target object320to the transport unit306.

For illustrative purposes, the robotic system300is described in the context processing the target object320and can be an extension of or be included in the robotic system100as described inFIG.1, although, it is understood that the robotic system300can be configured to operate separately from the operational example described inFIG.1. The robotic system300can execute tasks in other environments or for other purposes, such as for manufacturing, assembly, packaging, healthcare, or other types of automation. It is also understood that the robotic system300can include other units, such as manipulators, service robots, modular robots, that are not shown inFIG.3.

Also for illustrative purposes, the robotic system300is described with separate components, such as the robotic unit310and the controller312, although it is understood that the robotic system300can be organized differently. For example, the robotic system300can include the functions provided by the controller312distributed throughout the robotic system300and not as a separate enclosure from the transport unit306, the robotic unit310, or a combination thereof as shown inFIG.3. Also for example, the controller312can be included as a portion of the transport unit306, the robotic unit310, or a combination thereof. Further for example, the controller312can be multiple enclosure each providing intelligence to different portions or units of the robotic system300.

Referring now toFIG.4, therein is an example of the robotic unit310. Description for the robotic unit310can also be applicable to the robotic unit110ofFIG.1as well as other robotic devices inFIG.1, such as the loading unit108ofFIG.1.

In this example, the robotic unit310includes links340and joints342. The joints342provide the articulation or actuating portion of the robotic unit310. The joints342are typically between the links340. Collectively, the joints342can provide the articulation or the actuation of the robotic unit310as a whole with the links340providing the rigid members of the robotic unit310. As an example, movement of the joints342can be implemented by one or more of the actuation unit220ofFIG.2of the same or different types.

Some of the joints342can provide rotational articulation or, more specifically, rotational actuation of the links340attached to each of the joints342. Other instances of the joints342can provide translational or linear articulation or actuation, such as the instance of the joints342attached to or articulating a gripper422. The gripper422can be similarly described as the gripper122ofFIG.1.

Continuing this example, the robotic unit310is shown at one end attached to the interior of the operation environment324representing the interior chamber of the robotic system300where the robotic unit310operates. In this example, the joints342can include a first joint444, a second joint446, a third joint448, a fourth joint450, a fifth joint452, or a combination thereof. Also for example, the links340can include a first link454, a second link456, a third link458, a fourth link460, or a combination thereof.

As a specific example, the first joint444is shown connecting the interior of the chamber to the first link454. The first joint444is shown inFIG.4as providing rotational actuation and acts upon the first link454. The second joint446is shown as providing rotational actuation and acts upon the second link456from the first link454. The third joint448is shown as providing rotation actuation and acts upon the third link458from the second link456. The fourth joint450is shown as providing rotation actuation and acts upon the fourth link460from the third link458. The fifth joint452is shown as providing linear actuation and acts open the gripper422from the fourth link460.

As examples, the shelves334ofFIG.3, the target object320ofFIG.3, and the robotic unit310can each be modeled with a buffer zone462. The robotic system300, the controller312ofFIG.3, or a combination thereof operates the robotic unit310and can utilize the buffer zone462to avoid the collision128ofFIG.1or unintended contact within the operation environment324, the transport unit306ofFIG.3, or with the door336ofFIG.3.

The buffer zone462provides a threshold distance464around the robotic unit310to determine whether a collision can occur resulting from a movement. The threshold distance464is defined by a speed466, a cycle number468, and a cycle time470. The speed466is the rate of motion by at least a portion of the robotic unit310based on the jog command338. The cycle number468is the number of control intervals472. The cycle time470is the amount of time or duration for each of the control intervals472. The control intervals472will be described later.

As an example, the buffer zone462can surround the model representing the entire physical outline of the robotic unit310and the dimensionality of coverage for the buffer zone462can vary. For example, the buffer zone462can include three dimensional coverage surrounding the physical outline of the robotic unit310. Also for example, the buffer zone462can also provide a partial outline of the robotic unit310and can be two dimensional. The two dimensional version of the buffer zone462can be planar to the plane of movement. The two dimensional version of the buffer zone462can also be nonplanar to coincide with a nonlinear path of movement. The buffer zone462can also be limited, as needed, to a side of the robotic unit310where the side is to the direction of the movement. Further for example, the buffer zone462can also be one dimensional from the further extent along a side of the robotic unit310where the side is to the direction of the movement.

The size or extent of the buffer zone462as determined by the threshold distance464beyond the physical outline of the robotic unit310, the links340, the joints342, or a combination thereof can be fixed, can be dynamically adjusted, or can be interchangeable between fixed or adjusted. For example, the size of the buffer zone462can be fixed and based on the complexity of the operation environment324surrounding the robotic unit310, the links340, the joints342, or a combination thereof. Also for example, the threshold distance464of the buffer zone462can also be fixed or based on the speed466of movement of the robotic unit310, the links340, the joints342, or a combination thereof. The threshold distance464can be adjusted based on changes to the operation environment324, collision events, or a combination thereof.

As specific examples, the buffer zone462can include segments to avoid the unintended contact for each of the joints342, each of the links340, the gripper422, or a combination thereof. As an example, the segments of the buffer zone462can correspond to the joints342, the links340, the gripper422, or a combination thereof and can include a first buffer, a second buffer, a third buffer, a fourth buffer, a fifth buffer, or a combination thereof. For brevity, the description for the one dimensional, two dimensional, and three dimensional outlines as described for the buffer zone462can also apply to the individual buffers from the first buffer through the fifth buffer. The dimensionality and the threshold distance464of the buffer zone462can be the same or different between the first buffer through the fifth buffer.

Continuing the example, the first buffer can represent the collision avoidance threshold or potential collision detection threshold for the first joint444and the first link454. The second buffer can represent the collision avoidance threshold or potential collision detection threshold for the second joint446and the second link456. The third buffer can represent the collision avoidance threshold or potential collision detection threshold for the third joint448and the third link458. The fourth buffer can represent the collision avoidance threshold or potential collision detection threshold for the fourth joint450and the fourth link460. The fifth buffer can represent the collision avoidance threshold or the potential collision detection threshold for the fifth joint452and the gripper422.

As an example, the robotic unit310can include the sensor unit230to provide the sensor readings246ofFIG.2for the buffer zone462around the joints342, the links340, or a combination thereof. In this example, the sensor readings246can provide indications if the distance threshold or boundary outlined by the threshold distance464for the buffer zone462has been or can be triggered.

Also for example, the robotic unit310can include a single instance or the sensor unit230. As a specific example, the sensor unit230can be at the further extent of the robotic unit310away from the portion attached to the wall of the chamber in the operation environment324. Continue the specific example, the sensor unit230can be attached or located at the fifth joint452or the gripper422.

Further for example, the sensor unit230can include a first sensor, a second sensor, a third sensor, a fourth sensor, a fifth sensor, or combination thereof. The first sensor can provide the first buffer. The first sensor can monitor the first buffer and be located at the first link454at the opposite end where the first joint444is located or at the first joint444. The second sensor can provide the second buffer. The second sensor can monitor the second buffer and be located at the second link456at the opposite end where the second joint446is located or at the second joint446. The third sensor can provide the third buffer. The third sensor can monitor the third buffer and be located at the third link458at the opposite end where the third joint448is located or at the third joint448. The fourth sensor can provide the fourth buffer. The fourth sensor can monitor the fourth buffer and be located at the fourth link460at the opposite end where the fourth joint450is located or at the fourth joint450. The fifth sensor can provide the fifth buffer. The fifth sensor can monitor the fifth buffer and be located at the gripper422or at the gripper422.

The complexity of the operation environment324can determine the number of instances of the sensor unit230and the number of the sensor readings246for the robotic unit310, the robotic system300, or a combination thereof. The complexity of the operation environment324can also determine the type or types of the sensor unit230and the type of information from each the sensor readings494utilized or provided to the robotic unit310, the robotic system300, or a combination thereof.

For example, the operation environment324described as non-complex refers to very little obstructions near the robotic unit310, the items126surrounding the robotic unit310is static or stationary, the items126surrounding the robotic unit310do not move, or a combination thereof. The operation environment324can also be not complex if the examples noted earlier regarding the buffer zone462applies to the range of motion or paths of motion for the robotic unit310. The operation environment324can be more complex, as opposed to non-complex, if any of the examples noted earlier or if obstruction changes occur in the path of the range of motion for the robotic unit310.

Further, examples of a non-complex type of the operation environment324include a static environment where there are no other moving objects, with few obstacles or the items126, or a combination thereof such that the area around the robotic unit310can be operated for a threshold number of cycles without possibility of collision. The threshold number of cycles can refer to the operation of the robotic unit310to execute the jog command338ofFIG.3for a pre-defined period of time before the robotic unit310would be determined to collide with another object or item in the operation environment324. The least complex of the operation environment324would be an empty room where there are no objects or the items126for potential collision within the range of motion for the robotic unit310.

If the operation environment324is non-complex or is considered simple, then the controller312, the robotic system300, or a combination thereof can utilize less number or complexity of the sensor readings246to avoid the collision128for the robotic unit310. As an example, the robotic system100can utilize one instance of the sensor unit230and the sensor readings246can represent the buffer zone462that can be one dimensional for the distance threshold at a side of the links340facing the direction of movement. The selection for this example simplifies the calculation for the controller312, the robotic system300, or a combination thereof and can also reduce power consumption for the controller312, the robotic unit310, the sensor unit230, the robotic system300, or a combination thereof. The simplification also allows for a faster determination to avoid the collision128thereby improving the operation of the robotic system300, the robotic unit310, or a combination thereof.

Examples of a complex environment can be described as environments where the range of motion of the robotic unit310is limited by static obstacles, dynamic obstacles, or a combination thereof. For example, if the operation environment is complex or not simple or more towards complex along a complexity spectrum, then the controller312, the robotic system300, or a combination thereof can utilize more number or complexity of the sensor unit230to avoid the collision128for the robotic unit310. As an example, the robotic system100can utilize multiple instances of the sensor unit230and the sensor readings246can represent the buffer zone462. The buffer zone462can be partitioned to smaller segments, such as from the first buffer through the fifth buffer. The dimensionality can vary of each segment of the buffer zone462.

Continuing with the example, some of the segment of the buffer zone462can be one dimensional for the threshold distance464, while other segments can be two dimensional, while yet other segments can be three dimensional. The dimensionality for each of the sensor unit230, for each of the segment of the buffer zone462, for each of the links340, for each of the joints342, or a combination thereof can be determined by the complexity of the operation environment324relative to the potential path of motion for the different parts of the robotic unit310. In other words, the complexity level of the operation environment324can differ relative to different portions of the robotic unit310. The range of motion or potential path of motion for some of the links340can operate at a portion of the operation environment324that can be determined to be simple while other instances of the links340can operate at different portion of the operation environment324that are determined to be of a different level of complexities which can require different types of the sensor readings246, the sensor unit230, or a combination thereof to avoid the collision128.

The variation of the sensor unit230, the sensor readings246, the buffer zone462, or a combination thereof along the different instances of the links340of the robotic unit310provides the robotic system300the ability to simplify the calculations, as noted earlier, along with the attendant benefits but also provides the flexibility to handle different or varying complexities in the operation environment324as needed to balance the accuracy for collision avoidance. The ability to mix and match and account for different levels of complexity in the operation environment324allows the robotic system300to improve efficiency while increasing functional accuracy to avoid the collision128.

Referring now toFIG.5, therein is shown an example of a control flow for the robotic system300ofFIG.3. The description can also apply to the robotic system100described inFIG.1. The control flow can include a model generation module502, a command reception module504, a model update module506, a collision check module508, a calculation limit module510, a command flag module512, a command confirmation module514, a command transfer module516, a mode check module518, a mode termination module520, or a combination thereof.

For brevity, the portion of the control flow is described as a collision check thread522can include the model update module506, the collision check module508, the command confirmation module514, or a combination thereof. The portion of the control flow is described as a jog operation thread524can include the command reception module504, the calculation limit module510, the command transfer module516, the mode check module518, the mode termination module520, or a combination thereof.

The control flow can be implemented by the software210ofFIG.2and executed by the control unit202ofFIG.2, the controller312ofFIG.3, or a combination thereof. Commands can be generated by the control unit202, the controller312, the robotic unit310ofFIG.3, the robotic system300ofFIG.3, or a combination thereof. The commands can be initiated with the user interface216.

The software210can be stored in the storage unit206ofFIG.2and can be executed by the control unit202. The control flow can include transmitting commands or to invoke actions utilizing the communication unit212ofFIG.2, the communication interface214ofFIG.2, the control interface204ofFIG.2, the storage interface208ofFIG.2, the actuation interface222ofFIG.2, the sensor interface224ofFIG.2, or a combination thereof as needed. The control flow can be executed by the gripper422ofFIG.4, the robotic unit310, the controller312, the robotic system300, or a combination thereof.

The model generation module502, the command reception module504, the model update module506, the collision check module508, the calculation limit module510, the command flag module512, the command confirmation module514, the command transfer module516, the mode check module518, and the mode termination module520can be coupled to one another using wired or wireless connections, by including an output of one module as an input of the other, by including operations of one module influence operation of the other module, or a combination thereof. The portions of the control flow can be directly coupled without intervening structures or objects other than the connector there-between, or indirectly coupled to one another.

The model generation module502provides an environment model526for the operation environment324ofFIG.3as information about the surrounding environment for the robotic unit310. The model generation module502can also generate a robot model528for the robotic unit310. The environment model526can also include the robot model528.

The environment model526can include information about and to represent number of the items126ofFIG.1, types of the items126, location of the items126, dimension of the items126, mobility of the items126, status or state of the items126. Examples of the items126can include the target object320, other robotic devices such as the unloading unit102ofFIG.1or the transfer unit104ofFIG.1, the loading unit108ofFIG.1, the door336ofFIG.3, the shelves334ofFIG.3, other instances of the robotic unit310, or a combination thereof.

As a specific example, the environment model526can be implemented a coordinate system that includes a representation of the robotic unit310, the position and dimensions of objects, such walls, the door336, the shelves334, containers, around the robotic unit310. The robotic system300, the controller312, or a combination thereof can receive information about the items126in the operation environment324and those objects can be included in the environment model526.

The robot model528can represent the links340ofFIG.4, the joints342ofFIG.4, the sensor unit230ofFIG.2, or a combination thereof. The robot model528can include physical dimensions for each of the links340, each of the joints342, the physical relationship of the links340and the joints342to the robotic unit310, or a combination thereof. The robot model528can include range of motion or motion capability for each of the links340, each of the joints342, or a combination thereof. The robot model528can also include the buffer zone462ofFIG.4as well as a segmented form as the first buffer ofFIG.4, the second buffer ofFIG.4, the third buffer ofFIG.4, the fourth buffer ofFIG.4, the fifth buffer ofFIG.4, or a combination thereof.

The robot model528can be a mesh corresponding to the buffer zone462with a size that is larger than the actual physical measurements/dimensions of the robotic unit310to provide a buffer region around the robotic unit310. As an example, the model generation module502can size the buffer zone462, the threshold distance464ofFIG.3, or a combination thereof of the robot model528to an amount of time or number of the control intervals472to travel from the edge of the buffer zone462to the actual contact to the robotic unit310based on the speed466ofFIG.4of the jog command338ofFIG.3.

As a specific example for higher movement value for the speed466, the threshold distance464of the buffer zone462can be greater than for lower movement value for the speed466of the robotic unit310. In some embodiments, the threshold distance464of the buffer zone462can be limited to a maximum or minimum if the buffer zone462is too big, the robotic unit310cannot maneuver or operate in certain spaces, even though, in reality, the robotic unit310can be safely jogged through the operation environment324.

As an example, the size or the threshold distance464ofFIG.3of the buffer zone462can be calculated based a product of the speed466of motion of the robotic unit310and number of the control intervals472and the unit time for each of the control intervals472, such as 2 milliseconds (ms) or 4 ms, that are considered to be collision free. The control intervals472will be described later.

The model generation module502can also provide the recognition of the sensor unit230used relative to the robotic unit310to determine the buffer zone462. For example, the model generation module502can determine the number of instances of the sensor unit230is utilized with the robotic unit310, the location of the sensor unit230, the specification or functionality of each of the sensor unit230to the dimensionality of the buffer zone462monitored for that particular instance of the sensor unit230, or a combination thereof. The flow can progress to the command reception module504.

The command reception module504receives an instruction for a task for the robotic unit310. The command reception module504can be an input from the user interface216ofFIG.3. As an example, the command reception module504can receive a command for a task when operating the robotic unit310, the robotic system300, or a combination thereof in manual operation.

An example of a command for manual operation is a jog command338, which is a command for a task to move the robotic unit310, the gripper422, another portion of the robotic unit310, or a combination thereof. The jog command338does not have a pre-planned trajectory but is controlled by the input received for manual operation. As an example, the jog command338can actuate the robotic unit310as a whole, specific instances of the links340and the joints342, only single and selected instance of the links340and the joints342to perform the task, or a combination thereof. Also for example, the jog command338can provide a single motion or can represent more than a single motion for at least a portion of the jog command338.

As a specific example, the robotic system300, the robotic unit310, or a combination thereof can be in a jogging mode for the manual operation. In the jogging mode, the instructions sent to the robotic system300, the robotic unit310, or a combination thereof can include or be limited to direction and the speed466. The jog command338or instructions differs from a trajectory in that there is no predefined start or endpoint or the speed466. With a trajectory, the robotic system300, the controller312, the robotic unit310, or a combination thereof is provided with the start and end point for the robotic unit310or the portion of the robotic unit310being actuated and can plan the trajectory ahead of time.

For example, in jogging mode, when the user activates the jog function or initiates the jogging mode with the user interface216, the robotic system300, the controller312, or a combination thereof, the activation will begin jogging or moving the at least a portion of the robotic unit310in the direction and the speed466entered or set by the user and will stop when the user ceases the jog command338or sends an end command530. The end command530is an instruction for at least a portion of the robotic unit310to stop motion. As an example, the end command530can occur when a button or dial used by an operator to maintain the jogging mode is released. The robotic unit310receives and executes in real-time the jog command338, the robotic system300, the controller312, or a combination thereof cannot plan the trajectory for the motion in advance.

The flow can progress for real-time parallel processing of the collision check thread522and the jog operation thread524to the model update module506and the calculation limit module510, respectively. The parallel processing refers to both paths of the control flow are being executed concurrently or simultaneously. The parallel processing refers to both paths of the control flow are not required to work sequentially. Real-time refers to the processing of an instruction for manual operation of the robotic unit310before completion of that instance of the instruction or a manual invocation for a termination of that instruction. As a specific example, the real-time parallel processing for the jog command338refers to the parallel processing of that instance of the jog command338through the collision check thread522and the jog operation thread524beginning at the invocation of that instance of the jog command338and before the completion of the same instance of the jog command338or the invocation of the end command530for the same instance of the jog command338.

The model update module506can update the environment model526based on past invocations of the jog command338and if additional information was gathered such that the environment model526needs to be updated or modified. For example, the environment model526can be updated when new external sensor information based on the sensor readings246from the sensor unit230is received indicating there is a change in the operation environment324, which can include a collision. The environment model526can also be updated according to predetermined intervals. The predetermined intervals can be set or determine in a number of ways. For example, the predetermined intervals can be based on default sampling frequency of different sensors for the sensor unit230. Also for example, the predetermined intervals can be specified since updating the environment model526can be resource intensive for the robotic system300. Further for example, the predetermined intervals can also be based on complexity of the operation environment324and can be highly dynamic for updating the environment model526more frequently. Further for example, the predetermined intervals can be on the order of seconds since manual operation would not be able to react that quickly compared to millisecond time frame. The flow can progress to the collision check module508.

Further, the model update module506can update the environment model526with information received from the external systems relative to the robotic unit310, the robotic system300, or a combination thereof. The information be associated with movable type of the items126in the operation environment324. As an example, the information for the operation environment324can be represented in the environment model526is binary information. As a specific example, the information received from sensors location at the door336or for the door336to indicate whether the door336is open or closed. In this example, the information can be binary as simply “open” or “closed” and position as well as the location of the door336in “open” position and “closed” position based on information received from external sensors. As an example, the environment model526provides an approximation of the operation environment324and is not required to be an exact or precise representation. Example of information from the external system can include information from external sensors, such as perimeter sensors, actuator readings or joint values to indicate position or pose of other movable type of the items126or other robots in the operation environment324, three dimensional camera, encoder that monitor the angle/position/speed of joints/servos.

As an example, notifications from the external systems does not need to be received in real-time and can be delayed slightly, such as in the order of one or two instances of the control intervals472. In other words, updates to every aspect of the environment model526need not occur in every instance of the control intervals472. The external systems that provide environment information can lag or have periods that are greater than one of the control intervals472, so even if there is a change in the operation environment324. The external system can take time to report the change to the controller312providing the function as the jogging controller. Also for example, an opening or closing of the door336is provided as binary information, so even if the door336is in the process of closing, the external reporting system can still report the door336as closed, even if the door336is still partially open or can take one or more instances of the control intervals472to register and report that the door336is actually closed. The robotic system300can provide an application programming interface (API) that can interface with the external systems to work with various types of sensors.

The collision check module508performs real-time determination of potential collision. For example, to avoid a collision, the robotic system300, the controller312, or a combination thereof can determine whether the invocation of the jog command338can result in a collision.

As an example, during jogging mode of the robotic unit310, the robotic system300can include a real-time collision checking mechanism that provides a method for preventing or mitigating errors, and particularly errors that lead to collisions between the robotic unit310and the items126in the environment model526. As a specific example, during manual jogging operation, the operator or user can make a mistake in controlling the robotic unit310, such mis-pressing the controls on the user interface216for the jog command338for the robotic unit310to go forward when intending to command the robotic unit310to move in reverse.

The collision check module508can determine the potential collision in a number of ways. For example, the robotic system300, the controller312, or a combination thereof can utilize the robot model528, the sensor readings246, or a combination thereof to determine if a proximity of any of the items126in the environment model526passes or crosses the buffer zone462. The collision check module508can also determine if the jog command338directs the movement of the robotic unit310along a path where the collision128ofFIG.1occurred, a continued traversal of the buffer zone462of the robot model528, or a combination thereof will lead to the collision128. If so, there is a potential for the collision128and the flow can pass to the command flag module512.

Also for example, the robot model528can include the granularity or the number for the sensor readings246to represent the number of, the types, the location, or a combination thereof of the sensor unit230based on the complexity of the operation environment324as captured in the environment model526as discussed in earlier. As specific examples, the robot model528can include the sensor readings246to represent the dimensionality for the number of segments, such as the first buffer through the fifth buffer, of the buffer zone462as discussed earlier. If any item in the operation environment324crosses any of the threshold including the first buffer through the fifth buffer being monitored and if the jog command338directs the movement of the robotic unit310along a path where a collision, a continued traversal of the buffer zone462of the robot model528, or a combination thereof will lead to a collision, then there is a potential for a collision and the flow can pass to the command flag module512.

It has been discovered that the embodiments as examples improve the efficiency of operation and detecting potential collision and avoid detecting false detection of collision by providing the robot model528with the threshold distance464of the buffer zone462or the threshold distance464that is greater than physical measurements of the robotic unit310to provide the benefit of extra time for the collision check calculations to complete and reduce processing time, bandwidth, or a combination thereof. Even though the buffer zone462is colliding with the items126in the operation environment324, the robotic unit310itself can still be collision free.

It has been also discovered that the embodiments as examples improve the efficiency of detecting potential collision by being able to size or resize the buffer zone462around the robotic unit310. The robot model528can include the buffer zone462to be larger for less complexity of the operation environment324represented by the environment model526, thereby reducing the processing time for traversal of the buffer zone462when there are very few or no number of the items126in the operation environment324that can impede the motion of the robotic unit310. On the other hand, the robot model528can include the buffer zone462to be smaller or reduce the threshold distance464for more complex type of the operation environment324represented by the environment model526, thereby not over-constraining the movement of the robotic unit310as well as reducing the processing time for traversal of the buffer zone462when there are many number of the items126in the operation environment324that can impede the motion of the robotic unit310.

If an instance of the items126in the operation environment324crosses the buffer zone462, or as a more specific example any of the first buffer through the fifth buffer being monitored, and that instance of the items126not already found in the environment model526for the operation environment324, then the model update module506can update the environment model526with the additional information. The additional information can also be passed along to command flag module512.

If the robotic system300, the controller312, the sensor unit230, or a combination thereof does not detect a traversal of the buffer zone462or the path of the jog command338does not lead to the collision128, then the jog command338is determined not be result in the collision128with the items126, known or unknown, in the operation environment324. In this situation, the flow can pass to the command confirmation module514.

For illustrative purposes, the robotic system300is described for determining an instance of the jog command338that can lead to the collision128based on the traversal of the buffer zone462and the path of the jog command338, although it is understood that the robotic system300can operate differently. For example, the robotic system300, the controller312, the robotic unit310, the sensor unit230, or a combination thereof can determine a potential occurrence of the collision128for the jog command338without traversing the path of the robotic unit310as part of the determination. In this example, the traversal of the buffer zone462alone can indicate the collision128or potential collision based on the jog command338. Continuing with this example, the robotic system300, the controller312, the robotic unit310, the sensor unit230, or a combination thereof can resize the buffer zone462or any of the first buffer through the fifth buffer such that the path based off the jog command338is not included in the determination of the collision or potential collision.

Continuing with the command flag module512, the command flag module512provides action for the robotic unit310based on a traversal of the buffer zone462or any one of the segments from the first buffer to the fifth buffer being monitored. The command flag module512can take a number of actions.

For example, the command flag module512can provide generate a warning. The warning can be delivered to the user interface216. The warning can include a haptic feedback, audio feedback, display feedback, or a combination thereof to the user interface216to notify a user to discontinue the invocation of the jog command338, to invoke the end command530, or a combination thereof. The warning can include information about the items126detected by the proximity readings of the sensor unit230. The warning can provide which of the sensor unit230from the first sensor ofFIG.4to the fifth sensor ofFIG.4that detected a proximity traversal to the first buffer to the fifth buffer, respectively. The warning can also provide information for any of the items126detected that is not found in the environment model526for the operation environment324.

Also for example, the command flag module512can also terminate the jog command338instead of, before, during, or after the generation and deliver of the warning. There can be situations where the robotic system300, the robotic unit310, the controller312, or a combination thereof can determine the need to terminate the jog command338.

For illustrative purposes, the example of the control flow shown inFIG.5depicts the flow terminating at the command flag module512, although it is understood that the robotic system300, the controller312, the robotic unit310, or a combination thereof can function differently. For example, the flow can allow an override of the warning or termination of the jog command338. Also for example, the flow can allow for the items126to be removed allowing for the jog command338to continue and the flow can progress to the command confirmation module514.

The command confirmation module514can generate an unobstructed status532that the jog command338being process has been determined by the robotic system300, the controller312, the robotic unit310, the sensor unit230, or a combination thereof is collision free. The command confirmation module514can determine the unobstructed status532when the jog command338will not result in the collision128by the robotic unit310within the operation environment324as provided by the environment model526, including inadvertent physical contact by a portion of the robotic unit310to another portion of the robotic unit310that is not part of the jog command338. The flow can progress to the command transfer module516.

Returning to the calculation limit module510, the calculation limit module510can determine if the collision check function through the collision check thread522has exceeded a collision check time limit534. As an example, the robotic system300, the controller312, or a combination thereof can provide instructions or commands, such as the jog command338, to the robotic unit310at the control intervals472. The control intervals472represent the frequency at which commands are sent to the robotic unit310. The value for each of the control intervals472can differ for the different types of the robotic unit310or different configuration for the robotic unit310. For example, the robotic system300, the controller312, the user, or a combination thereof can configure the value the control intervals472for some operations to different durations, such as 2 milliseconds (ms), 4 ms, or 6 ms. Also for example, for other operations the control intervals472can cannot be modified.

As an example for the values the control intervals472being 2 ms, the robotic system300, the controller312, or a combination thereof can send the jogging instructions for the jog command338at 2 m intervals. The jog command338can be sent to the jog operation thread524, as an example, for calculating values for the joints342or the actuation unit220ofFIG.2for the robotic unit310, which can include instruction for direction, position, the speed466, or a combination thereof, every 2 ms according to the control or command inputs by the operator/user. Calculations are performed for every instance of the joints342that will move to follow the jogging instruction or the jog command338. For example, if the jog command338is to move one of the links340in a direction, joint values will need to be calculated for each of the joints342to make that motion occur. The joint values can be calculated for each of the joints342that will actually move to execute the jog command338.

To continue the example, in order to prevent the collision128between the robotic unit310and the items126in the operation environment324, the robotic system300, the controller312, or a combination thereof can perform real-time collision checking with the collision check thread522in parallel during each of the control intervals472.

Also for example, the robotic system300, the controller312, or a combination thereof can perform the collision check to verify that the jog command338for a current instance of the control intervals472will not result in a collision. If a collision would occur, the robotic system300, the controller312, the collision check module508, the command flag module512, or a combination thereof can reject the jog command338, terminate the jogging operation, or a combination thereof.

For examples when the robotic system300, the controller312, the collision check thread522, or a combination thereof cannot complete the collision check, such as with the collision check module508, in one instance of the control intervals472, the calculation limit module510can determine an incomplete collision check for the jog command338and the flow can progress to the command flag module512. As an example, a complexity of the operation environment324, the environment model526, or a combination thereof can lead to an incomplete collision check due to limitations of processing bandwidth.

To avoid situations where an incomplete collision check will cause termination of the jogging operation based on the jog command338, the robotic system300, the controller312, or a combination thereof operates the collision check thread522dedicated to collision checking, environment modeling or updating with the model update module506, or a combination thereof that is separate or independent from and in parallel with the jog operation thread524.

As a specific example, the operation environment324around the robotic unit310does not change frequently and does not need to be updated during every instance of the control intervals472. The robotic unit310receiving the jog command338can be included as part of the environment model526since the environment model526is also used during collision checking also accounts for the robotic unit310with respect to the operation environment324.

In another example, the robotic system300, the controller312, or a combination thereof can operate in the operation environment324that is more complex with dynamic objects, such as the door336ofFIG.3that can open and close or other moving robots, the environment model526can be more complex such that the collision check calculations by the collision check module508can take multiple instances of the control intervals472to complete. In situations where the collision check calculations are not completed within a single instances of the control intervals472, the robotic system300, the controller312, the user interface216, or a combination thereof can continue providing the jog command338to the robotic unit310until the collision check time limit534has been reached as determined by the calculation limit module510.

Continuing the example, the collision check time limit534can be the number of the control intervals472, such as a total time period as 4 ms or two instances of the control intervals472or 6 ms or three instances of the control intervals472, until which a possible collision can occur. In some embodiments, the collision check time limit534can be based on a robot model528for the robotic unit310. If the collision check time limit534is reached without a response for the unobstructed status532from the collision check thread522, the robotic system300, the controller312, or a combination thereof can determine that a collision will occur and can terminate the jogging operation for the jog command338with the flow progressing to the command flag module512.

It has been further discovered that the embodiments as examples improve the efficiency of the robotic system300to accurately and reliably avoid the collision128by the robotic unit310through the use of the sensor unit230attached to the robotic unit310. The sensor unit230can include proximity sensors to provide the sensor readings246indicating any traversal of the proximity range generated by the buffer zone462. Since collision analysis by the collision check module508is performed on a short time frame, such as 2 ms or within the collision check time limit534determined by the calculation limit module510, the information from the sensor readings246is processed in real-time and with a very short window. The embodiments do not need detailed models, such as detailed vision information, based on three-dimensional point cloud information, and positions of the items126for the environment model526. The embodiments operate by determining if an item exists in the jogging path with the use the sensor unit230as proximity sensors attached to the robotic unit310, the links340, the joints342, or a combination thereof. Cameras can also be used as visual confirmation or additional validation to verify collision or potential collision, information in the environment model526, or to update the environment model526.

It has been still further discovered that the embodiments as examples improves the function of the manual operation of the robotic unit310by dynamically updating the environment model526as well as the robot model528. As the operation environment324is changed, the embodiments can detect traversal to the buffer zone462and if the item is not expected, the environment model526can be updated based on the traversal of the buffer zone462. The environment model526can be continued to be refined as additional information is received by additional traversal of the buffer zone462or the segments of the buffer zone462for the same or different instance of the items126in the operation environment324.

It has been yet further discovered that the embodiments as examples improve the function and efficiency of manual operation of the robotic unit310by parallel processing of the collision check thread522and the jog operation thread524in real-time. The real-time collision checking provides the robotic system300the benefits of continuing to provide the jog command338to the robotic unit310while performing the collision checks, which can include generating or updating the environment model526that describes the operation environment324around the robotic unit310and the robotic unit310itself with respect to the operation environment324, even if a calculation period for the collision check calculations exceed an instance of the control intervals472. For example, in situations where the operation environment324does not include a significant number of obstacles or the items126, such as and empty room with large distance between the robotic unit310and walls or in a static environment), then the collision check thread522can more likely generate the environment model526and perform the collision check calculations within one of the control intervals472.

The flow can progress from the calculation limit module510to the command transfer module516. The command transfer module516determines the jog command338being invoked can continue to be processed by the robotic unit310. As an example, the command transfer module516determines that the jog command338being invoked has been confirmed by the command confirmation module514that the collision check thread522, the collision check module508, or a combination thereof determines that movement of the robotic unit310based on the jog command338being analyzed should be collision free.

Continuing the example, the command transfer module516can determine that the jog command338should be collision free based on the unobstructed status532. The command transfer module516can transmit the jog command338to the robotic unit310for execution based on the unobstructed status532provided by the collision check thread522before the jog operation thread524reached the collision check time limit534. The flow can progress to the mode check module518.

The mode check module518determines if the robotic system300, the controller312, the robotic unit310is still in manual operation mode. The mode check module518will determine if the robotic unit310can accept another invocation of another manual command or the jog command338. If so, the flow loops back to the command reception module504. If not, the flow can progress to the mode termination module520.

The mode termination module520prohibits the manual operation of the robotic unit310. For example, the mode termination module520can reject or prevent execution of manual commands or further invocation of the jog command338provided by the user interface216.

For illustrative purposes, the control flow is described inFIG.5with the partition of modules and the functions for each of the modules, although it is understood that control flow can operate and be configured differently. For example, generation of the robot model528can be set to a default setting and need not be generated with the model generation module502. Also for example, the command flag module512can flow to the mode check module518to move to accept the next manual operation command.

Referring now toFIG.6, therein is shown a flow chart of a method600of operation of a robotic system300in an embodiment of the present invention. The method600includes receiving a jog command for manually operating a robotic unit in a block602; real-time parallel processing the jog command including: executing a collision check thread to determine whether the jog command results in a collision or results in an unobstructed status for the robotic unit within an operation environment based on an environment model and a robot model, executing a jog operation thread to determine whether the unobstructed status is provided within a collision check time limit in a block604; and executing the jog command by the robotic unit based on the unobstructed status provided before the collision check time limit in a block606.

The method600further comprise generating the robot model representing the robotic unit with including a buffer zone surrounding the robotic unit; generating the environment model representing the operation environment including the robotic unit; and wherein executing the collision check thread to determine whether the jog command results in the collision includes determining a representation of an item in the environment model entering the buffer zone.

The method600include executing the collision check thread to determine whether the jog command results in the collision includes traversing the buffer zone without a representation of an item included in the environment model for the operation environment, further comprising updating the environment model with the representation of the item.

The method600further comprise resizing the buffer zone based on the environment model representing the operation environment as non-complex.

The method600further comprise generating the robot model of the robotic unit, the robot model including a buffer zone surrounding the robotic unit; wherein modeling the robotic unit includes: modeling a link and a joint, modeling a first buffer for the link and the joint; and executing the collision check thread to determine whether the jog command results in the collision includes determining an item in the environment model enters the first buffer.

The method600further comprise generating the environment model as complex for the operation environment; and reducing a size of the buffer zone based on the environment model being complex.

The method600further comprise terminating the jog command by the robotic unit based on the unobstructed status not being provided before the collision check time limit.

The resulting method, process, apparatus, device, product, and/or system is cost-effective, highly versatile, accurate, sensitive, and effective, and can be implemented by adapting known components for ready, efficient, and economical manufacturing, application, and utilization. Another important aspect of an embodiment of the present invention is that it valuably supports and services the historical trend of reducing costs, simplifying systems, and increasing performance.

These and other valuable aspects of an embodiment of the present invention consequently further the state of the technology to at least the next level.