Patent Description:
In a robotic conveyor system, a conveyor belt is generally controlled by a programmable logic unit where speed of the conveyor belt is fixed for providing smooth flow of an object (e.g., a package, a parcel, a box, a case, a carton, a pallet, etc.) along the conveyor belt. In certain robotic conveyor systems, different portions of a robotic conveyor system can include conveyor belts with different speeds. In other robotic conveyor systems, speed of a conveyor belt may be variable based on position of objects along the conveyor belt. However, conveyor systems are prone to inefficiencies. For example, an abundance of objects on a conveyor belt can result in decreased performance for a conveyor system, such as a jam causing delay of transportation of objects along the conveyor belt.

<NPL>, describes the concept of a reinforcement learning agent, which can deduce the correct control policy of a plant by acting in its digital twin (the HiL simulation).

<NPL>, describes the introduction of PoseCNN for estimating the 6D pose of known objects estimation.

The main embodiment of the invention is defined by the independent claim. Additional embodiments are set out in the dependent claims.

Various embodiments of the present invention now will be described more fully hereinafter with reference to the accompanying drawings, in which some, but not all embodiments of the invention are shown. Indeed, the invention may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will satisfy applicable legal requirements. The term "or" is used herein in both the alternative and conjunctive sense, unless otherwise indicated. The terms "illustrative," "example," and "exemplary" are used to be examples with no indication of quality level.

The phrases "in an embodiment," "in one embodiment," "according to one embodiment," and the like generally mean that the particular feature, structure, or characteristic following the phrase may be included in at least one embodiment of the present disclosure, and may be included in more than one embodiment of the present disclosure (importantly, such phrases do not necessarily refer to the same embodiment).

If the specification states a component or feature "can," "may," "could," "should," "would," "preferably," "possibly," "typically," "optionally," "for example," "often," or "might" (or other such language) be included or have a characteristic, that particular component or feature is not required to be included or to have the characteristic.

In material handling environments (e.g., distribution centers, shipping centers, warehouses, factories, etc.), it is often desirable to transport objects (e.g., packages, parcels, boxes, cases, cartons, pallets, etc.) along a conveyor belt of a conveyor system. A conveyor system is a robotic system that controls conveyor belts for transportation and/or singulation of objects. Generally, a conveyor belt is controlled by a programmable logic unit where speed of the conveyor belt is fixed for providing smooth flow of objects along the conveyor belt. In certain conveyor systems, different portions of a conveyor system can include conveyor belts with different speeds. In other conveyor systems, speed of a conveyor belt may be variable based on position of objects along the conveyor belt. For instance, a speed of a conveyor belt can be variable where positions of objects are determined by data provided by a vision system integrated in a conveyor system.

However, conveyor systems are prone to inefficiencies. With an example conveyor system, numerous objects can be provided to a conveyor belt of a conveyor system via a chute associated with singulation and/or actuators of the conveyor system (e.g., objects can be picked or swept by actuators of a conveyor system). In certain instances, an abundance of objects on the conveyor belt can result in a jam on the conveyor belt (e.g., an excess quantity of objects at a particular location on the conveyor belt) that results in a delay of transportation of the objects along the conveyor belt and/or a delay in unloading the objects from the conveyor belt. Furthermore, in certain instances, accuracy and/or efficiency of a conveyor screening process associated with a vision system of the conveyor system for identifying objects can be reduced as a result of the excess quantity of objects at the particular location on the conveyor belt.

Thus, to address these and/or other issues, reinforcement learning based conveyoring control is disclosed herein. The reinforcement learning based conveyoring control disclosed herein can be employed, for example, to provide an improved conveyor system with improved performance, improved efficiency, improved flow of objects, and/or improved singulation of objects is provided. In an embodiment, control of a speed of a conveyor belt for a conveyor system can be improved via machine learning. For instance, training can by employed to control a conveyor belt and machine learning can be employed to clear a jam associated with a conveyor system. In an aspect, a learned agent that is trained based on one or more machine learning techniques can be implemented to control one or more portions of the conveyor system. For instance, the learned agent can provide one or more control signals determined based on one or more machine learning techniques to control a speed of a conveyor belt for the conveyor system and/or a direction of a conveyor belt for the conveyor system. In certain embodiments, multiple learned agents can be employed during simulation to train with domain randomization in order to minimize differences between real data and simulated data. The domain randomization can enable training in simulation and/or execution during real-time operation of a conveyor system.

In certain embodiments, simulation associated with multiple conveyor systems are employed to train and/or gather data to train one or more reinforcement learning agents for a conveyor system. The training process can also employ domain randomization in certain embodiments to minimize differences between real data and simulated data. In an embodiment, multiple conveyor system can be utilized in parallel (e.g., at approximately the same time) to train two different machine learning models. For instance, different learned agents can be trained in parallel. The two different machine learning models can be dependent on one another. For example, a first machine learning model can learn one or more control policies for optimal belt speed of a conveyor belt using reinforcement learning. Additionally or alternatively, the first machine learning model can learn one or more control policies for jam recovery (e.g., to mitigate a jam condition) associated with a conveyor system. Furthermore, a second machine learning model can employ a convolutional neural network to learn object poses from data captured by a vision system of the conveyor system. The object poses can include, for example, translations and/or rotations for an object.

In another embodiment, the first machine learning model associated with reinforcement learning and the second machine learning model associated with the convolutional neural network can be employed to facilitate control of a conveyor system. In an implementation, a vision system (e.g., a two-dimensional (2D) vision sensor and/or a three-dimensional (3D) vision sensor) of a conveyor system can scan a conveyor belt for one or more objects. In certain embodiments, an imaging device (e.g., an RGB image sensor) of the vision system can capture one or more RGB images associated with the conveyor belt. Data captured by the conveyor system can then be provided to the second machine learning model associated with the convolutional neural network to determine object pose data (e.g., a position and/or an orientation) for the one or more objects. Based on the object pose data, the first machine learning model associated with the reinforcement learning can employ the one or more control policies to determine speed control data related to a belt speed control of the conveyor belt of the conveyor system. In certain embodiments, the speed control data can control one or more actuators of the conveyor belt.

In yet another embodiment, a reinforcement learning process can be employed to learn one or more control policies related to a speed of a conveyor belt for the conveyor system and/or a direction of a conveyor belt for the conveyor system. In certain embodiments, the reinforcement learning process can be employed to initially learn one or more control policies related to conveyoring of a conveyor belt for the conveyor system. Additionally, at a later stage after the one or more control policies related to the conveyoring is learned, the one or more control policies can be employed to clear a jam associated with a conveyor belt of the conveyor system. As such, the reinforcement learning based conveyoring control disclosed herein can employ the reinforcement learning process to evolve to a control policy for jam recovery based on the one or more control policies related to the conveyoring.

In certain embodiments, a simulation-to-real architecture can employ simulation data and/or real data to facilitate the reinforcement learning process. In an aspect, the simulation-to-real architecture can employ domain randomization while training in simulation. In an embodiment, multiple worker systems (e.g., multiple instances of the same simulation of a conveyor system) can be implemented in parallel with one or more aspects of simulation being different between the multiple workers. For example, for different simulations of a conveyor system, color of objects (e.g., boxes) transported via a conveyor belt can be different. In another example, depth noise can be altered (e.g., increased) for different simulations of a conveyor system. In certain embodiments, both simulated data and real data can be employed for a training process for a conveyor system, where an amount of simulated data employed by the training process for a conveyor system is greater than an amount of real data employed by the training process to minimize overfitting to only simulated data.

As such, an improved conveyor system with improved performance, improved efficiency, improved flow of objects, and/or improved singulation of objects can be provided.

<FIG> illustrates a system <NUM> that provides an exemplary environment within which one or more described features of one or more embodiments of the disclosure can be implemented. According to an embodiment, the system <NUM> includes a machine learning conveyor system <NUM> to facilitate a practical application of reinforcement learning based conveyoring control for a conveyor system. In an embodiment, the machine learning conveyor system <NUM> can be a processing device that provides reinforcement learning based conveyoring control for a conveyor system. In a non-limiting embodiment, the machine learning conveyor system <NUM> can be a learned agent device for a conveyor system. The machine learning conveyor system <NUM> can be related to one or more technologies to facilitate reinforcement learning based conveyoring control for a conveyor system. Moreover, the machine learning conveyor system <NUM> can provide an improvement to one or more technologies such as conveyor system technologies, conveyor belt technologies, actuator technologies, robotics technologies, material handling technologies, sortation system technologies, imaging technologies, scanning technologies, digital technologies and/or other technologies. In an implementation, the machine learning conveyor system <NUM> can improve performance of a conveyor system. For example, the machine learning conveyor system <NUM> can provide improved efficiency, improved flow of objects, and/or improved singulation of objects for a conveyor system, as compared to conventional conveyor systems.

The machine learning conveyor system <NUM> can include an object pose estimation component <NUM>, a reinforcement learning component <NUM> and/or a conveyor system control component <NUM>. Additionally, in certain embodiments, the machine learning conveyor system <NUM> can include a processor <NUM> and/or a memory <NUM>. In an alternate embodiment, the conveyor system control component <NUM> can be implemented separate from the machine learning conveyor system <NUM> (e.g., the conveyor system control component <NUM> can be implemented within a conveyoring control device and/or a conveyoring system). In certain embodiments, one or more aspects of the machine learning conveyor system <NUM> (and/or other systems, apparatuses and/or processes disclosed herein) can constitute executable instructions embodied within a computer-readable storage medium (e.g., the memory <NUM>). For instance, in an embodiment, the memory <NUM> can store computer executable component and/or executable instructions (e.g., program instructions). Furthermore, the processor <NUM> can facilitate execution of the computer executable components and/or the executable instructions (e.g., the program instructions). In an example embodiment, the processor <NUM> can be configured to execute instructions stored in the memory <NUM> or otherwise accessible to the processor <NUM>.

The processor <NUM> can be a hardware entity (e.g., physically embodied in circuitry) capable of performing operations according to one or more embodiments of the disclosure. Alternatively, in an embodiment where the processor <NUM> is embodied as an executor of software instructions, the software instructions can configure the processor <NUM> to perform one or more algorithms and/or operations described herein in response to the software instructions being executed. In an embodiment, the processor <NUM> can be a single core processor, a multi-core processor, multiple processors internal to the machine learning conveyor system <NUM>, a remote processor (e.g., a processor implemented on a server), and/or a virtual machine. In certain embodiments, the processor <NUM> be in communication with the memory <NUM>, the object pose estimation component <NUM>, the reinforcement learning component <NUM> and/or the conveyor system control component <NUM> via a bus to, for example, facilitate transmission of data among the processor <NUM>, the memory <NUM>, the object pose estimation component <NUM>, the reinforcement learning component <NUM> and/or the conveyor system control component <NUM>. The processor <NUM> can be embodied in a number of different ways and can, in certain embodiments, include one or more processing devices configured to perform independently. Additionally or alternatively, the processor <NUM> can include one or more processors configured in tandem via a bus to enable independent execution of instructions, pipelining of data, and/or multi-thread execution of instructions. The memory <NUM> can be non-transitory and can include, for example, one or more volatile memories and/or one or more non-volatile memories. In other words, for example, the memory <NUM> can be an electronic storage device (e.g., a computer-readable storage medium). The memory <NUM> can be configured to store information, data, content, one or more applications, one or more instructions, or the like, to enable the machine learning conveyor system <NUM> to carry out various functions in accordance with one or more embodiments disclosed herein. As used herein in this disclosure, the term "component," "system," and the like, can be and/or can include a computer-related entity. For instance, "a component," "a system," and the like disclosed herein can be either hardware, software, or a combination of hardware and software. As an example, a component can be, but is not limited to, a process executed on a processor, a processor, circuitry, an executable component, a thread of instructions, a program, and/or a computer entity.

The machine learning conveyor system <NUM> (e.g., the object pose estimation component <NUM> of the machine learning conveyor system <NUM>) receives sensor data <NUM>. The sensor data <NUM> can include one or more images (e.g., one or more 2D images and/or one or more 3D images) associated with one or more objects. Additionally or alternatively, the sensor data <NUM> can include 3D point cloud data associated with one or more objects. In an embodiment, the sensor data <NUM> can be captured by a vision system that scan one or more conveyor belts and/or one or more conveyor systems. The one or more objects can be one or more physical items, one or more elements, one or more devices, or the like that is transported via a conveyor belt of a conveyor system. For example, the one or more objects can be one or more packages, one or more parcels, one or more boxes, one or more cases, one or more cartons, one or more pallets and/or one or more other objects transported via a conveyor belt of a conveyor system in a material handling environment (e.g., a distribution center, a shipping center, a warehouse, a factory, etc.). In certain embodiments, the one or more objects can be one or more dynamic objects with a location that is not fixed. For example, the one or more objects can be shipped-in, shipped-out, or otherwise moved via a conveyor belt of a conveyor system. An object from the one or more objects can comprise a certain size, a certain shape, a certain color, and/or another physical characteristic. An object from the one or more objects can also comprise a certain position with respect to a conveyor belt and/or a certain orientation with respect to a conveyor belt. For example, an object from the one or more objects can comprise an x-axis position value Tx, a y-axis position value Ty, a z-axis position value Tz and/or a rotation value Rz.

In an embodiment, the sensor data <NUM> can be imaging data that includes a plurality of pixels for the one or more images. For example, each pixel in the plurality of pixels can represent a point in a scene related to an object. In an aspect, each pixel in the plurality of pixels can include color data, intensity data and/or depth data. The color data can be represented in the form of one or more color schemes such as, but not limited to, a RGB color scheme, a CMYK color scheme, a monochrome color scheme, a grayscale color scheme, and/or the another type of color scheme. The intensity data can be representative of a brightness associated with each pixel. The depth data can correspond to a distance of a respective point, represented by a respective pixel, from the vision system that captures the one or more images. In an example embodiment, the one or more images of the sensor data <NUM> can be encoded and/or represented in one or more formats such as JPEG, Bitmap, PNG, RAW, and/or another type of format.

The object pose estimation component <NUM> can determine object pose data for the one or more objects related to the sensor data <NUM>. In an embodiment, the object pose estimation component <NUM> can employ a machine learning model to determine the object pose data based on the one or more images included in the sensor data <NUM>. In an aspect, the machine learning model employed by the object pose estimation component <NUM> can be a machine learning model that is trained for object pose estimation. In another aspect, the one or more images associated with the sensor data <NUM> can be provided as input to the machine learning model associated with object pose estimation. In an embodiment, the machine learning model can be a convolutional neural network that is trained to identify object poses from images. For instance, in an embodiment, the convolutional neural network can be a deep neural network that is trained to analyze visual imagery based on a shared-weights architecture and/or translation invariance characteristics between a series of convolutional layers, one or more pooling layers, one or more fully connected layers and/or one or more normalization layers.

In an embodiment, the object pose data can include position data for the one or more objects related to the sensor data <NUM>. For example, the object pose estimation component <NUM> can employ the machine learning model associated with object pose estimation to determine the position data (e.g., the position data for the one or more objects related to the sensor data <NUM>) based on the one or more images included in the sensor data <NUM>. The position data can be, for example, translation data that includes an x-axis position value Tx, a y-axis position value Ty, and/or a z-axis position value Tz for each object from the one or more objects related to the sensor data <NUM>.

In another embodiment, the object pose data can additionally or alternatively include orientation data for the one or more objects related to the sensor data <NUM>. For example, the object pose estimation component <NUM> can employ the machine learning model associated with object pose estimation to determine the orientation data (e.g., the orientation data for the one or more objects related to the sensor data <NUM>) based on the one or more images included in the sensor data <NUM>. The orientation data can, for example, an orientation value Rz for each object from the one or more objects related to the sensor data <NUM>.

The reinforcement learning component <NUM> can employ a set of control policies to generate speed control data <NUM> for the conveyor belt of the conveyor system based on the object pose data. For example, the reinforcement learning component <NUM> can employ a set of control policies to generate the speed control data <NUM> for the conveyor belt of the conveyor system based on the position data and/or the orientation data. The speed control data <NUM> include a value to increase or decrease a current speed of the conveyor belt of the conveyor system. For example, the speed control data <NUM> can include a certain positive value to increase a speed of the conveyor belt of the conveyor system by a certain amount. In another example, the speed control data <NUM> can include a certain negative value to decrease a speed of the conveyor belt of the conveyor system by a certain amount. Additionally or alternatively, in certain embodiments, the speed control data <NUM> can include a value to control a direction of the conveyor belt of the conveyor system. For example, the speed control data <NUM> can include a certain value (e.g., a first binary value) to control direction of the conveyor belt of the conveyor system in a forward direction. In another example, the speed control data <NUM> can include another value (e.g., a second binary value) to control direction of the conveyor belt of the conveyor system in a backward direction. The set of control policies can be one or more control policies generated for the conveyor belt of the conveyor system based on reinforcement learning. The set of control policies can determine speed and/or direction for control of the conveyor belt of the conveyor system. For example, the set of control policies can be one or more control policies to provide an optimal belt speed for the conveyor belt of the conveyor system. The set of control policies can include one or more rules and/or one or more actions to facilitate an optimal belt speed for the conveyor belt of the conveyor system and/or mitigation of a jam condition associated with the conveyor belt of the conveyor system. The one or more rules and/or the one or more actions can be related to movement of the one or more objects from a certain area of the conveyor belt and/or the conveyor belt system. Additionally or alternatively, the one or more rules and/or the one or more actions can be related to a certain degree of forward movement or a certain degree of backward movement for the one or more objects with respect to the conveyor belt and/or the conveyor belt system. Additionally or alternatively, the one or more rules and/or the one or more actions can be related to a path of motion for the one or more objects with respect to the conveyor belt and/or the conveyor belt system. Furthermore, a machine learning model trained via a reinforcement learning process can generate the set of control policies. The reinforcement learning process can be related to simulation associated with a plurality of conveyor systems in order to determine an optimal belt speed for the conveyor belt of the conveyor system. In an embodiment, the reinforcement learning component <NUM> can employ the machine learning model to determine the speed control data <NUM> for the conveyor belt of the conveyor system based on one or more control policies from the set of control policies that are related to the object pose data. The speed control data <NUM> can be, for example, one or more belt speeds for the conveyor belt of the conveyor system.

The conveyor system control component <NUM> can generate one or more control signals for the conveyor system based on the speed control data <NUM>. For example, the conveyor system control component <NUM> can generate one or more control signals for the conveyor belt of the conveyor system based on the speed control data <NUM>. In certain embodiments, the conveyor system control component <NUM> can generate one or more control signals for one or more actuators of the conveyor system based on the speed control data <NUM>. In certain embodiments, the conveyor system control component <NUM> can modify one or more settings of the conveyor belt of the conveyor system based on the speed control data <NUM>. The conveyor system control component <NUM> can generate the one or more control signals for the conveyor system to facilitate, for example jam recovery and/or improved flow of objects associated with the conveyor belt of the conveyor system. As such, the machine learning conveyor system <NUM> can provide the speed control data <NUM> and/or the one or more control signals associated with the speed control data to the conveyor system to improve performance of the conveyor system, to improve efficiency of the conveyor system, to improve flow of objects transported via the conveyor belt of the conveyor system, and/or to improve singulation of objects transported via the conveyor belt of the conveyor system.

<FIG> illustrates a system <NUM> that provides an exemplary environment within which one or more described features of one or more embodiments of the disclosure can be implemented. According to an embodiment, the system <NUM> includes a machine learning training system <NUM> to facilitate a practical application of training one or more machine learning models for reinforcement learning based conveyoring control of a conveyor system. The machine learning training system <NUM> can be related to one or more technologies to facilitate reinforcement learning based conveyoring control for a conveyor system. Moreover, the machine learning training system <NUM> can provide an improvement to one or more technologies such as conveyor system technologies, conveyor belt technologies, actuator technologies, robotics technologies, material handling technologies, sortation system technologies, imaging technologies, scanning technologies, digital technologies, machine learning technologies, and/or other technologies. In an implementation, the machine learning training system <NUM> can facilitate generation of one or more improved machine learning models for reinforcement learning based conveyoring control of a conveyor system.

The machine learning training system <NUM> can include an object pose estimation training component <NUM> and/or a reinforcement learning training component <NUM>. Additionally, in certain embodiments, the machine learning training system <NUM> can include a processor <NUM> and/or a memory <NUM>. In certain embodiments, one or more aspects of the machine learning training system <NUM> (and/or other systems, apparatuses and/or processes disclosed herein) can constitute executable instructions embodied within a computer-readable storage medium (e.g., the memory <NUM>). For instance, in an embodiment, the memory <NUM> can store computer executable component and/or executable instructions (e.g., program instructions). Furthermore, the processor <NUM> can facilitate execution of the computer executable components and/or the executable instructions (e.g., the program instructions). In an example embodiment, the processor <NUM> can be configured to execute instructions stored in the memory <NUM> or otherwise accessible to the processor <NUM>.

The processor <NUM> can be a hardware entity (e.g., physically embodied in circuitry) capable of performing operations according to one or more embodiments of the disclosure. Alternatively, in an embodiment where the processor <NUM> is embodied as an executor of software instructions, the software instructions can configure the processor <NUM> to perform one or more algorithms and/or operations described herein in response to the software instructions being executed. In an embodiment, the processor <NUM> can be a single core processor, a multi-core processor, multiple processors internal to the machine learning training system <NUM>, a remote processor (e.g., a processor implemented on a server), and/or a virtual machine. In certain embodiments, the processor <NUM> be in communication with the memory <NUM>, the object pose estimation training component <NUM>, and/or the reinforcement learning training component <NUM> via a bus to, for example, facilitate transmission of data among the processor <NUM>, the memory <NUM>, the object pose estimation training component <NUM>, and/or the reinforcement learning training component <NUM>. The processor <NUM> can be embodied in a number of different ways and can, in certain embodiments, include one or more processing devices configured to perform independently. Additionally or alternatively, the processor <NUM> can include one or more processors configured in tandem via a bus to enable independent execution of instructions, pipelining of data, and/or multi-thread execution of instructions. The memory <NUM> can be non-transitory and can include, for example, one or more volatile memories and/or one or more non-volatile memories. In other words, for example, the memory <NUM> can be an electronic storage device (e.g., a computer-readable storage medium). The memory <NUM> can be configured to store information, data, content, one or more applications, one or more instructions, or the like, to enable the machine learning training system <NUM> to carry out various functions in accordance with one or more embodiments disclosed herein. As used herein in this disclosure, the term "component," "system," and the like, can be and/or can include a computer-related entity. For instance, "a component," "a system," and the like disclosed herein can be either hardware, software, or a combination of hardware and software. As an example, a component can be, but is not limited to, a process executed on a processor, a processor, circuitry, an executable component, a thread of instructions, a program, and/or a computer entity.

The machine learning training system <NUM> (e.g., the object pose estimation training component <NUM> of the machine learning training system <NUM>) can receive training data <NUM>. The training data <NUM> can include one or more images (e.g., one or more 2D images and/or one or more 3D images) associated with one or more objects to facilitate training of a machine learning model for object pose estimation. In an embodiment, the training data <NUM> can be captured by a vision system that scan one or more conveyor belts and/or one or more conveyor systems. The one or more objects can be one or more physical items, one or more elements, one or more devices, or the like that is transported via a conveyor belt of a conveyor system. For example, the one or more objects can be one or more packages, one or more parcels, one or more boxes, one or more cases, one or more cartons, one or more pallets and/or one or more other objects transported via a conveyor belt of a conveyor system in a material handling environment (e.g., a distribution center, a shipping center, a warehouse, a factory, etc.). In certain embodiments, the one or more objects can be one or more dynamic objects with a location that is not fixed. For example, the one or more objects can be shipped-in, shipped-out, or otherwise moved via a conveyor belt of a conveyor system. An object from the one or more objects can comprise a certain size, a certain shape, a certain color, and/or another physical characteristic. An object from the one or more objects can also comprise a certain position with respect to a conveyor belt and/or a certain orientation with respect to a conveyor belt. For example, an object from the one or more objects can comprise an x-axis position value Tx, a y-axis position value Ty, a z-axis position value Tz and/or a rotation value Rz.

In an aspect, the training data <NUM> can be imaging data that includes a plurality of pixels for the one or more images. For example, each pixel in the plurality of pixels can represent a point in a scene related to an object. In an aspect, each pixel in the plurality of pixels can include color data, intensity data and/or depth data. The color data can be represented in the form of one or more color schemes such as, but not limited to, a RGB color scheme, a CMYK color scheme, a monochrome color scheme, a grayscale color scheme, and/or the another type of color scheme. The intensity data can be representative of a brightness associated with each pixel. The depth data can correspond to a distance of a respective point, represented by a respective pixel, from the vision system that captures the one or more images. In an example embodiment, the one or more images of the training data <NUM> can be encoded and/or represented in one or more formats such as JPEG, Bitmap, PNG, RAW, and/or another type of format.

The object pose estimation training component <NUM> can train a machine learning model associated with object pose estimation based on the training data <NUM>. For instance, the machine learning model trained by the object pose estimation training component <NUM> can be a machine learning model that is trained for object pose estimation. In an aspect, sensor data (e.g., the one or more images) associated with the training data <NUM> can be provided as input to the machine learning model associated with object pose estimation. In an embodiment, the machine learning model can be a convolutional neural network that is trained based on the training data <NUM> collected from simulation to identify object poses. For instance, in an embodiment, the convolutional neural network can be a deep neural network that is trained based on the training data <NUM> to analyze visual imagery based on a shared-weights architecture and/or translation invariance characteristics between a series of convolutional layers, one or more pooling layers, one or more fully connected layers and/or one or more normalization layers. In certain embodiments, the object pose estimation training component <NUM> can employ the training data <NUM> to modify one or more weights and/or one or more parameters for one or more convolutional layers of the machine learning model associated with object pose estimation.

The reinforcement learning training component <NUM> can train a model (e.g., a machine learning model) to learn a set of control policies for optimal speed control of a conveyor belt based on object pose data. For example, the reinforcement learning training component <NUM> can train a model (e.g., a machine learning model) to learn a set of control policies for optimal speed control of a conveyor belt based on position data and/or orientation data for objects associated with the training data <NUM>. The set of control policies can be one or more control policies generated for the conveyor belt of the conveyor system based on reinforcement learning. For example, the set of control policies can be one or more control policies to provide an optimal belt speed for the conveyor belt of the conveyor system. In certain embodiments, the reinforcement learning training component <NUM> can employ a reinforcement learning process related to simulation associated with a plurality of conveyor systems in order to determine the set of control policies for optimal speed control of a conveyor belt.

<FIG> illustrates a system <NUM> that provides an exemplary environment within which one or more of the described features of one or more embodiments of the disclosure can be implemented. The system <NUM> includes a conveyoring control device <NUM> that determines speed control data (e.g., the speed control data <NUM>) for a conveyor system <NUM>. In an embodiment, the conveyoring control device <NUM> includes the machine learning conveyor system <NUM> and/or a vision system <NUM>. Additionally, in an embodiment, the conveyor system <NUM> includes a conveyor belt <NUM> and one or more objects <NUM><NUM>-N, where N is an integer. The conveyor belt <NUM> can be a mechanism that transports, directs and/or routs the one or more objects <NUM><NUM>-N through the conveyor system <NUM>. The one or more objects <NUM><NUM>-N can be, for example, one or more packages, one or more parcels, one or more boxes, one or more cases, one or more cartons, one or more pallets and/or one or more other objects. In an aspect, the conveyor belt <NUM> can be associated with a receiving lane of the conveyor system <NUM>, an accumulation buffering lane of the conveyor system <NUM>, a shipping lane of the conveyor system <NUM>, or another lane of the conveyor system <NUM>. In one embodiment, the conveyor belt <NUM> can be associated with a constant elevation. In another embodiment, the conveyor belt can be associated with a variable elevation (e.g., one or more elevation changes). The conveyor system <NUM> can be, for example, a case conveyor, a tote conveyor, a polybag conveyor, a transportation conveyor, a pallet conveyor, an accumulation conveyor, a vertical indexing conveyor, or another type of conveyor system. In certain embodiments, at least a portion of the conveyor system <NUM> can be a sortation system. For example, in certain embodiments, the conveyor system <NUM> can be a sweeper sorter, a strip-belt sorter, a crossbelt sorter, a tilt-tray sorter, a push-tray sorter, a sliding shoe sorter, a popup wheel sorter, a vertical sortation sorter, or another type of sorter system. In an embodiment, the conveyor system <NUM> can additionally include an actuator <NUM>. The actuator <NUM> can be a device that converts rotary motion into linear motion for the conveyor belt <NUM>. In one embodiment, the actuator <NUM> can be an electric linear actuator that employs a motor to control speed of the conveyor belt <NUM>.

In an embodiment, the vision system <NUM> can scan the conveyor system <NUM> to generate the sensor data <NUM>. For example, the vision system <NUM> can include one or more sensors configured to scan the conveyor belt <NUM> to generate one or more images associated with the one or more objects <NUM><NUM>-N. In an embodiment, the vision system <NUM> can include one or more imaging devices (e.g., one or more image capturing devices) such as one or more cameras (e.g., one or more camera units, one or more 2D cameras, one or more 3D cameras, etc.). For example, the vision system <NUM> can include one or more image sensors (e.g., one or more CMOS sensors, one or more CCD sensors, etc.) to facilitate generation of one or more images related to the one or more objects <NUM><NUM>-N. In certain embodiments, the vision system <NUM> can additionally or alternatively generate 3D point cloud data related to the one or more objects <NUM><NUM>-N. In certain embodiments, the vision system <NUM> can include an embedded processor (e.g., an embedded processor that is different than the processor <NUM> of the machine learning conveyor system <NUM>) configured to control the vision system <NUM>.

The machine learning conveyor system <NUM> receives the sensor data <NUM> generated by the vision system <NUM>. Furthermore, the machine learning conveyor system <NUM> (e.g., the object pose estimation component <NUM>, the reinforcement learning component <NUM> and/or the conveyor system control component <NUM>) employs the sensor data <NUM> to perform object pose estimation, reinforcement learning and/or conveyor system control for the conveyor belt <NUM> of the conveyor system <NUM>. For example, based on the sensor data <NUM> generated by the vision system <NUM>, the machine learning conveyor system <NUM> (e.g., the object pose estimation component <NUM>, the reinforcement learning component <NUM> and/or the conveyor system control component <NUM>) can generate the speed control data <NUM> for the conveyor belt <NUM> of the conveyor system <NUM>. In certain embodiments, the speed control data <NUM> generated by the machine learning conveyor system <NUM> and/or one or more control signal associated with the speed control data <NUM> can be provided to the actuator <NUM> of the conveyor system <NUM> to control a speed of the conveyor belt <NUM> of the conveyor system <NUM>. As such, conveyor system <NUM> can be provided with improved performance, improved efficiency, improved flow of the one or more objects <NUM><NUM>-N, and/or improved singulation of the one or more objects <NUM><NUM>-N.

In an alternate embodiment, at least a portion of the machine learning conveyor system <NUM> can be implemented on a server system <NUM>. For example, in certain embodiments, the vision system <NUM> can transmit the sensor data <NUM> (e.g., processed sensor data) to at least a portion of the machine learning conveyor system <NUM> implemented on the server system <NUM> via a network <NUM>. The network <NUM> can be a communications network that employs wireless technologies and/or wired technologies to transmit data between the vision system <NUM> and the server system <NUM>. For example, the network <NUM> can be a Wi-Fi network, a Near Field Communications (NFC) network, a Worldwide Interoperability for Microwave Access (WiMAX) network, a personal area network (PAN), a short-range wireless network (e.g., a Bluetooth® network), an infrared wireless (e.g., IrDA) network, an ultra-wideband (UWB) network, an induction wireless transmission network, and/or another type of network.

<FIG> illustrates a system <NUM> in accordance with one or more embodiments of the disclosure. The system <NUM> includes the vision system <NUM> and a plurality of conveyor systems <NUM><NUM>-M, where M is an integer. In an embodiment, the vision system <NUM> can scan plurality of conveyor systems <NUM><NUM>-M. For example, the vision system <NUM> can scan one or more first objects provided by a first conveyor belt of the conveyor system <NUM><NUM>, one or more second objects provided by a second conveyor belt of the conveyor system <NUM><NUM>, etc. In another embodiment, the machine learning conveyor system <NUM> can determine first object pose data for the one or more first objects provided by the first conveyor belt of the conveyor system <NUM><NUM>, second object pose data for the one or more second objects provided by the second conveyor belt of the conveyor system <NUM><NUM>, etc. For instance, in an embodiment, the vision system <NUM> can generate first image sensor data (e.g., one or more first images) for the one or more first objects provided by the first conveyor belt of the conveyor system <NUM><NUM>, second image sensor data (e.g., one or more second images) for the one or more second objects provided by the second conveyor belt of the conveyor system <NUM><NUM>, etc. Furthermore, the machine learning conveyor system <NUM> can perform respective object pose estimation, reinforcement learning, and/or conveyor system control for the plurality of conveyor systems <NUM><NUM>-M.

<FIG> illustrates a system <NUM> in accordance with one or more embodiments of the disclosure. The system <NUM> includes the vision system <NUM>, a convolutional neural network <NUM>, a control policy engine <NUM>, and/or a conveyor belt actuation engine <NUM>. In an embodiment, the vision system <NUM> provides the sensor data <NUM> as input to the convolutional neural network <NUM>. In one example, the sensor data <NUM> can be formatted as one or more RGB images. In another example, the sensor data <NUM> can be formatted as 3D point cloud data. However, it is to be appreciated that the sensor data <NUM> can be a different type of imaging data. The convolutional neural network <NUM> can be trained for object pose estimation. For example, the convolutional neural network <NUM> can be trained to identify object poses from the sensor data <NUM>. The convolutional neural network <NUM> can be a deep neural network that includes a series of convolutional layers, one or more pooling layers, one or more fully connected layers and/or one or more normalization layers to facilitate object pose estimation. In an aspect, the convolutional neural network <NUM> can determine one or more classifications, one or more correlations, one or more inferences, one or more patterns, one or more features and/or other information to facilitate object pose estimation and generation of object pose data <NUM> related to the sensor data <NUM>.

In certain embodiments, processing by the convolutional neural network <NUM> can be associated with image recognition, image analysis, 3D point clouds, and/or computer vision to facilitate object pose estimation. In an aspect, the convolutional neural network <NUM> can determine position data and/or orientation data for one or more objects included in the sensor data <NUM> based on a coordinate system associated with x-axis coordinates, y-axis coordinates, and/or a z-axis coordinates of respective points in a scene of the conveyor system associated with the sensor data <NUM>. For example, the convolutional neural network <NUM> can employ a mapping of two-dimensional features or three-dimensional features in a coordinate system to determine position data and/or orientation data for one or more objects included in the sensor data <NUM>. In certain embodiments, the convolutional neural network <NUM> can employ one or more object segmentation mask to identify one or more geometric features of the one or more objects included in the sensor data <NUM>. The geometric features can include, for example, corners of an object, edges of an object, portions of an object, interest points of an object, regions of interest points of an object, and/or another type of geometric feature of an object.

In an embodiment, the convolutional neural network <NUM> can generate object pose data <NUM> based on the sensor data <NUM>. The object pose data <NUM> can include position data for the one or more objects related to the sensor data <NUM>. For example, in an embodiment, the convolutional neural network <NUM> can determine the position data (e.g., the position data for the one or more objects related to the sensor data <NUM>) based on one or more images included in the sensor data <NUM>. In another embodiment, the convolutional neural network <NUM> can determine the position data (e.g., the position data for the one or more objects related to the sensor data <NUM>) based on 3D point cloud data included in the sensor data <NUM>. The position data included in the object pose data <NUM> can include an x-axis position value Tx, a y-axis position value Ty, and/or a z-axis position value Tz with respect to the coordinate system for each object from the one or more objects related to the sensor data <NUM>. In another embodiment, the object pose data <NUM> can additionally or alternatively include orientation data for the one or more objects related to the sensor data <NUM>. For example, in an embodiment, the convolutional neural network <NUM> can determine the orientation data (e.g., the orientation data for the one or more objects related to the sensor data <NUM>) based on one or more images included in the sensor data <NUM>. In another embodiment, the convolutional neural network <NUM> can determine the orientation data (e.g., the orientation data for the one or more objects related to the sensor data <NUM>) based on 3D point cloud data included in the sensor data <NUM>. The orientation data included in the object pose data <NUM> an orientation value Rz with respect to the coordinate system for each object from the one or more objects related to the sensor data <NUM>.

The control policy engine <NUM> can determine the speed control data <NUM> based on the object pose data <NUM>. In an embodiment, the control policy engine <NUM> can employ the set of control policies to select the speed control data <NUM> based on the object pose data <NUM>. The set of control policies can be one or more policies to control the conveyor belt (e.g., control actuation of the conveyor belt) for optimal speed control of the conveyor belt. For instance, the control policy engine <NUM> can employ the object pose data <NUM> and the set of control policies to determine the speed control data <NUM> for the conveyor belt (e.g., an actuator that controls the conveyor belt). In one example, a control policy can be an action and/or a rule to facilitate an optimal belt speed for the conveyor belt of the conveyor system and/or mitigation of a jam condition associated with the conveyor belt of the conveyor system. Additionally or alternatively, a control policy can be related to movement of the one or more objects from a certain area of the conveyor belt and/or the conveyor belt system. Additionally or alternatively, a control policy can be related to a certain degree of forward movement or a certain degree of backward movement for the one or more objects with respect to the conveyor belt and/or the conveyor belt system. Additionally or alternatively, a control policy can be to a path of motion for the one or more objects with respect to the conveyor belt and/or the conveyor belt system.

The conveyor belt actuation engine <NUM> can apply the speed control data <NUM> to the conveyor belt of the conveyor system. For example, the conveyor belt actuation engine <NUM> can apply the speed control data <NUM> to an actuator that controls the conveyor belt of the conveyor system. In an embodiment, the conveyor belt actuation engine <NUM> can provide the speed control data <NUM> to the actuator via one or more control signals associated with the speed control data <NUM>. In certain embodiments, the one or more control signals can include a value to increase or decrease a current speed of the conveyor belt of the conveyor system. For example, the one or more control signals can include a certain positive value to increase a speed of the conveyor belt of the conveyor system by a certain amount. In another example, the one or more control signals can include a certain negative value to decrease a speed of the conveyor belt of the conveyor system by a certain amount. Additionally or alternatively, the one or more control signals can include a value to control a direction of the conveyor belt of the conveyor system. For example, the one or more control signals can include a certain a certain value (e.g., a first binary value) to control direction of the conveyor belt of the conveyor system in a forward direction. In another example, the one or more control signals can include another value (e.g., a second binary value) to control direction of the conveyor belt of the conveyor system in a backward direction.

<FIG> illustrates a system <NUM> in accordance with one or more embodiments of the disclosure. The system <NUM> includes a process <NUM> for machine learning associated with object pose estimation. The system <NUM> also includes a process <NUM> for machine learning associated with reinforcement learning. In an embodiment, the process <NUM> can be a process performed by the object pose estimation component <NUM> and/or the convolutional neural network <NUM>. Furthermore, the process <NUM> can be a process performed by the reinforcement learning component <NUM> and/or the control policy engine <NUM>. The process <NUM> can perform the machine learning associated with object pose estimation based on the sensor data <NUM>. In certain embodiments, the process <NUM> can employ image recognition associated with machine learning, image analysis associated with machine learning, and/or computer vision associated with machine learning to facilitate object pose estimation. In an aspect, the process <NUM> can perform the machine learning associated with object pose estimation to determine one or more classifications, one or more correlations, one or more inferences, one or more patterns, one or more features and/or other information related to geometric features of the one or more objects included in the sensor data <NUM>. For example, the process <NUM> can perform the machine learning associated with object pose estimation to determine corners of the one or more objects included in the sensor data <NUM>, edges of the one or more objects included in the sensor data <NUM>, portions of the one or more objects included in the sensor data <NUM>, interest points of the one or more objects included in the sensor data <NUM>, regions of interest points of the one or more objects included in the sensor data <NUM>, and/or another type of geometric feature of the one or more objects included in the sensor data <NUM>. In another aspect, the process <NUM> can perform the machine learning associated with object pose estimation to determine the object pose data <NUM> based on the geometric features of the one or more objects included in the sensor data <NUM>.

The process <NUM> can perform the machine learning associated with the reinforcement learning based on the object pose data <NUM> and/or control policy data <NUM>. The control policy data <NUM> can include a set of control policies to provide an optimal belt speed associated with the speed control data <NUM>. The control policy data <NUM> can be one or more control policies generated for the conveyor belt of the conveyor system based on the machine learning associated with reinforcement learning. For example, the control policy data <NUM> can include one or more rules and/or one or more actions to facilitate an optimal belt speed for the conveyor belt of the conveyor system and/or mitigation of a jam condition associated with the conveyor belt of the conveyor system. The one or more rules and/or the one or more actions included in the control policy data <NUM> can be related to movement of the one or more objects from a certain area of the conveyor belt and/or the conveyor belt system. Additionally or alternatively, the one or more rules and/or the one or more actions included in the control policy data <NUM> can be related to a certain degree of forward movement or a certain degree of backward movement for the one or more objects with respect to the conveyor belt and/or the conveyor belt system. Additionally or alternatively, the one or more rules and/or the one or more actions included in the control policy data <NUM> can be related to a path of motion for the one or more objects with respect to the conveyor belt and/or the conveyor belt system.

<FIG> illustrates a system <NUM> in accordance with one or more embodiments of the disclosure. The system <NUM> includes one or more training processes for training one or more machine learning models for object pose estimation and/or reinforcement learning. The system <NUM> includes one or more conveyor system simulations <NUM><NUM>-S, where S is an integer. In an embodiment, the one or more conveyor system simulations <NUM><NUM>-S provide the training data <NUM>. The training data <NUM> can be, for example, simulated data provided by the one or more conveyor system simulations <NUM><NUM>-S configured in randomized environments. For example, the conveyor system simulation <NUM><NUM> can transport one or more first objects with a first color via a conveyor belt of the conveyor system simulation <NUM><NUM>, the conveyor system simulation <NUM><NUM> can transport one or more second objects with a second color via a conveyor belt of the conveyor system simulation <NUM><NUM>, etc. Additionally or alternatively, a vision system of the conveyor system simulation <NUM><NUM> can be associated with a first depth noise, a vision system of the conveyor system simulation <NUM><NUM> can be associated with a second depth noise, etc. Additionally or alternatively, the conveyor system simulation <NUM><NUM> can include one or more first settings for a conveyor belt of the conveyor system simulation <NUM><NUM>, the conveyor system simulation <NUM><NUM> can include one or more second settings for a conveyor belt of the conveyor system simulation <NUM><NUM>, etc. In certain embodiments, both simulated data and real data can be employed by the one or more conveyor system simulations <NUM><NUM>-S, where an amount of simulated data employed by the one or more conveyor system simulations <NUM><NUM>-S is greater than an amount of real data employed by the one or more conveyor system simulations <NUM><NUM>-S to, for example, minimize overfitting to only simulated data. In certain embodiments, a simulation-to real-architecture associated with the one or more conveyor system simulations <NUM><NUM>-S can employ domain randomization while training to, for example, minimize a gap between real data and simulated data.

In another embodiment, object pose estimation training <NUM> and reinforcement learning training <NUM> can be performed based on the training data <NUM>. The object pose estimation training <NUM> can be associated with the object pose estimation training component <NUM> and the reinforcement learning training <NUM> can be associated with the reinforcement learning training component <NUM>, in an embodiment. In certain embodiments, the object pose estimation training <NUM> can be performed, for example, in parallel to the reinforcement learning training <NUM>. The object pose estimation training <NUM> can train, for example, a convolutional neural network (e.g., the convolutional neural network <NUM>) for object pose estimation based on the training data <NUM>. Furthermore, the reinforcement learning training <NUM> can determine one or more control policies and/or can train a machine learning model associated with reinforcement learning to actuate a conveyor belt based on the training data <NUM>.

<FIG> illustrates a computer-implemented method <NUM> for facilitating reinforcement learning based conveyoring control in accordance with one or more embodiments described herein. The computer-implemented method <NUM> can be associated with the machine learning conveyor system <NUM>, for example. In one or more embodiments, the computer-implemented method <NUM> begins with receiving, by a device comprising a processor (e.g., by the object pose estimation component <NUM>), sensor data associated with one or more objects transported via a conveyor belt of a conveyor system (block <NUM>). The sensor data can include one or more images (e.g., one or more 2D images and/or one or more 3D images) associated with the one or more objects. Additionally or alternatively, the sensor data can include 3D point cloud data associated with the one or more objects. In an embodiment, the sensor data can be imaging data that includes a plurality of pixels for the one or more images. For example, each pixel in the plurality of pixels can represent a point in a scene related to an object. In an embodiment, the one or more images of the sensor data can be one or more RGB images. The one or more objects can be one or more physical items, one or more elements, one or more devices, or the like that is transported via a conveyor belt of a conveyor system. For example, the one or more objects can be one or more packages, one or more parcels, one or more boxes, one or more cases, one or more cartons, one or more pallets and/or one or more other objects transported via a conveyor belt of a conveyor system in a material handling environment (e.g., a distribution center, a shipping center, a warehouse, a factory, etc.).

The computer-implemented method <NUM> further includes determining, by the device (e.g., by the object pose estimation component <NUM>), object pose data associated with the one or more objects by employing a machine learning model that infers the object pose data based on the sensor data (block <NUM>). In an embodiment, the machine learning model can be a convolutional neural network that infers the object pose data based on the sensor data. The convolutional neural network can be a deep neural network that is trained to analyze visual imagery for object pose estimation based on a shared-weights architecture and/or translation invariance characteristics between a series of convolutional layers, one or more pooling layers, one or more fully connected layers and/or one or more normalization layers. In another embodiment, the determining the object pose data comprises determining position data associated with the one or more objects based on the sensor data. The position data can include an x-axis position value, a y-axis position value, and/or a z-axis position value for each object from the one or more objects related to the sensor data. Additionally or alternatively, the determining the object pose data comprises determining orientation data associated with the one or more objects based on the sensor data. The orientation data can, for example, an orientation value for each object from the one or more objects related to the sensor data.

Furthermore, the computer-implemented method <NUM> includes generating, by the device (e.g., by the reinforcement learning component <NUM>), speed control data for the conveyor belt of the conveyor system based on a set of control policies associated with the object pose data (block <NUM>). The set of control policies can be one or more control policies generated for the conveyor belt of the conveyor system based on reinforcement learning. For example, the set of control policies can be one or more control policies to provide an optimal belt speed for the conveyor belt of the conveyor system. In an embodiment, the set of control policies can include one or more rules and/or one or more actions to facilitate an optimal belt speed for the conveyor belt of the conveyor system and/or mitigation of a jam condition associated with the conveyor belt of the conveyor system. The one or more rules and/or the one or more actions included in the set of control policies can be related to movement of the one or more objects from a certain area of the conveyor belt and/or the conveyor belt system. Additionally or alternatively, the one or more rules and/or the one or more actions included in the set of control policies can be related to a certain degree of forward movement or a certain degree of backward movement for the one or more objects with respect to the conveyor belt and/or the conveyor belt system. Additionally or alternatively, the one or more rules and/or the one or more actions included in the set of control policies can be related to a path of motion for the one or more objects with respect to the conveyor belt and/or the conveyor belt system. In certain embodiments, the speed control data can include a value to increase or decrease a current speed of the conveyor belt of the conveyor system. For example, the speed control data can include a certain positive value to increase a speed of the conveyor belt of the conveyor system by a certain amount. In another example, the speed control data can include a certain negative value to decrease a speed of the conveyor belt of the conveyor system by a certain amount. Additionally or alternatively, in certain embodiments, the speed control data can include a value to control a direction of the conveyor belt of the conveyor system. For example, the speed control data can include a certain value (e.g., a first binary value) to control direction of the conveyor belt of the conveyor system in a forward direction. In another example, the speed control data can include another value (e.g., a second binary value) to control direction of the conveyor belt of the conveyor system in a backward direction.

In certain embodiments, the computer-implemented method <NUM> further includes receiving, by the device (e.g., by the object pose estimation component <NUM>), the sensor data from a vision system that scans the conveyor system. In certain embodiments, the machine learning model is a first machine learning model and the computer-implemented method <NUM> further includes generating, by the device (e.g., by the reinforcement learning component <NUM>), the set of control policies based on a second machine learning model associated with reinforcement learning related to a plurality of conveyor systems. In certain embodiments, the computer-implemented method <NUM> further includes training, by the device (e.g., by the object pose estimation training component <NUM> and/or the reinforcement learning training component <NUM>), the first machine learning model and/or the second machine learning model based on simulated data associated with the plurality of conveyor systems. In certain embodiments, the computer-implemented method <NUM> further includes providing, by the device (e.g., by the conveyor system control component <NUM>), a control signal associated with the speed control data to an actuator of the conveyor system.

The hardware used to implement the various illustrative logics, logical blocks, modules, and circuits described in connection with the aspects disclosed herein may include a general purpose processor, a digital signal processor (DSP), a special-purpose processor such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA), a programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but, in the alternative, the processor may be any processor, controller, microcontroller, or state machine. Alternatively, or in addition, some steps or methods may be performed by circuitry that is specific to a given function.

In one or more example embodiments, the functions described herein may be implemented by special-purpose hardware or a combination of hardware programmed by firmware or other software. In implementations relying on firmware or other software, the functions may be performed as a result of execution of one or more instructions stored on one or more non-transitory computer-readable media and/or one or more non-transitory processor-readable media. These instructions may be embodied by one or more processorexecutable software modules that reside on the one or more non-transitory computer-readable or processor-readable storage media. Non-transitory computer-readable or processor-readable storage media may in this regard comprise any storage media that may be accessed by a computer or a processor. By way of example but not limitation, such non-transitory computer-readable or processor-readable media may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), FLASH memory, disk storage, magnetic storage devices, or the like. Disk storage, as used herein, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc™, or other storage devices that store data magnetically or optically with lasers. Combinations of the above types of media are also included within the scope of the terms non-transitory computer-readable and processor-readable media. Additionally, any combination of instructions stored on the one or more non-transitory processor-readable or computer-readable media may be referred to herein as a computer program product.

Claim 1:
A system (<NUM>), comprising:
a conveyor system (<NUM>) configured to transport one or more objects (<NUM><NUM>-N) via a conveyor belt (<NUM>);
a vision system (<NUM>) that comprises one or more sensors configured to scan the one or more objects associated with the conveyor system; and
a processing device configured to employ a first machine learning model to determine object pose data associated with the one or more objects based on sensor data captured by the one or more sensors of the vision system and wherein the object pose data comprises a position and/or orientation data of each of the one or more objects with respect to the conveyor belt, and wherein the processing device is further configured to generate speed control data (<NUM>) for the conveyor belt of the conveyor system based on a set of control policies associated with the object pose data, wherein the speed control data comprises a belt speed for the conveyor belt, wherein the processing device is further configured to generate the set of control policies by employing a simulation associated with multiple conveyor systems, to minimize differences between, at least, real speed control data and simulated speed control data, wherein the simulated speed control data is gathered using said simulation, and to determine an optimal belt speed for the conveyor belt to mitigate a jam condition.