Patent Description:
This specification generally relates to controllers for robotic arms.

Robotic arms are used for many purposes, including performing tasks in industrial facilities and performing everyday tasks to assist people (e.g., by assisting with eating, drinking, etc.). A robotic arm typically includes links that are connected by joints that enable translational and/or rotational movement of the links and an end effector (e.g., tool or gripper) for performing the tasks. Controllers can control the motion of the joints to move the links and the end effector into target positions.

<CIT> discloses an apparatus for training and controlling of robotic devices. A robot is utilized to perform a target task characterized by a target trajectory. The robot is trained by a user using supervised learning. The robot comprises an adaptive controller comprising a neuron network, which is configured to generate actuator control commands based on the user input and output of the learning process.

This specification generally describes neuromorphic controllers that provide smooth control of robotic arms. The control can be realized using a spiking neural network (SNN), with control models that imitate experimentally identified neuronal connectomes that allow animals and humans to achieve smooth reaching movements. Being biologically plausible, the control models inherit not only the potential to replicate the dynamics of human reaching movements, but also the minimal complexity of their biological counterpart.

Smooth control should be used when robotic arms need to interact with humans, e.g., for industrial robotic arms and assistive robotic arms. The smooth control for industrial robotic arms ensures the safety of the workers that work alongside and/or interact with the robotic arms. The smooth control provides the same safety while also making the user experience more pleasant when used to control assistive robotic arms. The prolonged life expectancy of robot motors when smooth control is employed is an additional advantage of the robotic arm control techniques described in this document.

Robotic arm controllers can exploit neuronal ensembles to approximate the differential equations used in classical control theory and achieve robotic arm control. The techniques described in this document exploit not only the units of brain computation, i.e., neurons, but also the connectome of brain controllers to tackle specific tasks. In this way, the robotic arm controllers can achieve similar results with less complicated SNNs, resulting in improvements in energy efficiency, latency, and explainability. For example, the SNNs described in this document are substantially smaller and more efficient than neuron ensembles that typically require thousands of neurons per ensemble. The controllers can be applied to robotic arms to allow for smooth control of discrete and continuous movements. The controllers can be used for industrial robot arms, patient-assistive arms, and other types of robotic arms or other types of robots, to address safety and user-friendliness.

Neuromorphic controllers described in this document are based on the neuronal connectome of some animals (e.g., primates and rodents) and humans into an SNN to control the rotation of a single joint. The neuromorphic controller can be used as a building block to scale up to the multiple joints of real-world robotic arms, e.g., robotic arms with multiple joints providing multiple degrees of freedom such as a six degree of freedom robotic arm. The neuromorphic controllers of multiple joints of a robotic arm can be coordinated to improve the dynamics of the movements of the robotic arm, e.g., to make the movements smoother and similar to that of humans, and can enable fast adaption of the robotic arm to a new environment.

According to some implementations, a method for controlling a joint of a robotic arm can include receiving, by a neuromorphic controller comprising a spiking neural network (SNN), a target value of a joint control variable for a joint of the robotic arm. The SNN includes a first and a second position control proprioceptor neuron, a first and a second speed control proprioceptor neuron, an extensor motor neuron, and a flexor motor neuron. The neuromorphic controller generates an actuation signal that causes the joint of the robotic arm to move based on a difference between the target value of the joint control variable and an actual value of the joint control variable. Generating the actuation signal includes: determining, by the first position control proprioceptor neuron, that the actual value of the joint control variable is less than the target value of the joint control variable and, in response, generating first spikes to the extensor motor neuron; or determining, by the second position proprioceptor neuron, that the actual value of the target control variable is greater than the target value of the joint control variable and, in response, generating the first spikes to the flexor motor neuron; and determining, by the first speed control proprioceptor neuron, that the speed of movement of the joint of the robotic arm is positive and, in response, generating second spikes to the flexor motor neuron; or determining, by the second speed control proprioceptor neuron, that the speed of movement of the joint of the robotic arm is negative and, in response, generating the second spikes to the extensor motor neuron; and generating a value of the actuation signal based on spiking activity, caused by the first spikes and the second spikes, of the extensor motor neuron or the flexor motor neuron. The actuation signal is output to the joint to cause the joint to move.

Implementations can include one or more of the following features. In some aspects, generating, by one of the position control proprioceptor neurons, a first spike to one of the extensor motor neuron or the flexor motor neuron based on a difference between the actual value of the joint control variable and the target value of the joint control variable includes determining, by a first position control proprioceptor neuron, that the actual value of the joint control variable is less than the target value of the joint control variable and, in response, generating the first spikes to the extensor motor neuron; or determining, by a second position proprioceptor neuron, that the actual value of the target control variable is greater than the target value of the joint control variable and, in response, generating the first spikes to flexor motor neuron. A quantity of the first spikes can be based on a magnitude of the difference between the actual value of the joint control variable and the target value of the joint control variable.

In some aspects, generating, by one of the speed control proprioceptor neurons of the SNN, the second spikes to one of the extensor motor neuron or the flexor motor neuron based on whether a change in speed of movement of the joint of the robotic arm is positive or negative includes determining, by a first speed control proprioceptor neuron, that the change in speed of movement of the joint of the robotic arm is positive and, in response, generating the second spikes to the flexor motor neuron; or determining, by a second speed control proprioceptor neuron, that the change in speed of movement of the joint of the robotic arm is negative and, in response, generating the second spikes to the extensor motor neuron.

In some aspects, updating the actual value of the joint control variable based on spiking activity of the extensor motor neuron or the flexor motor neuron includes reducing a speed of movement of the joint as the actual value of the target control variable joint approaches to the target value of the target control variable.

Some aspects adjusting, by a presynaptic inhibitory (PSI) neuron of the SNN, at least one of (i) a first connection weight between the first position control proprioceptor neuron and the extensor motor neuron or (ii) a second connection weight between the second position control proprioceptor neuron and the flexor motor neuron. The PSI neuron can adjust the first connection weight and/or the second connection weight based on a difference between the target value of the joint control variable and the actual value of the joint control variable.

Some aspects can include updating, by an additional neuromorphic controller for an additional joint of the robotic arm, an additional actual value of an additional joint control variable of the additional joint based on a target value of the additional joint control variable, on whether a change in speed of movement of the additional joint of the robotic arm is positive or negative, and at least one of (i) spikes generated by the extensor motor neuron of the neuromorphic controller or (ii) spikes generated by the flexor motor neuron of the neuromorphic controller. Some aspects include coordinating, by the additional neuromorphic controller, movement of the additional joint based on movement of the joint. Some aspects include determining the target value of the joint control variable of the joint of the robotic arm based on user input received from one of (i) a joystick, (ii), a speech recognition image, or (iii) a camera.

The methods in accordance with the present disclosure can include any combination of the aspects and features described herein. That is, methods in accordance with the present disclosure are not limited to the combinations of aspects and features specifically described herein, but also may include any combination of the aspects and features provided.

Particular embodiments of the subject matter described in this specification can be implemented to realize one or more of the following advantages. Using neuromorphic controllers that imitate neuronal connectomes of humans and animals enables smooth control of the various links of robotic arms, including, for example, gradually increasing the speed during movement initiation and timely decreasing it as the joint approaches its target arrangement, e.g., target angle of a revolute joint or target length of a prismatic joint. This smooth motion ensures the safety of people that work alongside and/or interact with robotic arms, improves the user experience for people who interact with assistive robotic arms, and increases life expectancy and reduces failures in robot motors. The control models implemented using SNNs described in this document improve energy efficiency, latency, and explainability relative to conventional control models. Coordinating the motion of multiple joints using neuromorphic controllers improves the quality of movement of the overall robotic arm, e.g., by providing smoother motion, reducing overshoots of links of the arm, and leveraging the inertial motion of primary joints for the movement of subordinate joints to imitate the motion of human arms.

The inclusion of speed control proprioceptor neurons enables the neuromorphic controller to provide smooth motion by confining the speed at which a variable for the joint changes (e.g., angle for revolute joints or length for prismatic joints) within a specific range. The inclusion of a presynaptic inhibitory neuron further enhances this smooth motion by adjusting presynaptic weights of the motor neurons in a range that reduces or prevents overshoots, while allowing for accurate correction of small differences between targets and actual arrangements.

Other features and advantages of the present disclosure will be apparent from the description, drawings, and from the claims.

<FIG> depicts an example neuromorphic controller <NUM>. The neuromorphic controller <NUM> can be configured to control a joint <NUM> of a robotic arm or other type of robot or moveable object, such as objects with a hinge or a vehicle. For example, as the neuromorphic controller <NUM> can dictate any scalar amount, the neuromorphic controller <NUM> can be used to control the linear or angular speed of a wheeled or flying vehicle. For brevity, the description that follows is in terms of joints of robotic arms.

In general, a robotic arm can include multiple links and a mechanical joint between each pair of adjacent links that controls the movement of the adjacent links relative to one another. For example, a robotic arm can include two links with a revolute joint that controls an angle between the two joints. In another example, a robotic arm can include three links with a revolute joint between each pair of links. In yet another example, a robotic arm can include two links with a prismatic joint that provides linear sliding movement between the two links.

The neuromorphic controller <NUM> can be used to control a joint <NUM> of a single joint robotic arm or a joint of a robotic arm that has multiple joints. For example, a control system can include an individual neuromorphic controller for each of one or more joints of a robotic arm. As described below, the controller output of two or more neuromorphic controllers can be coordinated to improve the control smoothness of adjacent links and provide human-like arm movements.

The neuromorphic controller <NUM> can be implemented as a spiking neural network (SNN). A spiking neural network is an artificial neural network that uses biologically realistic models of neurons to closely imitate natural neural networks. The neuromorphic controller <NUM> can be implemented using special-purpose circuitry, e.g., special-purpose machine learning circuits. For example, the neuromorphic controller <NUM> can be implemented using a neuromorphic processor, e.g., a neuromorphic research chip, such as Intel's Loihi chip, or a neural network processor, such as BrainChip's Akida™ chip. In such special-purpose circuitry, the neurons can be implemented as hardware neuron cores that output spikes based on voltage or current inputs to the neurons. The spikes can be signals, e.g., messages, sent between neurons and, in some cases, can carry a message payload. A spike can be a voltage signal having a positive voltage level and a short duration, e.g., having a spike width of about <NUM> millisecond.

In some implementations, the neurons used for the neuromorphic controller <NUM> follow the one-step leaky-integrate-and-fire (LIF) dynamics (each presynaptic spike directly increases the membrane voltage of the postsynaptic neuron). However, the model could be effectively implemented with non-leaky neurons (IF), two-step LIF neurons (each presynaptic spike generates a current, which is in turn integrated into the voltage of the postsynaptic neuron's membrane potential) or Izhikevich neurons.

The voltage, or a voltage value that represents a voltage in software if the SNN is modeled in software, can be increased at the neuron upon receipt of each spike until a threshold is reached using the LIF dynamics. When the threshold is reached, that neuron can fire, e.g., output spike signals to a next neuron in the network. Connection weights at the input to a neuron can control the increase in the voltage value at the neuron caused by each spike.

The neuromorphic controller <NUM> can receive a desired target value of a joint control variable (labeled as θd in the figures) and determines, e.g., computes, an actuation signal that is provided to an actuator of the joint <NUM> being controlled by the neuromorphic controller <NUM>. The joint control variable can be any scalar amount, such as the angle of a revolute joint, or the length of a prismatic joint, or the angular speed/linear speed of a motor.

In some implementations, the target value of the joint control variable is determined based on user input. For example, the neuromorphic controller <NUM> or another component can determine the target value for the joint control variable for the joint <NUM> based on input from a user through a joystick, using voice input through a speech recognition engine, or from a camera using object recognition. The control of the joint <NUM> is realized by comparing the actual value of the joint control variable (labeled as θ in the figures) with the target value of the joint control variable (θd).

The actuation signal can correspond to the actual value of the joint control variable and/or represent a change to the actual value of the joint control variable. The neuromorphic controller <NUM> can continuously determine and output the actuation signal to move the joint <NUM> from a current arrangement (e.g., current position, length, or angle) to the target arrangement corresponding to the target value of the joint control variable. For example, motor neurons <NUM> and <NUM> of the neuromorphic controller <NUM>, which are described in detail below, can generate spiking activity (e.g., a series of spikes) that are translated into torques that represent changes in the actual value of the joint control variable. In this example, the actuation signal that the neuromorphic controller <NUM> sends to the joint <NUM> can be the torques (or values that represent the torques) that are generated based on the spiking activity.

A robot controller can provide, to the neuromorphic controller <NUM>, target values of the joint control variable, e.g., in a sequence, to guide the neuromorphic controller <NUM> to provide the desired robot arm movement. The neuromorphic controller <NUM> uses the target value of the joint control variable and the actual value of the control variable to control the joint <NUM> in accordance with the target values received from the robot controller. The neuromorphic controller <NUM> can receive the actual value of the joint control variable from a sensor <NUM> that measures the actual value of the joint control variable, e.g., continuously.

The neuromorphic controller <NUM> includes several types of neurons that make up the SNN. In particular, the neuromorphic controller <NUM> includes position control proprioceptor neurons <NUM> and <NUM>, speed control proprioceptor neurons <NUM> and <NUM>, a presynaptic inhibitory (PSI) neuron <NUM>, and motor neurons <NUM> and <NUM>. The presynaptic inhibitory neuron <NUM> is optional and can be used to prevent, or reduce the amount by which, the joint <NUM> overshoots its target arrangement as described below.

The neuromorphic controller <NUM> includes current converter modules (CCMs) <NUM>-<NUM>. The CCMs <NUM> and <NUM> encode the actual value of the joint control variable into current values. The CCMs <NUM> and <NUM> can receive the actual value of the joint control variable from a sensor <NUM> that measures that actual value at the joint <NUM>.

Similarly, the CCMs <NUM> and <NUM> convert the target value of the joint control variable into current values. The current values can be actual electrical currents when the SNN of the neuromorphic controller <NUM> is implemented using hardware components, e.g., analog components, or values that represent currents when the SNN is implemented in software.

The CCMs <NUM>-<NUM> generate current pulse outputs based on their respective inputs. The current pulse outputs of the CCMs <NUM>-<NUM> stimulate the neurons of the network, driving their spiking activity.

The neuromorphic controller <NUM> includes two CCMs for each value of the joint control variable to encode both positive and negative values of current values and target values of the joint control variable. For that reason, the two values are provided to their two respective CCMs with opposite signs. The CCMs <NUM>-<NUM> can be governed by a rectified linear activation function (ReLU)-like transfer function. That is, the CCMs <NUM>-<NUM> can output a value of zero when its input argument is negative, while outputting linear values for positive input arguments, returning a proportion of the input argument. The current pulse output of a CCM is proportional, e.g., directly proportional, to the input of the CCM when the input is non-negative. Otherwise, the output of the CCM is zero. In that way, the activity in a pair of CCMs (e.g., CCM <NUM> and CCM <NUM>) is mutually exclusive. For example, when the actual value of the joint control variable is positive, the CCM <NUM> is inactive, while the CCM <NUM> is active, and vice versa. As the current pulse outputs are proportional to the input of the CCM, the current pulse outputs of the CCM <NUM> are proportional to negative actual values of the joint control variable and the current pulse outputs of the CCM <NUM> are proportional to positive actual values of the joint control variable.

Similarly, when the target value of the joint control variable is positive, the CCM <NUM> is inactive, while the CCM <NUM> is active, and vice versa. In particular, the current pulse outputs of the CCM <NUM> are proportional to negative target values of the joint control variable and the current pulse outputs of the CCM <NUM> are proportional to positive target values of the joint control variable.

The output current pulses of each CCM are provided as input to position control proprioceptor (PPC) neurons <NUM> and <NUM>. The position control PPC neurons <NUM> and <NUM> are part of a sensing stage <NUM> of the neuromorphic controller <NUM> that also includes speed control PPC neurons <NUM> and <NUM>. The PPC neurons <NUM>-<NUM> can each be implemented as spiking neurons in an SNN. The PPC neurons <NUM>-<NUM> are also referred to as PPCs for brevity. In an SNN implementation, the position control PPC neuron <NUM> may only spike, e.g., output or fire spikes, when the actual value of the joint control variable is less than the target value of the joint control variable. For example, if the actual angle of the joint <NUM> is less than the target angle for the joint <NUM>, the position control PPC neuron <NUM> can output spikes. If the actual value of the joint control variable is equal to or greater than the target value of the joint control variable, the position control PPC neuron <NUM> will not output spikes.

Similarly, the PPC neuron <NUM> may only spike when the actual position of the joint <NUM> is greater than the target position. For example, if the actual angle of the joint <NUM> is greater than the target angle for the joint <NUM>, the position control PPC neuron <NUM> can output spikes. If the actual value of the joint control variable is equal to or less than the target value of the joint control variable, the position control PPC neuron <NUM> will not output spikes.

The sign of the connections from each CCM <NUM>-<NUM> is selected to dictate the following behavior of the position control PPC neurons <NUM> and <NUM>. The goal of these two PPC <NUM> and <NUM> is to fire spikes in a mutually exclusive manner, with spikes from PPC <NUM> indicating that the actual value of the joint control variable is less than the target value of the joint control variable, while the spikes of PPC <NUM> indicate that the actual value of the joint control variable is greater than the target value of the joint control variable. Notably, the magnitude of these differences may not be encoded in the spiking activity itself, e.g., the presence of spikes in each neuron, but in the firing rate of each neuron, e.g., the number of spikes per unit time. In this example, it will always be PPC <NUM> firing if the actual value of the joint control variable is x less than the target value or 2x less than the target value, but it will fire twice as many spikes, e.g., per unit time, when the actual value is 2x less than the target value. That is, the rate of spikes is based on, e.g., proportional to, the magnitude of the difference between the target value of the joint control variable and the actual value of the joint control variable.

To achieve this behavior, the sign of the connections from the CCMs <NUM> and <NUM> to the PPCs <NUM> and <NUM> are selected as follows. In <FIG>, a connection with an arrow indicates an excitatory connection while a connection with a dot indicates an inhibitory connection. CCM <NUM> excites PPC <NUM>, while CCM <NUM> inhibits PPC <NUM>. Therefore, when the target value of the joint control variable is positive making CCM <NUM> active and the actual value is positive making CCM <NUM> active but less than the target value, the overall current value stimulating PPC <NUM> will be positive. For example, the overall current value input to the PPC <NUM> can be based on the difference between the excitatory current value input from CCM <NUM> and the inhibitory current value input from CCM <NUM>. This overall positive input causes PPC <NUM> to output spikes (indicating that θ is less than θd) at a rate that is based on, e.g., proportional to, the magnitude of the difference is (how much less is θ than θd). At the same time, the same CCMs <NUM> and <NUM> excite and inhibit PPC <NUM>, respectively. In that way, the overall current value stimulating PPC <NUM> will be negative (e.g., the difference between the excitatory current value input from CCM <NUM> and the inhibitory current value input from CCM <NUM>), and no spikes will be elicited there (indicating that θ is not greater than θd). This ensures the mutually exclusive activation of PPC <NUM> and <NUM>.

CCM <NUM> inhibits PPC <NUM>, while CCM <NUM> excites PPC <NUM>. Therefore, when the target value of the joint control variable is positive, making CCM <NUM> active, and the actual value is positive making CCM <NUM> active but greater than the target, the overall input to PPC <NUM> will be positive. For example, the overall current value input to the PPC <NUM> can be based on the difference between the excitatory current value input from CCM <NUM> and the inhibitory current value input from CCM <NUM>. This overall positive input causes PPC <NUM> to output spikes (indicating that θ is greater than θd) at a rate that is based on, e.g., proportional to, the magnitude of the difference (how much greater is θ than θd). At the same time, the exact same CCM <NUM> and <NUM> inhibit and excite PPC <NUM>, respectively. In that way, the overall current value stimulating PPC <NUM> will be negative (e.g., difference between the excitatory current value input from CCM <NUM> and the inhibitory current value input from CCM <NUM>), and no spikes will be elicited there (indicating that θ is not less than θd). This ensures the mutually exclusive activation of PPC <NUM> and <NUM>.

The connectivity of the other two CCMs <NUM> and <NUM> to the PPC <NUM> and <NUM> can be selected using the same rationale, when considering negative actual and target values for the joint control variable. CCM <NUM> excites PPC <NUM>, while CCM <NUM> inhibits PPC <NUM>. Therefore, when the target value of the joint control variable is negative making CCM <NUM> active and the actual value is negative making CCM <NUM> active but is less than the target value, the overall current value stimulating PPC <NUM> will be positive. For example, the overall current value input to the PPC <NUM> can be based on the difference between the excitatory current value input from CCM <NUM> and the inhibitory current value input from CCM <NUM>. This overall positive input causes PPC <NUM> to output spikes (indicating that θ is less than θd) at a rate that is based on, e.g., proportional to, the magnitude of the difference is (how much less is θ than θd). At the same time, the same CCMs <NUM> and <NUM> excite and inhibit PPC <NUM>, respectively. In that way, the overall current value stimulating PPC <NUM> will be negative (e.g., the difference between the excitatory current value input from CCM <NUM> and the inhibitory current value input from CCM <NUM>), and no spikes will be elicited there (indicating that θ is not greater than θd). This ensures the mutually exclusive activation of PPC <NUM> and <NUM>.

CCM <NUM> inhibits PPC <NUM>, while CCM <NUM> excites PPC <NUM>. Therefore, when the target value of the joint control variable is negative, making CCM <NUM> active, and the actual value is positive making CCM <NUM> active but greater than the target, the overall input to PPC <NUM> will be positive. For example, the overall current value input to the PPC <NUM> can be based on the difference between the excitatory current value input from CCM <NUM> and the inhibitory current value input from CCM <NUM>. This overall positive input causes PPC <NUM> to output spikes (indicating that θ is greater than θd) at a rate that is based on, e.g., proportional to, the magnitude of the difference (how much greater is θ than θd). At the same time, the exact same CCM <NUM> and <NUM> inhibit and excite PPC <NUM>, respectively. In that way, the overall current value stimulating PPC <NUM> will be negative (e.g., difference between the excitatory current value input from CCM <NUM> and the inhibitory current value input from CCM <NUM>), and no spikes will be elicited there (indicating that θ is not less than θd). This ensures the mutually exclusive activation of PPC <NUM> and <NUM>.

The indication of whether the actual value of the joint control variable is less than or greater than the target value is used to update the actual value towards the target value. For that reason, the position control PPCs <NUM> and <NUM> are connected to their respective motor neurons, the extensor motor neuron <NUM> and the flexor motor neuron <NUM>. These motor neurons <NUM> and <NUM> are part of an actuation stage <NUM> of the neuromorphic controller <NUM>.

The extensor motor neuron <NUM> integrates the output spikes of PPC <NUM>, indicating that the actual value of the joint control variable is less than the target value, into its spiking activity. That is, the output spikes of PPC <NUM> cause the extensor motor neuron <NUM> to output spikes if a sufficient number of outputs spikes reach the extensor motor neuron <NUM> to satisfy its threshold. More frequent spikes from PPC <NUM> result in more spikes output by the extensor motor neuron <NUM>. The spikes of the extensor motor neuron <NUM> cause an increase in the actual value (θ) of the joint control variable to reach the target value (θd) of the joint control variable. The output of the extensor motor neuron <NUM> is connected to an actuation signal generator <NUM> that translates the spikes to actuation signals, e.g., to torques. The actuation signal generator <NUM> sends the torques to the joint <NUM>. Therefore, each of the spikes of the extensor motor neuron <NUM> are translated to an incremental increase of the actual value of the joint control variable.

The flexor motor neuron <NUM> integrates the output spikes of PPC <NUM>, indicating that the actual value of the joint control variable is greater than the target value, into its spiking activity. That is, the output spikes of PPC <NUM> cause the flexor motor neuron <NUM> to output spikes if a sufficient number of outputs spikes reach the flexor motor neuron <NUM> to satisfy its threshold. More frequent spikes from PPC <NUM> result in more spikes output by the flexor motor neuron <NUM>. The spikes of the flexor motor neuron <NUM> decrease the actual value (θ) of the joint control variable to reach the desired value (θd). The output of the flexor motor neuron <NUM> is connected to the actuation signal generator <NUM>, which translates the spikes to actuation signals as described above. Therefore, each of the spikes of the flexor motor neuron <NUM> are translated to an incremental decrease of the actual value of the joint control variable.

The connections between the position control PPCs <NUM> and <NUM> and their respective motor neurons <NUM> and <NUM> differ from the other synaptic connections of the network, by incorporating facilitation using facilitating synapses <NUM> and <NUM>. This means that the synaptic efficacy of the connections is not static, but it increases with each presynaptic spike until saturating at a maximum value. This property is used for these connections to improve the smoothness of the control. When an abrupt change occurs in the target value for the joint control variable, the activity of either of the PPC <NUM> or <NUM> increases abruptly as well. In that way, absent facilitation, the abrupt increase would be propagated to the motor neurons <NUM> and <NUM> and could cause a sudden and jerky effect on the actual controlled value (θ) of the joint control variable. Incorporating facilitation, this effect is mitigated as follows. The initial, abrupt increase in the activity of either PPC <NUM> or <NUM> is not propagated wholly to either motor neuron <NUM> or <NUM>, because the initial synaptic weights are low, but instead gradually boosts the efficacy of these synaptic weights. The gradual increase of the synaptic weights results in a gradual activation of motor neurons <NUM> or <NUM>, which, in turn, leads to a smooth increase or decrease of the actual value (θ) of the joint control variable.

Facilitating synapses are synapses whose weight increases with an increase in received spikes. That is, the weight of the facilitating synapse <NUM> increases gradually upon receipt of spikes from the position control PPC <NUM>. Similarly, the weight of the facilitating synapse <NUM> increases upon receipt of spikes from the position control PPC <NUM>. As time passes without receiving spikes, the weight of the facilitating synapse <NUM> or <NUM> decreases, e.g., decays over time. The presynaptic spikes output by PPC <NUM> are scaled by the facilitating synapse <NUM> based on the weight of the facilitating synapse <NUM> before affecting the extensor motor neuron <NUM>. Similarly, the presynaptic spikes output by PPC <NUM> are scaled by the facilitating synapse <NUM> based on the weight of the facilitating synapse <NUM> before affecting the flexor motor neuron <NUM>.

In addition to the position control described above, the neuromorphic controller <NUM> also incorporates speed control, which is the first derivative of the actual value of the joint control variable. To incorporate speed control, the neuromorphic controller <NUM> includes an additional pair of speed control PPC neurons <NUM> and <NUM> whose firing activity encodes how the speed of the joint control variable varies. Notably, there is no target speed, as there is with position, as described above. The two PPC <NUM> and <NUM> fire in a mutually exclusive manner in response to negative or positive speed, respectively. The neuromorphic controller <NUM> includes a speed tracker <NUM> that keeps track of the change in the actual value of the joint control variable over time using a sliding window and estimates the first derivative of the speed of change of the joint control variable by computing the difference between the actual values of the joint control variable at the endpoints of that window and dividing by the length of the window, simulating the behavior of a digital differentiator. The speed tracker <NUM> transforms the speed value at every time instance to a current value (e.g., electric current value), using a concept similar to the CCMs <NUM>-<NUM> described above. More specifically, the speed tracker <NUM> translates the speed value to a current value via a linear transformation, which is then multiplied by the two opposite signs of the connections targeting the PPCs <NUM> and <NUM>. Before feeding the currents into these neurons, the negative one is rectified to zero using a ReLU function and the respective neuron receives no current, while the positive current is directly fed to its respective neuron. The resulting current value is fed to PPCs <NUM> and <NUM> with opposite signs, to ensure mutually exclusive firing. In this way, when the estimated speed is positive, PPC <NUM> will be excited, while PPC <NUM> will be inhibited, and vice versa.

The inclusion of PPCs <NUM> and <NUM> in the neuromorphic controller <NUM> improves the smoothness of the control in a way that is explained by their connections to the motor neurons <NUM> and <NUM>. Specifically, PPC <NUM>, whose spiking indicates an increase in the speed of the actual value of the joint control variable is activating the flexor motor neuron <NUM> to decrease the actual value of the joint control variable. Therefore, PPCs <NUM> and <NUM> are counteracting the effect that their own activity indicates, introducing some push-pull dynamics that improve the control. The rationale behind these dynamics is the desire to have a timely deceleration of the actual value of the joint control variable when approaching the target value. Therefore, in this context, a large speed magnitude is a good indication for the network to start decelerating. The opposite effect is realized by the connection of PPC <NUM> to the extensor motor neuron <NUM>.

The PSI neuron <NUM> further improves the control behavior, by introducing adaptive feedback, which minimizes the overshoot inflicted on the actual value of the joint control variable. The PSI neuron <NUM> is part of a feedback adaption stage <NUM> of the neuromorphic controller <NUM>. The PSI neuron <NUM> adjusts a connection weight <NUM> used to weight the spikes output by the position control PPC <NUM>, which may also be weighted by the facilitating synapse <NUM>. The PSI neuron <NUM> also adjusts a connection weight <NUM> used to weight the spikes output by the position control PPC <NUM>, which may also be weighted by the facilitating synapse <NUM>.

The advantages of this PSI neuron <NUM> are shown when considering two extreme cases of difference between target and actual values of the joint control variable. When the difference between the target and actual values is small, the PSI neuron <NUM> provides a large connection weight <NUM> between PPC <NUM> and the extensor motor neuron <NUM> and/or a large connection weight <NUM> between PPC <NUM> and the flexor motor neuron <NUM>, in order to propagate the low spiking activity of PPC <NUM> or PPC <NUM> to motor neurons <NUM> and <NUM> to decrease the difference between the target and actual values of the joint control variable.

However, with that weight selection configuration, when a really large difference between target and actual value appears, the network may not handle the previously selected weight ideally, giving rise to an overshoot past the target value of the joint control variable. The strong connection weight can drive the change in the current value and thus the spikes output by the PPCs <NUM> and <NUM> strongly and the network may not decelerate on time, leading to the overshoot.

With these two extreme cases in mind, it may be preferred to have the weight of that connection to be large when the network needs to cope with small differences, while getting smaller when facing larger changes. To introduce this kind of adaptive feedback, the PSI neuron <NUM> is included in the neuromorphic controller <NUM>. The spiking activity of the PSI neuron <NUM> reflects the spiking activity of the error position of PPCs <NUM> and <NUM>. In this way, the PSI neuron <NUM> fires intensively when the positional error is large, e.g., when one of the position control PPCs has high spiking activity, while it only fires slightly when the position error is small, when neither position control PPC has spiking activity (e.g., no output spikes) or the one that has spiking activity has low spiking activity (e.g., output spikes at less than a threshold rate). For example, the rate of spikes output by the PSI neuron <NUM> can be proportional to the rate of spikes being received by the PSI neuron from one of the position PPCs <NUM> or <NUM>.

The spikes of the PSI neuron <NUM> are used to modulate the connection weight <NUM> between PPC <NUM> and the extensor motor neuron <NUM> and the connection weight <NUM> between PPC <NUM> and the flexor motor neuron <NUM>. The weights <NUM> and <NUM> increase with an increase in spikes from the PSI neuron <NUM> and decrease with a decrease in spikes from the PSI neuron <NUM>. The modulation of the weight is divisive, to simulate how presynaptic inhibition works in real synapses (increasing PSI activity induces a proportional decrease on the weight).

As described above, the extensor motor neuron <NUM> integrates the output spikes of PPC <NUM> and outputs spikes to the actuation signal generator <NUM> to increase the actual value (θ) of the joint control variable to reach the target value (θd) of the joint control variable. The flexor motor neuron <NUM> integrates the output spikes of PPC <NUM> and outputs spikes to the actuation signal generator <NUM> to decrease the actual value (θ) of the joint control variable to reach the desired value (θd). The actuation signal generator <NUM> translates the spikes into appropriate actuation signals, e.g., torques, and outputs the actuation signals to the actuator of the joint <NUM>.

The actuation signal generator <NUM> translates spikes received from the extensor motor neuron <NUM> into actuation signals that cause the actuator of the joint <NUM> to move its links such that the actual value of the joint control variable increases. The amount of movement represented by each actuation signal can be proportional to the spiking activity of the extensor motor neuron <NUM>. Similarly, the actuation signal generator <NUM> translates spikes received from the flexor motor neuron <NUM> into actuation signals that cause the actuator of the joint <NUM> to move its links such that the actual value of the joint control variable decreases. The amount of movement represented by each actuation signal can be proportional to the spiking activity of the flexor motor neuron <NUM>.

The sensor <NUM> of the joint <NUM> can measure the actual value of the joint control variable and provide the actual value of the joint control variable to the neuromorphic controller <NUM>. The neuromorphic controller can provide the actual value of the joint control variable to the CCMs <NUM>-<NUM> to continue generating actuation signals to control the joint.

Although the neuromorphic controller <NUM> includes two position control PPCs <NUM> and <NUM>, two speed PPCs <NUM> and <NUM>, two motor neurons <NUM> and <NUM>, one of each of these components can be used for unidirectional control, e.g., only increasing the value of the joint control variable. In this example, the decrease can happen without control, e.g., decay with time.

<FIG> depicts an example neuromorphic controller <NUM> configured to provide coordinated smooth movement of two joints of a robotic arm. The example neuromorphic controller <NUM> includes a first neuromorphic controller <NUM> for controlling a first joint of a robotic arm and a second neuromorphic controller <NUM> for controlling a second joint of a robotic arm. The first and second neuromorphic controllers <NUM> and <NUM> include the same components as the neuromorphic controller <NUM> of <FIG> and can operate in a similar manner. However, the neuromorphic controllers <NUM> and <NUM> can coordinate movements to provide a smooth overall movement of the robotic arm. Although the example neuromorphic controller <NUM> includes two neuromorphic controllers <NUM> and <NUM>, more controllers can be used for robotic arms having more than two joints. The neuromorphic controller <NUM> can be expanded to provided coordinated smooth movement of more than two joints as well, e.g., by including an individual neuromorphic controller for each joint and connecting the extensor and flexor motor neurons of adjacent joints.

In this example, the first neuromorphic controller <NUM> is used to control a primary joint and the second neuromorphic controller <NUM> is used to control a subordinate joint that is subordinate to the primary joint. This arrangement enables the neuromorphic controller <NUM> to control the subordinate joint based on the position and speed of the primary joint, which is similar to the biological control of animal and human joints. For example, the timing of flexion/extension of the wrist (subordinate joint) of a human is typically dependent on the movement of the elbow (primary joint). In particular the extensor or flexor of the wrist can have a <NUM> degree phase difference from the extensor or flexor of the elbow.

To mimic this relationship, the outputs of the extensor motor neuron and the flexor motor neuron of the first neuromorphic controller <NUM> are provided to the extensor motor neuron and the flexor motor neuron, respectively, of the second neuromorphic controller <NUM>. The extensor and flexor motor neurons of the second neuromorphic controller <NUM> can be configured to adjust their outputs based on a combination of the outputs of its PPC neurons for the subordinate joint and whether and how much the primary joint is being flexed or extended. For example, if the primary joint is being transitioned from flex to extend, the extensor and flexor motor neurons of the second neuromorphic controller <NUM> can detect a change in the sign of the outputs of the extensor and flexor motor neurons of the first neuromorphic controller <NUM>. In response, the extensor and flexor motor neurons of the second neuromorphic controller <NUM> can adjust the position of the subordinate joint to provide a smoother motion for the robotic arm, e.g., by making a similar transition between extend and flex as the primary joint at a later time to mimic the <NUM> degree phase difference.

A coordination controller can be used to coordinate the control of the subordinate joint based on movement of the primary joint. For example, the coordination controller can receive the signals output by the extensor and flexor motor neurons of the first neuromorphic controller <NUM>. The coordination controller can then determine how to affect the control of the extensor and/or flexor motor neurons of the second neuromorphic controller <NUM> and send output signals, e.g., spikes, to the extensor and/or flexor motor neurons of the second neuromorphic controller <NUM>. In this way, the coordination between two joints is flexible and configurable for various use cases.

<FIG> depicts an environment <NUM> in which a robot control system <NUM> controls the joints of a robotic arm <NUM>. In this example, the robotic arm <NUM> has three joints <NUM>, <NUM>, and <NUM>. The joint <NUM> adjusts the position of a link <NUM> between the joint <NUM> and the joint <NUM>. For example, the joint <NUM> can rotate the link <NUM> about an axis. The joint <NUM> adjusts the position of a link <NUM> between the joint <NUM> and an end effector <NUM>. For example, the joint <NUM> can rotate the link <NUM> about an axis. The joint <NUM> can adjust the end effector <NUM>, e.g., by rotating the end effector <NUM>.

The robot control system <NUM> includes a neuromorphic controller <NUM> for each joint of the robotic arm <NUM>. A neuromorphic controller <NUM>-<NUM> can control the joint <NUM> by sending actuation signals to an actuator of the joint <NUM>. Similarly, a neuromorphic controller <NUM>-<NUM> can control the joint <NUM> by sending actuation signals to an actuator of the joint <NUM> and a neuromorphic controller <NUM>-<NUM> can control the joint <NUM> by sending actuation signals to an actuator of the joint <NUM>.

Each neuromorphic controller <NUM> includes an SNN <NUM> that can be implemented using the neuromorphic controller <NUM> of <FIG> or a neuromorphic controller similar thereto. For example, one or more of the neuromorphic controllers <NUM> can include a PSI neuron while others may not.

In some implementations, each neuromorphic controller <NUM> can control its joint independent of each other neuromorphic controller <NUM>. That is, the robot control system <NUM> may not coordinate the control of joints.

In some implementations, two or more of the neuromorphic controllers <NUM> can coordinate control of their joints. In this example, the joint <NUM> would be the primary joint and the joint <NUM> is subordinate to joint <NUM>. In addition, the joint <NUM> is subordinate to joint <NUM>. The neuromorphic controller <NUM>-<NUM> can control the joint <NUM> in coordination with the neuromorphic controller <NUM>-<NUM>. For example, the neuromorphic controllers <NUM>-<NUM> and <NUM>-<NUM> can be connected in the same or a similar manner as the neuromorphic controllers <NUM> and <NUM> of <FIG>. In this way, the link <NUM> and be moved based on the movement of the link <NUM>. For example, the link <NUM> can be moved using a <NUM> degree phase shift, as described above. The neuromorphic controller <NUM>-<NUM> can control the joint <NUM> in coordination with the neuromorphic controller <NUM>-<NUM> or the neuromorphic controller <NUM>-<NUM>, or without coordination with either controller.

<FIG> is a flow diagram of an example process <NUM> for controlling a joint of a robotic arm. The process <NUM> can be performed by a robot control system that includes a neuromorphic controller, such as the robot control system <NUM> of <FIG> or the neuromorphic controller <NUM> of <FIG>. For brevity, the operations of the process are described as being performed by a neuromorphic controller, which can be implemented on a computer or in special-purpose circuitry as described above.

The neuromorphic controller receives a target value of a joint control variable for a joint of a robotic arm (<NUM>). The joint control variable can be any scalar amount, such as the angle of a revolute joint, or the length of a prismatic joint, or the angular speed/linear speed of a motor. In some implementations, the target value of the joint control variable is determined based on user input. For example, the neuromorphic controller or another component can determine the target value for the joint control variable for a joint based on input from a user through a joystick, using voice input through a speech recognition engine, or from a camera using object recognition.

As described above, the neuromorphic controller can include an SNN. The SNN can include position control proprioceptor neurons, speed control proprioceptor neurons, an extensor motor neuron, and a flexor motor neuron. For example, the SNN can include two position control proprioceptor neurons and two speed control proprioceptor neurons, as shown in <FIG>. In some implementations, the SNN also includes a PSI neuron.

The neuromorphic controller generates an actuation signal based on a difference between the target value of the joint control variable and actual value of the joint control variable (<NUM>). This updating can be performed using the SNN and can include constituent operations <NUM>-<NUM>.

One of the position control proprioceptor neurons generates the first spikes to one of the extensor motor neuron or the flexor motor neuron based on a difference between the actual value of the joint control variable and the target value of the joint control variable (<NUM>). As described above, each position control proprioceptor neuron receives current pulse inputs from CCMs based on the sign of the actual and target values of the joint control variables and the difference between the actual value of the joint control variable and the target value of the joint control variable. When the actual value is greater than the target value, a first position control proprioceptor neuron can output spikes to the extensor motor neuron. When the actual value is less than the target value, a second position control proprioceptor neuron can output spikes to the flexor motor neuron. The operation of the position control proprioceptor neurons is described with reference to <FIG>.

One of the speed control proprioceptor neurons generates the second spikes to one of the extensor motor neuron or the flexor motor neuron based on whether a change in speed of movement of the joint of the robotic arm is positive or negative (<NUM>). When the change in speed is positive, a first speed control proprioceptor neuron can output spikes to the flexor neuron. When the change in speed is negative, a second first speed control proprioceptor neuron can output spikes to the extensor neuron. The operation of the speed control proprioceptor neurons is described with reference to <FIG>.

The neuromorphic controller generates a value of the actuation signal based on spiking activity of the extensor motor neuron or the flexor motor neuron (<NUM>). The spiking activity of the proprioceptor neurons causes one of the extensor motor neuron or the flexor motor neuron to output spikes. If the extensor motor neuron outputs spikes, an actuation signal generator can translate the spikes into an actuation signal that causes the joint to move such that the actual value of the joint control variable increases. If the flexor motor neuron spikes, the actuation signal generator can translate the spikes into an actuation signal that causes the joint to move such that the actual value of the joint control variable decreases.

As described above, if a PSI neuron is included, the PSI neuron can adjust the connection weights between one of the position control proprioceptor motor neurons and the extensor motor neuron and/or the connection weights between one of the position control proprioceptor motor neurons and the flexor motor neuron. The adjustment to the weights can be based on spiking activity of the PSI neuron, which can be based on a difference between the target value of the joint control variable and the actual value of the joint control variable.

The neuromorphic controller causes the joint to move by outputting the actuation signal to the joint (<NUM>). The actuator of the joint causes the joint to move in response to the actuation signal.

A determination is made whether to continue moving the robotic arm (<NUM>). This determination can be based on whether the joint has moved such that the actual value of the joint control variable matches the target value for the joint control variable. If so, the neuromorphic controller can determine to end the movement of the joint, e.g., until a new target value for the joint control variable is received. If not, the process <NUM> can return to operation <NUM> at which a new target value is received or to operation <NUM> where the neuromorphic controller continues moving the joint until the actual value matches the target value.

<FIG> depicts graphs that illustrate some advantages of using a neuromorphic controller to control a joint of a robotic arm. The graphs illustrate the differences in changes of a joint based on a change in a target angle <NUM> of the joint from about zero degrees to about <NUM> degrees.

The graph <NUM> shows a comparison between a non-facilitating movement <NUM> of joint without using a neuromorphic controller and facilitating movement <NUM> of the joint using a neuromorphic controller. As shown in the graph <NUM>, the facilitating movement <NUM> has a more gradual increase in the angle relative to the non-facilitating movement <NUM>, which can be caused by the PSI neuron and/or the facilitating synapses adjusting the connection weight to the motor neurons. As the joint approaches the target angle, the rate of change of the angle slows down in the non-facilitating movement, which can be caused by the speed control PPC neurons and/or the adjustment to the connection weights. This provides a smoother motion by reducing any jerking motions at the beginning and end of the transition between angles.

The graph <NUM> shows a comparison between the angular velocity of the non-facilitating movement <NUM> of the joint without using a neuromorphic controller and the angular velocity of the facilitating movement <NUM> of the joint using a neuromorphic controller. Similar to the change in the angle shown in the graph <NUM>, this graph <NUM> shows that the change in angular velocity increases and decreases more gradually in the facilitating movement <NUM> than in the non-facilitating movement <NUM>.

The graph <NUM> shows a comparison between the angular acceleration of the non-facilitating movement <NUM> of the joint without using a neuromorphic controller and the angular acceleration of the facilitating movement <NUM> of the joint using a neuromorphic controller. The facilitating movement <NUM> has a more gradual increase in angular acceleration and a more gradual decrease in angular acceleration than the non-facilitating movement <NUM>. The peak accelerations of the facilitating movement <NUM> are also lower in magnitude than the peak accelerations of the non-facilitating movement <NUM>, which demonstrates the smoother and less abrupt motions of the facilitating movement <NUM> as compared to the non-facilitating movement <NUM>. The reduction in angular jerk is shown in the graph <NUM>. This graph <NUM> shows that the angular jerk of the facilitating movement <NUM> is less than the angular jerk of the non-facilitating movement.

Embodiments of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, in tangibly embodied computer software or firmware, in computer hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non transitory program carrier for execution by, or to control the operation of, data processing apparatus. Alternatively or in addition, the program instructions can be encoded on an artificially generated propagated signal, e.g., a machine generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus. The computer storage medium is not, however, a propagated signal.

The apparatus can include special purpose logic circuitry, e.g., an FPGA (field programmable gate array), an ASIC (application specific integrated circuit), a neuromorphic research chip, such as Intel's Loihi chip, or a neural network processor, such as BrainChip's AkidaTM chip.

A computer program (which may also be referred to or described as a program, software, a software application, a module, a software module, a script, or code) can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and it can be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

Each engine can be implemented on any appropriate type of computing device, e.g., servers, mobile phones, tablet computers, notebook computers, music players, e book readers, laptop or desktop computers, PDAs, smart phones, or other stationary or portable devices, that includes one or more processors and computer readable media.

The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array), an ASIC (application specific integrated circuit) a neuromorphic research chip, such as Intel's Loihi chip, or a neural network processor, such as BrainChip's Akida™ chip.

Computer readable media suitable for storing computer program instructions and data include all forms of non volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto optical disks; and CD ROM and DVD ROM disks.

To provide for interaction with a user, embodiments of the subject matter described in this specification can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) monitor, an LCD (liquid crystal display) monitor, or an OLED display, for displaying information to the user, as well as input devices for providing input to the computer, e.g., a keyboard, a mouse, or a presence sensitive display or other surface. In addition, a computer can interact with a user by sending resources to and receiving resources from a device that is used by the user; for example, by sending web pages to a web browser on a user's client device in response to requests received from the web browser.

The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client server relationship to each other.

Claim 1:
A method for controlling a joint (<NUM>, <NUM>, <NUM>, <NUM>) of a robotic arm (<NUM>), the method comprising:
receiving, by a neuromorphic controller (<NUM>; <NUM>; <NUM>; <NUM>; <NUM>) comprising a spiking neural network, SNN, (<NUM>), a target value of a joint control variable for the joint of the robotic arm, wherein the SNN comprises a first and a second position control proprioceptor neuron (<NUM>, <NUM>), a first and a second speed control proprioceptor neuron (<NUM>, <NUM>), an extensor motor neuron (<NUM>), and a flexor motor neuron (<NUM>);
generating, by the neuromorphic controller, an actuation signal that causes the joint of the robotic arm to move based on a difference between the target value of the joint control variable and an actual value of the joint control variable, the generating comprising:
determining, by the first position control proprioceptor neuron, that the actual value of the joint control variable is less than the target value of the joint control variable and, in response, generating first spikes to the extensor motor neuron; or
determining, by the second position proprioceptor neuron, that the actual value of the target control variable is greater than the target value of the joint control variable and, in response, generating the first spikes to the flexor motor neuron; and
determining, by the first speed control proprioceptor neuron, that the speed of movement of the joint of the robotic arm is positive and, in response, generating second spikes to the flexor motor neuron; or
determining, by the second speed control proprioceptor neuron, that the speed of movement of the joint of the robotic arm is negative and, in response, generating the second spikes to the extensor motor neuron; and
generating a value of the actuation signal based on spiking activity, caused by the first spikes and the second spikes, of the extensor motor neuron or the flexor motor neuron; and
causing the joint to move by outputting the actuation signal to the joint.