Patent Document ID: 10152676
Application ID: 14087852
Patent Flag: 1

Claim One:
1. A system comprising: a first computing device and a second computing device; wherein the first computing device is configured by executable instructions to: process, using a neural network comprising a plurality of trainable parameters, a training data input vector to generate an output vector; compute a gradient based at least on the output vector, the gradient comprising a plurality of gradient values and the gradient corresponding to the plurality of trainable parameters; add the gradient to a residual gradient comprising a first plurality of residual gradient values, wherein the first plurality of residual gradient values comprises values by which corresponding trainable parameters of the neural network are to be updated; select a first subset of the first plurality of residual gradient values, wherein individual residual gradient values of the first subset of the first plurality of residual gradient values are selected for the first subset based on the individual residual gradient values exceeding a threshold; transmit a first transmission comprising the first subset of the first plurality of residual gradient values to the second computing device configured to process training data using a separate version of the neural network, wherein the first transmission comprises residual gradient values corresponding to less than all of the plurality of trainable parameters; receive a second transmission comprising a second subset of a second plurality of residual gradient values from the second computing device; and modify one or more of the trainable parameters using the first subset of the first plurality of residual gradient values and the second subset of the second plurality of residual gradient values; wherein the second computing device is configured by executable instructions to: transmit the second transmission to the first computing device, wherein the second transmission comprises residual gradient values corresponding to less than all of a plurality of trainable parameters of the separate version of the neural network; receive the first transmission from the first computing device; and modify one or more of the trainable parameters of the separate version of the neural network using the first subset of the first plurality of residual gradient values and the second subset of the second plurality of residual gradient values; and wherein the system is configured to generate a trained version of the neural network using the one or more trainable parameters of the separate version of the neural network that have been modified.