A computer-implemented method to compress neural networks, the method comprising: obtaining, by one or more computing devices, a neural network, wherein the neural network comprises a plurality of layers that each include one or more nodes, and a plurality of weights that each connect nodes of different layers; selecting, by the one or more computing devices, a weight of the neural network; quantizing, by the one or more computing devices, the weight; determining, by the one or more computing devices, a quantization error associated with the weight, wherein the quantization error associated with the weight is a change in the weight due to quantization; distributing, by the one or more computing devices, at least a part of the quantization error to one or more non-quantized weights of the neural network; quantizing, by the one or more computing devices, one or more of the non-quantized weights; and providing, by the one or more computing devices, a quantized neural network, wherein the step of distributing the quantization error comprises: determining, by the one or more computing devices, one or more non-quantized weights associated with the weight, wherein the associated non-quantized weights are determined from nodes within a same layer of a node connected by the weight, nodes from a previous layer or following layer, and/or within connections of a single node connected by the weight; and transferring, by the one or more computing devices, a fraction of the quantization error associated with the weight to each of the associated non-quantized weights by summing an associated non-quantized weight with said fraction of the quantization error associated with the weight, and based on a correlation of activity between said associated non-quantized weight and the weight, wherein the correlation of activity is descriptive of an activation relationship between the weight and an associated non-quantized weight. obtaining, by one or more computing devices, a neural network, wherein the neural network comprises a plurality of layers that each include one or more nodes, and a plurality of weights that each connect nodes of different layers; selecting, by the one or more computing devices, a weight of the neural network; quantizing, by the one or more computing devices, the weight; determining, by the one or more computing devices, a quantization error associated with the weight, wherein the quantization error associated with the weight is a change in the weight due to quantization; distributing, by the one or more computing devices, at least a part of the quantization error to one or more non-quantized weights of the neural network; quantizing, by the one or more computing devices, one or more of the non-quantized weights; and providing, by the one or more computing devices, a quantized neural network, wherein the step of distributing the quantization error comprises: determining, by the one or more computing devices, one or more non-quantized weights associated with the weight, wherein the associated non-quantized weights are determined from nodes within a same layer of a node connected by the weight, nodes from a previous layer or following layer, and/or within connections of a single node connected by the weight; and transferring, by the one or more computing devices, a fraction of the quantization error associated with the weight to each of the associated non-quantized weights by summing an associated non-quantized weight with said fraction of the quantization error associated with the weight, and based on a correlation of activity between said associated non-quantized weight and the weight, wherein the correlation of activity is descriptive of an activation relationship between the weight and an associated non-quantized weight. determining, by the one or more computing devices, one or more non-quantized weights associated with the weight, wherein the associated non-quantized weights are determined from nodes within a same layer of a node connected by the weight, nodes from a previous layer or following layer, and/or within connections of a single node connected by the weight; and transferring, by the one or more computing devices, a fraction of the quantization error associated with the weight to each of the associated non-quantized weights by summing an associated non-quantized weight with said fraction of the quantization error associated with the weight, and based on a correlation of activity between said associated non-quantized weight and the weight, wherein the correlation of activity is descriptive of an activation relationship between the weight and an associated non-quantized weight.