A computing system comprising: a computer-readable memory storing an operational parameter of a given layer of a neural network; and a hardware accelerator in communication with the computer-readable memory, the hardware accelerator configured to: receive an input tensor for a given layer of a multi-layer neural network; convert the input tensor from a normal-precision floating-point format to a quantized-precision floating-point format; perform a tensor operation using the input tensor converted to the quantized-precision floating-point format; convert a result of the tensor operation from the quantized-precision floating-point format to the normal-precision floating-point format; and use the converted result in the normal-precision floating-point format to update the operational parameter stored in the computer-readable memory, where the parameter is stored in normal-precision floating-point format, wherein the input tensor is a convolutional filter, and the quantized-precision floating-point format is a block floating-point format wherein mantissa values within a sub-division of a spatial pixel share a common exponent, and wherein mantissa values within different sub-divisions of the spatial pixel have different respective exponents, each sub-division comprising a subset of channels of the spatial pixel. a computer-readable memory storing an operational parameter of a given layer of a neural network; and a hardware accelerator in communication with the computer-readable memory, the hardware accelerator configured to: receive an input tensor for a given layer of a multi-layer neural network; convert the input tensor from a normal-precision floating-point format to a quantized-precision floating-point format; perform a tensor operation using the input tensor converted to the quantized-precision floating-point format; convert a result of the tensor operation from the quantized-precision floating-point format to the normal-precision floating-point format; and use the converted result in the normal-precision floating-point format to update the operational parameter stored in the computer-readable memory, where the parameter is stored in normal-precision floating-point format, wherein the input tensor is a convolutional filter, and the quantized-precision floating-point format is a block floating-point format wherein mantissa values within a sub-division of a spatial pixel share a common exponent, and wherein mantissa values within different sub-divisions of the spatial pixel have different respective exponents, each sub-division comprising a subset of channels of the spatial pixel. receive an input tensor for a given layer of a multi-layer neural network; convert the input tensor from a normal-precision floating-point format to a quantized-precision floating-point format; perform a tensor operation using the input tensor converted to the quantized-precision floating-point format; convert a result of the tensor operation from the quantized-precision floating-point format to the normal-precision floating-point format; and use the converted result in the normal-precision floating-point format to update the operational parameter stored in the computer-readable memory, where the parameter is stored in normal-precision floating-point format, wherein the input tensor is a convolutional filter, and the quantized-precision floating-point format is a block floating-point format wherein mantissa values within a sub-division of a spatial pixel share a common exponent, and wherein mantissa values within different sub-divisions of the spatial pixel have different respective exponents, each sub-division comprising a subset of channels of the spatial pixel.