A computing system comprising: one or more processors (120, 122, 180); memory comprising computer-readable storage devices and/or memory; an outlier quantizer (186, 765) formed from at least one of the processors, the outlier quantizer being in communication with the memory; the computing system being configured to: with at least one of the processors, produce first activation values of a neural network (810, 1020) in a first floating-point format, with the outlier quantizer, convert one or more of the first activation values to an outlier block floating-point format, producing second activation values (850, 860) comprising: second values (850, 1061) in a second, block floating-point format for all of the first activation values, and outlier values (860, 1071) for at least one but not all of the first activation values, and with at least one of the processors, storing the second activation values and the outlier values in the memory; and the outlier quantizer comprises a shifter configured to shift mantissas of the first activation values according to a shared exponent selected for the second values; a portion of each of the shifted first activation values forms a mantissa for a respective one of the second values; and a different portion of the shifted first activation values forms an outlier mantissa for the second activation values having an outlier value. one or more processors (120, 122, 180); memory comprising computer-readable storage devices and/or memory; an outlier quantizer (186, 765) formed from at least one of the processors, the outlier quantizer being in communication with the memory; the computing system being configured to: with at least one of the processors, produce first activation values of a neural network (810, 1020) in a first floating-point format, with the outlier quantizer, convert one or more of the first activation values to an outlier block floating-point format, producing second activation values (850, 860) comprising: second values (850, 1061) in a second, block floating-point format for all of the first activation values, and outlier values (860, 1071) for at least one but not all of the first activation values, and with at least one of the processors, produce first activation values of a neural network (810, 1020) in a first floating-point format, with the outlier quantizer, convert one or more of the first activation values to an outlier block floating-point format, producing second activation values (850, 860) comprising: second values (850, 1061) in a second, block floating-point format for all of the first activation values, and outlier values (860, 1071) for at least one but not all of the first activation values, and with at least one of the processors, storing the second activation values and the outlier values in the memory; and the outlier quantizer comprises a shifter configured to shift mantissas of the first activation values according to a shared exponent selected for the second values; a portion of each of the shifted first activation values forms a mantissa for a respective one of the second values; and a different portion of the shifted first activation values forms an outlier mantissa for the second activation values having an outlier value.