An apparatus, comprising: a plurality of arithmetic logic units each having: an accumulator; and an integer arithmetic unit that receives and performs integer arithmetic operations on integer inputs and accumulates integer results of a series of the integer arithmetic operations into the accumulator as an integer accumulated value; a register programmable with: an indication of a number of fractional bits of the integer accumulated values; and an indication of a number of fractional bits of integer outputs; a first bit width of the accumulator is greater than twice a second bit width of the integer outputs; and a plurality of activation function units that scale and saturate the first bit width integer accumulated values to generate the second bit width integer outputs based on the indications of the number of fractional bits of the integer accumulated values and outputs programmed into the register, and that perform activation functions on the scaled and saturated integer accumulated values to normalize them in a non-linear fashion to produce results that are within predetermined ranges of values; wherein: the apparatus is an integrated circuit; each of the plurality of arithmetic logic units comprises: an integer multiplier that performs an integer multiplication of a pair of the integer inputs to generate an integer product; and an integer adder that performs an integer addition of the integer product and the integer accumulated value to generate an integer sum for storage back into the accumulator. a plurality of arithmetic logic units each having: an accumulator; and an integer arithmetic unit that receives and performs integer arithmetic operations on integer inputs and accumulates integer results of a series of the integer arithmetic operations into the accumulator as an integer accumulated value; an accumulator; and an integer arithmetic unit that receives and performs integer arithmetic operations on integer inputs and accumulates integer results of a series of the integer arithmetic operations into the accumulator as an integer accumulated value; a register programmable with: an indication of a number of fractional bits of the integer accumulated values; and an indication of a number of fractional bits of integer outputs; a first bit width of the accumulator is greater than twice a second bit width of the integer outputs; and an indication of a number of fractional bits of the integer accumulated values; and an indication of a number of fractional bits of integer outputs; a first bit width of the accumulator is greater than twice a second bit width of the integer outputs; and a plurality of activation function units that scale and saturate the first bit width integer accumulated values to generate the second bit width integer outputs based on the indications of the number of fractional bits of the integer accumulated values and outputs programmed into the register, and that perform activation functions on the scaled and saturated integer accumulated values to normalize them in a non-linear fashion to produce results that are within predetermined ranges of values; the apparatus is an integrated circuit; each of the plurality of arithmetic logic units comprises: an integer multiplier that performs an integer multiplication of a pair of the integer inputs to generate an integer product; and an integer adder that performs an integer addition of the integer product and the integer accumulated value to generate an integer sum for storage back into the accumulator. an integer multiplier that performs an integer multiplication of a pair of the integer inputs to generate an integer product; and an integer adder that performs an integer addition of the integer product and the integer accumulated value to generate an integer sum for storage back into the accumulator.