An apparatus, comprising: a first memory that holds rows of N weight words, wherein N is at least 512; a second memory that holds rows of N data words; an array of N processing units, PU, indexed as 0 through N-1, each PU of index J having: a register; an accumulator having an output; an arithmetic unit having first, second and third inputs and that performs an operation thereon to generate a result to store in the accumulator, the first input receives the output of the accumulator, the second input receives a respective weight word output by the first memory, the third input receives a respective data word output by the register, wherein the result corresponds to the sum of the output of the accumulator and the product of the weight word and the data word of the register; and multiplexing logic that receives a respective data word output by the second memory and that receives a data word output by the register of PU J-1 and that outputs a selected data word to the register of PU J, wherein PU J-1 for PU 0 is PU N-1; wherein the multiplexing logic of PU N/4 also receives the data word output by the register of PU (3N/4)-1; and wherein the multiplexing logic of PU 3N/4 also receives the data word output by the register of PU (N/4)-1. a first memory that holds rows of N weight words, wherein N is at least 512; a second memory that holds rows of N data words; an array of N processing units, PU, indexed as 0 through N-1, each PU of index J having: a register; an accumulator having an output; an arithmetic unit having first, second and third inputs and that performs an operation thereon to generate a result to store in the accumulator, the first input receives the output of the accumulator, the second input receives a respective weight word output by the first memory, the third input receives a respective data word output by the register, wherein the result corresponds to the sum of the output of the accumulator and the product of the weight word and the data word of the register; and multiplexing logic that receives a respective data word output by the second memory and that receives a data word output by the register of PU J-1 and that outputs a selected data word to the register of PU J, wherein PU J-1 for PU 0 is PU N-1; a register; an accumulator having an output; an arithmetic unit having first, second and third inputs and that performs an operation thereon to generate a result to store in the accumulator, the first input receives the output of the accumulator, the second input receives a respective weight word output by the first memory, the third input receives a respective data word output by the register, wherein the result corresponds to the sum of the output of the accumulator and the product of the weight word and the data word of the register; and multiplexing logic that receives a respective data word output by the second memory and that receives a data word output by the register of PU J-1 and that outputs a selected data word to the register of PU J, wherein PU J-1 for PU 0 is PU N-1; wherein the multiplexing logic of PU N/4 also receives the data word output by the register of PU (3N/4)-1; and wherein the multiplexing logic of PU 3N/4 also receives the data word output by the register of PU (N/4)-1.