A computing system comprising multiple computing units for accelerating tensor computations of a neural network, each computing unit comprising: a first memory bank having a first data width for storing at least one of input activations or output activations; a second memory bank having a second data width that is larger than the first data width for storing one or more parameters used in performing computations; a direct memory access tensor traversal unit configured to: access at least one memory location of the first memory bank and at least one memory location of the second memory bank based on instructions received from a source that is external to the computing unit; and provide a control signal for writing the parameters to the second memory bank; and provide a control signal for writing the input activations to the first memory bank; at least one cell comprising at least one multiply accumulate, MAC, operator that receives parameters from the second memory bank and performs computations; a data bus accessible by the MAC operator; a tensor traversal unit in data communication with at least the first memory bank, the tensor traversal unit configured to provide a control signal to the first memory bank to cause an input activation to be provided to the data bus; a portion of a ring bus that extends outside of the computing unit, wherein the ring bus provides a data path between the first memory bank and a memory bank of another adjacent computing unit, and between the second memory bank and a memory bank of another adjacent computing unit, the ring bus originating from a controller and providing communications in a ring back to the controller; and wherein the computing unit performs one or more computations associated with at least one element of a data array, the one or more computations being performed by the MAC operator and comprising, in part, a multiply operation of the input activation received from the data bus and a parameter received from the second memory bank. a first memory bank having a first data width for storing at least one of input activations or output activations; a second memory bank having a second data width that is larger than the first data width for storing one or more parameters used in performing computations; a direct memory access tensor traversal unit configured to: access at least one memory location of the first memory bank and at least one memory location of the second memory bank based on instructions received from a source that is external to the computing unit; and provide a control signal for writing the parameters to the second memory bank; and provide a control signal for writing the input activations to the first memory bank; access at least one memory location of the first memory bank and at least one memory location of the second memory bank based on instructions received from a source that is external to the computing unit; and provide a control signal for writing the parameters to the second memory bank; and provide a control signal for writing the input activations to the first memory bank; at least one cell comprising at least one multiply accumulate, MAC, operator that receives parameters from the second memory bank and performs computations; a data bus accessible by the MAC operator; a tensor traversal unit in data communication with at least the first memory bank, the tensor traversal unit configured to provide a control signal to the first memory bank to cause an input activation to be provided to the data bus; a portion of a ring bus that extends outside of the computing unit, wherein the ring bus provides a data path between the first memory bank and a memory bank of another adjacent computing unit, and between the second memory bank and a memory bank of another adjacent computing unit, the ring bus originating from a controller and providing communications in a ring back to the controller; and wherein the computing unit performs one or more computations associated with at least one element of a data array, the one or more computations being performed by the MAC operator and comprising, in part, a multiply operation of the input activation received from the data bus and a parameter received from the second memory bank.