A method for transposing a neural network matrix on a hardware circuit, comprising: iteratively performing the following operations on a matrix computation unit of a hardware circuit, wherein the neural network matrix is a weight matrix and wherein the hardware circuit performs matrix multiplication operations as a series of vector multiplication operations using the matrix computation unit, wherein the matrix computation unit is a two-dimensional systolic array, including multiple cells, and wherein on each iteration, each cell processes a given weight input to the cell and a given activation input to the cell to generate a product, and adds the product to an accumulated output from a first adjacent cell to generate an accumulated output, and pass the generated accumulated output to a second adjacent cell: dividing an intermediate neural network matrix as of an iteration into a plurality of submatrices in a current subdivision, creating a plurality of vectors, each of which corresponding to each row of the intermediate neural network matrix and including values of each row of the intermediate neural network matrix, obtaining, for the plurality of vectors, a plurality of partial identity matrices that are configured to extract a portion of values from each of the plurality of vectors while nullifying a remaining portion of values, when the plurality of partial identity matrices is multiplied by the plurality of vectors, multiplying, by one or more of the plurality of partial identity matrices, each of the plurality of vectors to generate a row of an updated neural network matrix, wherein the updated neural network matrix includes elements of the neural network matrix but with swapped upper right and lower left quadrants of the plurality of submatrices in the current subdivision, and combining the generated rows to update the intermediate neural network matrix; and generating, based on all iterations of the operations being completed, the intermediate neural network matrix as a transposed neural network matrix. dividing an intermediate neural network matrix as of an iteration into a plurality of submatrices in a current subdivision, creating a plurality of vectors, each of which corresponding to each row of the intermediate neural network matrix and including values of each row of the intermediate neural network matrix, obtaining, for the plurality of vectors, a plurality of partial identity matrices that are configured to extract a portion of values from each of the plurality of vectors while nullifying a remaining portion of values, when the plurality of partial identity matrices is multiplied by the plurality of vectors, multiplying, by one or more of the plurality of partial identity matrices, each of the plurality of vectors to generate a row of an updated neural network matrix, wherein the updated neural network matrix includes elements of the neural network matrix but with swapped upper right and lower left quadrants of the plurality of submatrices in the current subdivision, and combining the generated rows to update the intermediate neural network matrix; and generating, based on all iterations of the operations being completed, the intermediate neural network matrix as a transposed neural network matrix.