A method for compressing a recurrent neural network - RNN-, wherein the RNN is implemented by one or more computers and comprises a plurality of recurrent layers (120,130), wherein each of the plurality of recurrent layers of the RNN is configured to, for each of a plurality of time steps, receive a respective layer input for the time step and process the layer input to generate a respective layer output for the time step, and wherein each recurrent layer has a respective recurrent weight matrix, W h, and a respective inter-layer weight matrix, W x, the method comprising, for one of the plurality of recurrent layers: generating a first compressed weight matrix, Z h l, and a projection matrix, P l, wherein the product of the first compressed weight matrix and the projection matrix is a lower-rank approximation of the recurrent weight matrix of the one of the plurality of recurrent layers; and generating a second compressed weight matrix, Z x l, based on the first compressed weight matrix, Z h l, and the projection matrix, P l, wherein the product of the second compressed weight matrix and the projection matrix is a lower-rank approximation of the inter-layer weight matrix of the one of the plurality of recurrent layers, the second compressed matrix being determined by solving a least squares problem: Z x l = arg min Y ‖ YP l − W x l ‖ F 2, where ∥X∥ F denotes the Frobenius norm of the matrix X; and re-configuring the RNN with the first and second compressed weight matrices by: replacing the respective recurrent weight matrix with the product of the first compressed weight matrix and the projection matrix; and replacing the respective inter-layer weight matrix with the product of the second compressed weight matrix and the projection matrix; and transmitting the weight matrices of the re-configured RNN and other data defining the configuration of the reconfigured RNN to a mobile device. wherein the RNN is implemented by one or more computers and comprises a plurality of recurrent layers (120,130), wherein each of the plurality of recurrent layers of the RNN is configured to, for each of a plurality of time steps, receive a respective layer input for the time step and process the layer input to generate a respective layer output for the time step, and wherein each recurrent layer has a respective recurrent weight matrix, W h, and a respective inter-layer weight matrix, W x, the method comprising, for one of the plurality of recurrent layers: generating a first compressed weight matrix, Z h l, and a projection matrix, P l, wherein the product of the first compressed weight matrix and the projection matrix is a lower-rank approximation of the recurrent weight matrix of the one of the plurality of recurrent layers; and generating a second compressed weight matrix, Z x l, based on the first compressed weight matrix, Z h l, and the projection matrix, P l, wherein the product of the second compressed weight matrix and the projection matrix is a lower-rank approximation of the inter-layer weight matrix of the one of the plurality of recurrent layers, the second compressed matrix being determined by solving a least squares problem: Z x l = arg min Y ‖ YP l − W x l ‖ F 2, where ∥X∥ F denotes the Frobenius norm of the matrix X; and re-configuring the RNN with the first and second compressed weight matrices by: replacing the respective recurrent weight matrix with the product of the first compressed weight matrix and the projection matrix; and replacing the respective inter-layer weight matrix with the product of the second compressed weight matrix and the projection matrix; and generating a first compressed weight matrix, Z h l, and a projection matrix, P l, wherein the product of the first compressed weight matrix and the projection matrix is a lower-rank approximation of the recurrent weight matrix of the one of the plurality of recurrent layers; and generating a second compressed weight matrix, Z x l, based on the first compressed weight matrix, Z h l, and the projection matrix, P l, wherein the product of the second compressed weight matrix and the projection matrix is a lower-rank approximation of the inter-layer weight matrix of the one of the plurality of recurrent layers, the second compressed matrix being determined by solving a least squares problem: Z x l = arg min Y ‖ YP l − W x l ‖ F 2, where ∥X∥ F denotes the Frobenius norm of the matrix X; and re-configuring the RNN with the first and second compressed weight matrices by: replacing the respective recurrent weight matrix with the product of the first compressed weight matrix and the projection matrix; and replacing the respective inter-layer weight matrix with the product of the second compressed weight matrix and the projection matrix; and replacing the respective recurrent weight matrix with the product of the first compressed weight matrix and the projection matrix; and replacing the respective inter-layer weight matrix with the product of the second compressed weight matrix and the projection matrix; and transmitting the weight matrices of the re-configured RNN and other data defining the configuration of the reconfigured RNN to a mobile device.