A computer-implemented method of training a data compression system comprising an encoder neural network, a decoder neural network and a memory storing a plurality of latent embedding vectors, the method comprising: receiving a training data item; processing the training data item through the encoder neural network in accordance with current values of the encoder network parameters of the encoder neural network to generate a training encoder output that comprises, for each of one more latent variables, a respective training encoded vector; selecting, for each latent variable and from the plurality of latent embedding vectors currently stored in the memory, a current latent embedding vector that is nearest to the training encoded vector for the latent variable, for providing a discrete latent representation as a compressed representation of the training data item, wherein the discrete latent representation comprises, for each latent variable, an identifier of the nearest latent embedding vector to the training encoded vector for the latent variable; generating a training decoder input that includes the nearest current latent embedding vectors; processing the training decoder input through the decoder neural network in accordance with current values of the decoder network parameters of the decoder neural network to generate a training reconstruction of the training data item; determining a reconstruction update to the current values of the decoder network parameters and the encoder network parameters by determining a gradient with respect to the current values of the decoder network parameters and the encoder network parameters to optimize a reconstruction error between the training reconstruction and the training data item; wherein determining the gradient with respect to the current values of the encoder network parameters to optimize the reconstruction error comprises backpropagating gradients into the encoder from the decoder by: copying gradients from the decoder input to the encoder output without updating the current latent embedding vectors; and for each latent variable, determining an update to the nearest current latent embedding vector for the latent variable by determining a gradient with respect to the nearest current latent embedding vector to minimize an error between the training encoded vector for the latent variable and the nearest current latent embedding vector to the training encoded vector for the latent variable; or determining an update to the current latent embedding vectors as a function of moving averages of the training encoded vectors in the training encoder outputs. receiving a training data item; processing the training data item through the encoder neural network in accordance with current values of the encoder network parameters of the encoder neural network to generate a training encoder output that comprises, for each of one more latent variables, a respective training encoded vector; selecting, for each latent variable and from the plurality of latent embedding vectors currently stored in the memory, a current latent embedding vector that is nearest to the training encoded vector for the latent variable, for providing a discrete latent representation as a compressed representation of the training data item, wherein the discrete latent representation comprises, for each latent variable, an identifier of the nearest latent embedding vector to the training encoded vector for the latent variable; generating a training decoder input that includes the nearest current latent embedding vectors; processing the training decoder input through the decoder neural network in accordance with current values of the decoder network parameters of the decoder neural network to generate a training reconstruction of the training data item; determining a reconstruction update to the current values of the decoder network parameters and the encoder network parameters by determining a gradient with respect to the current values of the decoder network parameters and the encoder network parameters to optimize a reconstruction error between the training reconstruction and the training data item; wherein determining the gradient with respect to the current values of the encoder network parameters to optimize the reconstruction error comprises backpropagating gradients into the encoder from the decoder by: copying gradients from the decoder input to the encoder output without copying gradients from the decoder input to the encoder output without updating the current latent embedding vectors; and for each latent variable, determining an update to the nearest current latent embedding vector for the latent variable by determining a gradient with respect to the nearest current latent embedding vector to minimize an error between the training encoded vector for the latent variable and the nearest current latent embedding vector to the training encoded vector for the latent variable; or determining an update to the current latent embedding vectors as a function of moving averages of the training encoded vectors in the training encoder outputs.