A computer-implemented method of regularizing the training of a neural network, wherein the neural network is configured to receive an input data item and to process the input data item to generate an output comprising a respective score for each label in a predetermined set of multiple labels, wherein either: the input data item corresponds to an image or features that have been extracted from an image, the set of multiple labels corresponds to a set of object categories, and the output generated by the neural network for the input data item are scores for each of the set of object categories, with each score representing an estimated likelihood that the image contains an image of an object belonging to the category, or the input data item corresponds to features of a spoken utterance, the set of multiple labels corresponds to a set of pieces of text, and the output generated by the neural network for the input data item are scores for each of the set of pieces of text, each score representing an estimated likelihood that the piece of text is the correct transcript for the spoken utterance, the method of regularizing the training of the neural network comprising: obtaining a set of training data that includes a plurality of training items, wherein each training item is associated with an initial target label distribution that assigns a respective target score to each label in the predetermined set of multiple labels; and modifying the training data to generate regularizing training data that regularizes the training of the neural network, comprising: for each training item, generating a modified target label distribution by combining the initial target label distribution with a smoothing label distribution, wherein the smoothing label distribution is independent of the training item; and training the neural network on the regularizing training data. the input data item corresponds to an image or features that have been extracted from an image, the set of multiple labels corresponds to a set of object categories, and the output generated by the neural network for the input data item are scores for each of the set of object categories, with each score representing an estimated likelihood that the image contains an image of an object belonging to the category, or the input data item corresponds to features of a spoken utterance, the set of multiple labels corresponds to a set of pieces of text, and the output generated by the neural network for the input data item are scores for each of the set of pieces of text, each score representing an estimated likelihood that the piece of text is the correct transcript for the spoken utterance, the method of regularizing the training of the neural network comprising: obtaining a set of training data that includes a plurality of training items, wherein each training item is associated with an initial target label distribution that assigns a respective target score to each label in the predetermined set of multiple labels; and modifying the training data to generate regularizing training data that regularizes the training of the neural network, comprising: for each training item, generating a modified target label distribution by combining the initial target label distribution with a smoothing label distribution, wherein the smoothing label distribution is independent of the training item; and training the neural network on the regularizing training data. obtaining a set of training data that includes a plurality of training items, wherein each training item is associated with an initial target label distribution that assigns a respective target score to each label in the predetermined set of multiple labels; and modifying the training data to generate regularizing training data that regularizes the training of the neural network, comprising: for each training item, generating a modified target label distribution by combining the initial target label distribution with a smoothing label distribution, wherein the smoothing label distribution is independent of the training item; and training the neural network on the regularizing training data.