A method for performing domain adaptation of an end-to-end, E2E, automatic speech recognition, ASR, model, the method comprising: - obtaining an un-adapted version of the E2E ASR model that is trained using a first set of transcriptions, wherein the un-adapted version of the E2E ASR model comprises - an encoder network that receives acoustic signal frames (Z t) and outputs encoded acoustic signal frames (E(Z t)); - a first prediction network, wherein a given prediction network receives at least one previous output speech element (X i-1) and outputs a predicted output speech element, the predicted output speech element being predicted based on the at least one previous output speech element; and - a joint network that receives and processes the encoded acoustic signal frames and the predicted output speech element to output a conditional probability distribution P(x i | x i-1, Z t)) over a next output speech element; - using the first set of transcriptions, while keeping parameters of the first prediction network fixed, to train a language model output component to match the first prediction network, wherein the language model output component and a given prediction network collectively induce a language model; characterized by - using a second set of transcriptions, while keeping parameters of the language model output component fixed, to fine-tune the first prediction network for obtaining a second prediction network; and - generating an adapted version of the E2E ASR model, wherein the adapted version of the E2E ASR model comprises the encoder network, the second prediction network, the language model output component, and the joint network. - obtaining an un-adapted version of the E2E ASR model that is trained using a first set of transcriptions, wherein the un-adapted version of the E2E ASR model comprises - an encoder network that receives acoustic signal frames (Z t) and outputs encoded acoustic signal frames (E(Z t)); - a first prediction network, wherein a given prediction network receives at least one previous output speech element (X i-1) and outputs a predicted output speech element, the predicted output speech element being predicted based on the at least one previous output speech element; and - a joint network that receives and processes the encoded acoustic signal frames and the predicted output speech element to output a conditional probability distribution P(x i | x i-1, Z t)) over a next output speech element; - an encoder network that receives acoustic signal frames (Z t) and outputs encoded acoustic signal frames (E(Z t)); - a first prediction network, wherein a given prediction network receives at least one previous output speech element (X i-1) and outputs a predicted output speech element, the predicted output speech element being predicted based on the at least one previous output speech element; and - a joint network that receives and processes the encoded acoustic signal frames and the predicted output speech element to output a conditional probability distribution P(x i | x i-1, Z t)) over a next output speech element; - using the first set of transcriptions, while keeping parameters of the first prediction network fixed, to train a language model output component to match the first prediction network, wherein the language model output component and a given prediction network collectively induce a language model; characterized by - using a second set of transcriptions, while keeping parameters of the language model output component fixed, to fine-tune the first prediction network for obtaining a second prediction network; and - generating an adapted version of the E2E ASR model, wherein the adapted version of the E2E ASR model comprises the encoder network, the second prediction network, the language model output component, and the joint network.