Patent ID: 11886812
Assignee: GRAMMARLY, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 7:
8. A method for training a grammatical error correction model, the method comprising:
inputting, to a digital model, a first dataset that comprises domain-independent training data that comprises a set of uncorrected text items and for an uncorrected text item, a corresponding corrected text item, and a second dataset that comprises in-domain training data that comprises a set of text sequences and, for a text sequence, a set of corresponding features, the set of corresponding features comprising a proficiency label and a native language label, wherein at least one type of error is present in the text sequences and the location of the error within the text sequences for the in-domain training data, and wherein the digital model comprises fluency-adjusted grammatical error correction model;
training the digital model to generate grammatically corrected and fluency-adjusted text sequences using an encoder-decoder neural network with an attention mechanism and at least one long term short term memory (LSTM) unit, including first training on the first dataset from text sequences of a plurality of different topics and writing styles received from by native and non-native speakers of various native backgrounds, fine-tuning training on the second dataset;
the digital model comprising a plurality of artificial neural network layers and model parameters associated with the artificial neural network layers, a value of a model parameter indicative of a relationship between the proficiency label, or a proficiency label-native language label combination, and the text sequence, and the corresponding corrected text item; and
fine-tuning the digital model using an in-domain data set comprising text sequences that have been labeled with native languages and proficiency levels to only adjust a subset of the values of the model parameters associated with an encoding layer or an embedding layer or both the encoding layer and the embedding layer, the subset comprising one or more of a number of epochs, batch size, learning rate, and start decay.