Grammatical error correction (GEC) literature has reported on the effectiveness of pretraining a Seq2Seq model with a large amount of pseudo data. In this study, we explored two generic pretrained encoder-decoder (Enc-Dec) models, including BART, which reported the state-of-the-art (SOTA) results for several Seq2Seq tasks other than GEC. We found that monolingual and multilingual BART models achieve high performance in GEC, including a competitive result compared with the current SOTA result in English GEC. Our implementations will be publicly available at GitHub .
Studies on grammatical error correction (GEC) have reported the effectiveness of pretraining a Seq2Seq model with a large amount of pseudodata. However, this approach requires time-consuming pretraining for GEC because of the size of the pseudodata. In this study, we explore the utility of bidirectional and auto-regressive transformers (BART) as a generic pretrained encoder-decoder model for GEC. With the use of this generic pretrained model for GEC, the time-consuming pretraining can be eliminated. We find that monolingual and multilingual BART models achieve high performance in GEC, with one of the results being comparable to the current strong results in English GEC. Our implementations are publicly available at GitHub URL
R
Grammatical
Studies on grammatical
R
literature has reported on
have reported
R
pseudo data.
pseudodata. However, this approach requires time-consuming pretraining for GEC because of the size of the pseudodata.
R
explored two
explore the utility of bidirectional and auto-regressive transformers (BART) as a
R
(Enc-Dec) models, including BART, which reported the state-of-the-art (SOTA) results for several Seq2Seq tasks other than GEC. We found
model for GEC. With the use of this generic pretrained model for GEC, the time-consuming pretraining can be eliminated. We find
R
including a competitive result compared with the current SOTA result
with one of the results being comparable to the current strong results
R
will be
are
R
.
URL