Original: E FFECT OF T OKENISATION S TRATEGIES FOR L OW -R ESOURCED S OUTHERN A FRICAN L ANGUAGES
Revised: E FFECT OF T OKENISATION S TRATEGIES FOR L OW -R ESOURCED S OUTHERN A FRICAN L ANGUAGES

Original: A BSTRACT
Revised: A BSTRACT

Original: Research into machine translation for African languages is very limited and lowresourced in terms of datasets and model evaluations.
Revised: Research into machine translation for African languages is very limited and lowresourced in terms of datasets and model evaluations.

Original: This work aims to add to the ﬁeld of neural machine translation research, for four low-resourced Southern
Revised: This work aims to add to the ﬁeld of neural machine translation research, for four low-resourced Southern

Original: African languages.
Revised: African languages.

Original: The effect of two byte pair encoding tokenisation algorithms (subword nmt and SentencePiece), with various parameters, are evaluated.
Revised: The effect of two byte pair encoding tokenisation algorithms (subword nmt and SentencePiece), with various parameters, are evaluated.

Original: The paper builds upon previous research in the ﬁeld for comparison, using an optimised transformer architecture and pre-cleaned data to translate English to Northern Sotho, Setswana, Xitsonga and isiZulu.
Revised: The paper builds upon previous research in the ﬁeld for comparison, using an optimised transformer architecture and pre-cleaned data to translate English to Northern Sotho, Setswana, Xitsonga and isiZulu.

Original: The results obtained show improvements in the previous BLEU scores obtained for Setswana and isiZulu.
Revised: The results obtained show improvements in the previous BLEU scores obtained for Setswana and isiZulu.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: There has been a slight increase in the focus of machine translation for low-resourced languages in recent years, most notably in African languages through the collaboration of African curators, annotators and language technologists (Abbott, 2021).
Revised: There has been a slight increase in the focus of machine translation for low-resourced languages in recent years, most notably in African languages through the collaboration of African curators, annotators and language technologists (Abbott, 2021).

Original: However there is still a large amount of work to be done in the ﬁeld, with varying approaches and optimal models to be tested in the setting of low and unconsolidated resources.
Revised: However there is still a large amount of work to be done in the ﬁeld, with varying approaches and optimal models to be tested in the setting of low and unconsolidated resources.

Original: This research evaluates two different Byte Pair Encoding (BPE) algorithms (subword nmt and SentencePiece) using an optimised transformer architecture.
Revised: This research evaluates two different Byte Pair Encoding (BPE) algorithms (subword nmt and SentencePiece) using an optimised transformer architecture.

Original: The models are trained on four agglutinative (Zerbian, 2007)
Revised: The models are trained on four agglutinative (Zerbian, 2007)

Original: Southern African Bantu languages (Mesthrie, 2002), namely Northern Sotho,
Revised: Southern African Bantu languages (Mesthrie, 2002), namely Northern Sotho,

Original: Setswana, Xitsonga and isiZulu.
Revised: Setswana, Xitsonga and isiZulu.

Original: The work builds directly on research, into Neural Machine Translation (NMT) for low-resourced African Languages, by Martinus & Abbott (2019) and Biljon et al.
Revised: The work builds directly on research, into Neural Machine Translation (NMT) for low-resourced African Languages, by Martinus & Abbott (2019) and Biljon et al.

Original: The results show increased performance for Setswana and isiZulu when using SentencePiece
Revised: The results show increased performance for Setswana and isiZulu when using SentencePiece

Original: BPE tokenisation, in comparison to the previous work on the same dataset.
Revised: BPE tokenisation, in comparison to the previous work on the same dataset.

Original: Section 2 describes previous related work and the inspiration for this research.
Revised: Section 2 describes previous related work and the inspiration for this research.

Original: The methodology including the datasets, tokenisation strategies and trained models are described in Section 3.
Revised: The methodology including the datasets, tokenisation strategies and trained models are described in Section 3.

Original: The results are presented and analysed in Section 4 and 5 respectively.
Revised: The results are presented and analysed in Section 4 and 5 respectively.

Original: A impact statement for this work is outlined in Section 6, followed by the Conclusion.
Revised: A impact statement for this work is outlined in Section 6, followed by the Conclusion.

Original:  B ACKGROUND
Revised:  B ACKGROUND

Original: In 2019, Martinus & Abbott (2019) wrote a position piece on the problems facing NMT for lowresourced African languages.
Revised: In 2019, Martinus & Abbott (2019) wrote a position piece on the problems facing NMT for lowresourced African languages.

Original: They highlighted a number of problem areas, including lack of research into adequate models.
Revised: They highlighted a number of problem areas, including lack of research into adequate models.

Original: To address this, their research provided results from the implementation of NMT models, using the transformer architecture, for ﬁve Southern African languages from the Autshumato dataset (Groenewald & Fourie, 2009).
Revised: To address this, their research provided results from the implementation of NMT models, using the transformer architecture, for ﬁve Southern African languages from the Autshumato dataset (Groenewald & Fourie, 2009).

Original: Additionally, they provided an ablation study on the BPE token size, for each of the languages, using the subword nmt algorithm.
Revised: Additionally, they provided an ablation study on the BPE token size, for each of the languages, using the subword nmt algorithm.

Original: In the ablation study they varied the token size (5k - 40k) implemented, showing that it directly affected the model’s ﬁnal performance, and determined an optimal token size for each language.
Revised: In the ablation study they varied the token size (5k - 40k) implemented, showing that it directly affected the model’s ﬁnal performance, and determined an optimal token size for each language.

Original: These results are discussed in Section 3.2.1 .
Revised: These results are discussed in Section 3.2.1 .

Original: Following this, Biljon et al.
Revised: Following this, Biljon et al.

Original: (2020) expanded on the above research by testing the optimal transformer depth for use on low-resourced languages.
Revised: (2020) expanded on the above research by testing the optimal transformer depth for use on low-resourced languages.

Original: Making use of the same dataset, but only analysing a subset of the ﬁve languages, they determined that medium-depth transformers worked optimally for the low-resource language translation models; and provided their open source codealong with the implementation results.
Revised: Making use of the same dataset, but only analysing a subset of the ﬁve languages, they determined that medium-depth transformers worked optimally for the low-resource language translation models; and provided their open source codealong with the implementation results.

Original: Both of these studies made use of BPE tokenisation, speciﬁcally using the subword nmt BPE algorithm.
Revised: Both of these studies made use of BPE tokenisation, speciﬁcally using the subword nmt BPE algorithm.

Original: BPE has shown good results when used for low-resourced languages as it helps with rare and out-of-vocabulary words, which is common in the low-resource setting (Sennrich et al., 2015).
Revised: BPE has shown good results when used for low-resourced languages as it helps with rare and out-of-vocabulary words, which is common in the low-resource setting (Sennrich et al., 2015).

Original: The objective of this study is to further expand on the work of Martinus & Abbott (2019) and Biljon et al.
Revised: The objective of this study is to further expand on the work of Martinus & Abbott (2019) and Biljon et al.

Original: (2020) in determining if the SentencePiece BPE tokenisation strategy will assist performance for these low-resourced languages.
Revised: (2020), in determining if the SentencePiece BPE tokenisation strategy will assist performance for these low-resourced languages.

Original: SentencePiece BPE encodes white space and therefore does not require the words in the corpus to be white-space separated.
Revised: SentencePiece BPE encodes white space and therefore does not require the words in the corpus to be white-space separated.

Original: Since the Bantu languages tested are agglutinative, by not requiring the text to be ‘pre-tokenised’ it could improve translation performance (Zerbian, 2007)
Revised: Since the Bantu languages tested are agglutinative, by not requiring the text to be ‘pre-tokenised’ it could improve translation performance (Zerbian, 2007)

Original: (Horan).
Revised: (Horan).

Original: Currently, to the best of the author’s knowledge, an in-depth comparison has not be done between subword nmt BPE and SentencePiece BPE tokenisation on African languages, therefore this research aims to support further expansion of African Natural Language Processing.
Revised: Currently, to the best of the author’s knowledge, an in-depth comparison has not be done between subword nmt BPE and SentencePiece BPE tokenisation on African languages, therefore this research aims to support further expansion of African Natural Language Processing.

Original:  M ETHODOLOGY
Revised:  M ETHODOLOGY

Original: In order for comparability to the previous work described, the pre-processed Autshumato (Groenewald & Fourie, 2009) dataset from Martinus & Abbott (2019) and the open-source optimised transformer implementation by Biljon et al.
Revised: In order for comparability to the previous work described, the pre-processed Autshumato (Groenewald & Fourie, 2009) dataset from Martinus & Abbott (2019) and the open-source optimised transformer implementation by Biljon et al.

Original: (2020) is used to train the NMT models for each language.
Revised: (2020) is used to train the NMT models for each language.

Original: The data processing, data splits, transformer architecture and hyper-parameterin settings in Biljon et al.
Revised: The data processing, data splits, transformer architecture and hyper-parameter settings in Biljon et al.

Original: (2020) are kept the same for direct result comparisons.
Revised: (2020) are kept the same for direct result comparisons.

Original: Only the tokenisation strategy has been modiﬁed to include the SentencePiece BPE algorithm and is compared to the existing subword nmt BPE implementation.
Revised: Only the tokenisation strategy has been modiﬁed to include the SentencePiece BPE algorithm and is compared to the existing subword nmt BPE implementation.

Original: The models are trained to perform English to Target Language translation for the following Bantu languages:
Revised: The models are trained to perform English to Target Language translation for the following Bantu languages:

Original: • Northern Sotho •
Revised: • Northern Sotho •

Original: Setswana • Xitsonga • isiZulu
Revised: Setswana • Xitsonga • isiZulu

Original: The SentencePiece BPE implementation is tested using various vocabulary sizes (4k, 8k and 16k), while the subword nmt algorithm is evaluated using only the optimal token size for each language respectively (determined from the ablation study by Martinus & Abbott (2019)).
Revised: The SentencePiece BPE implementation is tested using various vocabulary sizes (4k, 8k and 16k), while the subword nmt algorithm is evaluated using only the optimal token size for each language respectively (determined from the ablation study by Martinus & Abbott (2019)).

Original: An overview of the described methodology is shown in Figure 1.
Revised: An overview of the described methodology is shown in Figure 1.

Original: The code for the below experiments have been published on GitHub ( https://github.com/JenaleaR/Tokenisation_
Revised: The code for the below experiments have been published on GitHub ( https://github.com/JenaleaR/Tokenisation_

Original: African_Languages ) and have been made open-source to promote reproducibility, and enable further research.
Revised: African_Languages ) and have been made open-source to promote reproducibility, and enable further research.

Original: 3. L ANGUAGES
Revised: 3. L ANGUAGES

Original: The target languages evaluated are agglutinative Southern Bantu African languages (Mesthrie, 2002)(Zerbian, 2007).
Revised: The target languages evaluated are agglutinative Southern Bantu African languages (Mesthrie, 2002)(Zerbian, 2007).

Original: All the languages are similar in structure and vocabulary (Herbert & Bailey, 2002), with a subject-verb-object word order and a rich noun class system Zerbian (2007).
Revised: All the languages are similar in structure and vocabulary (Herbert & Bailey, 2002), with a subject-verb-object word order and a rich noun class system Zerbian (2007).

Original: Additionally the Bantu languages share root words (Mills, 2005), with Setswana and Northern Sotho being mutually-intelligible (Martinus & Abbott, 2019).
Revised: Additionally the Bantu languages share root words (Mills, 2005), with Setswana and Northern Sotho being mutually-intelligible (Martinus & Abbott, 2019).

Original: 3. D ATA
Revised: 3. D ATA

Original: The Autshumato dataset is a public parallel corpora of South African government data created by
Revised: The Autshumato dataset is a public parallel corpora of South African government data created by

Original: Groenewald & Fourie (2009) for NMT systems.
Revised: Groenewald & Fourie (2009) for NMT systems.

Original: The dataset contains aligned sentences for the four languages trained on (Martinus & Abbott, 2019).
Revised: The dataset contains aligned sentences for the four languages trained on (Martinus & Abbott, 2019).

Original: The dataset was pre-processed by Martinus & Abbott (2019), where duplicate sentences were found and removed along with their translations.
Revised: The dataset was pre-processed by Martinus & Abbott (2019), where duplicate sentences were found and removed along with their translations.

Original: Biljon et al.
Revised: Biljon et al.

Original: (2020) further processed the data, removing translation duplicates as well as ‘almost duplicates’, and ensured test data was ﬁltered from the training and dev sets to avoid data leakage.
Revised: (2020) further processed the data, removing translation duplicates as well as ‘almost duplicates’, and ensured test data was ﬁltered from the training and dev sets to avoid data leakage.

Original: 3000 parallel sentences were used for testing, the data was shufﬂed to remove bias and 1000 parallel sentences were set aside for validation, with the rest being used for training the models.
Revised: 3000 parallel sentences were used for testing, the data was shufﬂed to remove bias and 1000 parallel sentences were set aside for validation, with the rest being used for training the models.

Original: The summary statistics of the post-processed sentences per language and their respective splits are shown in Table 1.
Revised: The summary statistics of the post-processed sentences per language and their respective splits are shown in Table 1.

Original: [Table]
Revised: [Table]

Original: 3.2. S UBWORD BPE
Revised: 3.2. S UBWORD BPE

Original: Subword nmt is a BPE tokenisation method that is deﬁned by the number of merges (Sennrich et al., 2015).
Revised: Subword nmt is a BPE tokenisation method that is deﬁned by the number of merges (Sennrich et al., 2015).

Original: The algorithm separates the corpus by white space ‘into words’, counts the all pairs of adjacent characters, merges the characters of the most frequent pair and adds it to the vocabulary (Jurafsky & Martin, 2020).
Revised: The algorithm separates the corpus by white space ‘into words’, counts the all pairs of adjacent characters, merges the characters of the most frequent pair and adds it to the vocabulary (Jurafsky & Martin, 2020).

Original: This process is repeated for the deﬁned number of merges/tokens (Jurafsky & Martin, 2020).
Revised: This process is repeated for the deﬁned number of merges/tokens (Jurafsky & Martin, 2020).

Original: In the work by Martinus & Abbott (2019) they noted initial experimentation showed that the merge/token number affected the model’s ﬁnal performance.
Revised: In the work by Martinus & Abbott (2019) they noted, in initial experimentation, that the merge/token number affected the model’s ﬁnal performance.

Original: To test this they performed an ablation study using subword nmt and showed the optimal number of tokens for each language.
Revised: To test this they performed an ablation study using subword nmt and showed the optimal number of tokens for each language.

Original: Their results are listed in Table 2.
Revised: Their results are listed in Table 2.

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table][Figure] [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: Based on these results, the models in this research were trained using subword nmt tokenisation, with the optimal token number per language, to provide a baseline evaluation before training the models using SentancePiece BPE tokenisation.
Revised: Based on these results, the models in this research were trained using subword nmt tokenisation, with the optimal token number per language, to provide a baseline evaluation before training the models using SentancePiece BPE tokenisation.

Original: 3.2. S ENTENCE P IECE BPE
Revised: 3.2. S ENTENCE P IECE BPE

Original: Subword nmt requires the input text to be word tokenised in order to perform the merge operations, however since not all languages are space segmented between words this process could hinder the training and performance of NMT models, speciﬁcally on agglutinative languages (Horan).
Revised: Subword nmt requires the input text to be word tokenised in order to perform the merge operations, however since not all languages are space segmented between words this process could hinder the training and performance of NMT models, speciﬁcally on agglutinative languages (Horan).

Original: The
Revised: The

Original: SentencePiece BPE method by Kudo & Richardson (2018), addresses this problem by treating the input text as a Unicode character sequence, including the white spaces (Horan).
Revised: SentencePiece BPE method by Kudo & Richardson (2018), addresses this problem by treating the input text as a Unicode character sequence, including the white spaces (Horan).

Original: White spaces are encoded with the meta symbol ‘ ’, as part of a token, which assists in decoding and makes the algorithm language agnostic (Horan).
Revised: White spaces are encoded with the meta symbol ‘ ’, as part of a token, which assists in decoding and makes the algorithm language agnostic (Horan).

Original: SentencePiece BPE uses the ﬁnal vocabulary size as its parameter, differing from subword nmt which requires the deﬁned number of merge operations (Kudo & Richardson, 2018), this is done since SentencePiece also supports other segmentation algorithms (e.g. Unigram, Char and Word) (Kudo & Richardson, 2018).
Revised: SentencePiece BPE uses the ﬁnal vocabulary size as its parameter, differing from subword nmt which requires the deﬁned number of merge operations (Kudo & Richardson, 2018), this is done since SentencePiece also supports other segmentation algorithms (e.g. Unigram, Char and Word) (Kudo & Richardson, 2018).

Original: The models are trained using the SentencePiece BPE tokenisation algorithm with varying vocabulary sizes (4k,8k and 16k), to determine if encoding the white space could improve performance for the agglutinative languages tested.
Revised: The models were trained using the SentencePiece BPE tokenisation algorithm with varying vocabulary sizes (4k,8k and 16k), to determine if encoding the white space could improve performance for the agglutinative languages tested.

Original: 3. A LGORITHM
Revised: 3. A LGORITHM

Original: The medium-depth transformer architecture from Biljon et al.
Revised: The medium-depth transformer architecture from Biljon et al.

Original: (2020) was used to train the translation models for each language, using the different tokenisation strategies.
Revised: (2020) was used to train the translation models for each language, using the different tokenisation strategies.

Original: The tuned hyper-parameters where kept exactly as they had deﬁned them.
Revised: The tuned hyper-parameters where kept exactly as they had deﬁned them.

Original: The medium transformer consists of 6 transformer layers (3 encoder and 3 decoder) with the learning rate set to 0.0003 and a batch token size of 4096.
Revised: The medium transformer consists of 6 transformer layers (3 encoder and 3 decoder) with the learning rate set to 0.0003 and a batch token size of 4096.

Original: Beam search with a width of 5 was used to decode the test data.
Revised: Beam search with a width of 5 was used to decode the test data.

Original: The transformer was implemented using the JoeyNMT toolkit, which is based on PyTorch and provides NMT features with simple adaptable code (Kreutzer et al., 2019).
Revised: The transformer was implemented using the JoeyNMT toolkit, which is based on PyTorch and provides NMT features with simple adaptable code (Kreutzer et al., 2019).

Original: JoeyNMT supports BPE subword encoding learned with subword nmt as well as SentencePiece (Kreutzer et al., 2019).
Revised: JoeyNMT supports BPE subword encoding learned with subword nmt as well as SentencePiece (Kreutzer et al., 2019).

Original: A model for each language was ﬁrst trained and tested using the subword nmt algorithm, with the respective optimal token size, in order to get a baseline comparison.
Revised: A model for each language was ﬁrst trained and tested using the subword nmt algorithm, with the respective optimal token size, in order to get a baseline comparison.

Original: The SentencePiece BPE algorithm was then implemented and language models were trained for each vocabulary size (4k, 8k and 16k).
Revised: The SentencePiece BPE algorithm was then implemented and language models were trained for each vocabulary size (4k, 8k and 16k).

Original: Each model was trained for 30 epochs in order to determine the effect of each conﬁguration, while limiting the computation time.
Revised: Each model was trained for 30 epochs in order to determine the effect of each conﬁguration, while limiting the computation time.

Original: Following the initial results, in order to further test the performance seen by using SentencePiece BPE tokenisation, the models were trained for 100 epochs using the optimal vocabulary size noted for each language.
Revised: Following the initial results, in order to further test the performance seen by using SentencePiece BPE tokenisation, the models were trained for 100 epochs using the optimal vocabulary size determined for each language.

Original:  R ESULTS
Revised:  R ESULTS

Original: The initial BLEU results after 30 epochs are shown in Table 3.
Revised: The initial BLEU results after 30 epochs are shown in Table 3.

Original: It is seen that the SentencePiece BPE (sp bpe) achieved very good results across all languages, surpassing the subword nmt (sw bpe) performance for the initial tests.
Revised: It is seen that the SentencePiece BPE (sp bpe) achieved very good results across all languages, surpassing the subword nmt (sw bpe) performance for the initial tests.

Original: The optimal results for each language are achieved for different vocabulary sizes, which aligns with Martinus & Abbott (2019) ﬁndings that the token number affects the model performance.
Revised: The optimal results for each language are achieved for different vocabulary sizes, which aligns with Martinus & Abbott (2019) ﬁndings that the token number affects the model performance.

Original: The models for Northern Sotho, Setswana and isiZulu achieved higher
Revised: The models for Northern Sotho, Setswana and isiZulu achieved higher

Original: BLEU scores for lower vocabulary sizes, while the model for Xitsonga (with the largest dataset) shows higher BLEU scores as the vocabulary size increases.
Revised: BLEU scores for lower vocabulary sizes, while the model for Xitsonga (with the largest dataset) shows higher BLEU scores as the vocabulary size increases.

Original: [Table]
Revised: [Table]

Original: Following the initial tests, the models were retrained for 100 epochs using the optimal vocabulary size determined.
Revised: Following the initial tests, the models were retrained for 100 epochs using the optimal vocabulary size determined.

Original: The BLEU results after 100 epochs are shown in Table 4 with a comparison to the results achieved by Biljon et al.
Revised: The BLEU results after 100 epochs are shown in Table 4 with a comparison to the results achieved by Biljon et al.

Original: and Martinus & Abbott (2019).
Revised: and Martinus & Abbott (2019).

Original: It is seen that the Northern
Revised: It is seen that the Northern

Original: Sotho and Setswana BLEU results surpass those achieved by Biljon et al.
Revised: Sotho and Setswana BLEU results surpass those achieved by Biljon et al.

Original: (2020), with the result for Setswana surpassing the performance seen in Martinus & Abbott (2019) as well.
Revised: (2020), with the result for Setswana surpassing the performance seen in Martinus & Abbott (2019) as well.

Original: A four fold increase in the baseline isiZulu performance by Martinus & Abbott (2019) is also achieved, using the SentencePiece BPE method with a vocabulary size of 4000.
Revised: A four fold increase in the baseline isiZulu performance by Martinus & Abbott (2019) is also achieved, using the SentencePiece BPE method with a vocabulary size of 4000.

Original: The Xitsongamodel and Northern Sotho models were unable to achieve the performance seen in Martinus & Abbott (2019), with the implemented vocabulary sizes.
Revised: The Xitsonga and Northern Sotho models, with the implemented vocabulary sizes, were unable to achieve the performance seen in Martinus & Abbott (2019).

Original: A higher BLEU result was expected for Northern Sotho since  is closely related to Setswana (which achieved good results) as they are linguistically similar (Martinus & Abbott, 2019) and further investigation into the optimal vocabulary size needs to be done.
Revised: A higher BLEU result was expected for Northern Sotho since it is closely related to Setswana (which achieved good results) as they are linguistically similar (Martinus & Abbott, 2019) and further investigation into the optimal vocabulary size needs to be done.

Original: The results conﬁrm those in Martinus & Abbott (2019), where the model performance is also related to the number of parallel sentences in the corpus.
Revised: The results conﬁrm those in Martinus & Abbott (2019), where the model performance is also related to the number of parallel sentences in the corpus.

Original: [Table]
Revised: [Table]

Original:  A NALYSIS
Revised:  A NALYSIS

Original: The SentencePiece BPE tokenisation resulted in an overall increase in performance when compared with the same models trained by Biljon et al.
Revised: The SentencePiece BPE tokenisation resulted in an overall increase in performance when compared with the same models trained by Biljon et al.

Original: (2020) using subword nmt.
Revised: (2020) using subword nmt.

Original: Example model outputs for Setswana and isiZulu have been included in Appendix A for a qualitative comparison.
Revised: Example model outputs for Setswana and isiZulu have been included in Appendix A for a qualitative comparison.

Original: An indepth qualitative analysis of the model results still needs to be done to determine if the high BLEU scores achieved correspond to accurate translations.
Revised: An indepth qualitative analysis of the model results still needs to be done to determine if the high BLEU scores achieved correspond to accurate translations.

Original: It is highlighted by Martinus & Abbott (2019) that there exists quality issues within the isiZulu data, including mismatched translations and spacing between letters, therefore despite achieving a big performance increase in the BLEU score the translations may be inaccurate.
Revised: It is highlighted by Martinus & Abbott (2019) that there exists quality issues within the isiZulu data, including mismatched translations and spacing between letters, therefore despite achieving a big performance increase in the BLEU score the translations may be inaccurate.

Original: Additionally the BLEU metric might not be expressive enough to evaluate the performance of the agglutinative language translations and a deeper analysis will assist in conﬁrming the results (Biljon et al., 2020).
Revised: Additionally the BLEU metric might not be expressive enough to evaluate the performance of the agglutinative language translations and a deeper analysis will assist in conﬁrming the results (Biljon et al., 2020).

Original: The initial tests show convincing results, therefore further research includes implementing an ablation study on the vocabulary size for the different language models.
Revised: The initial tests show convincing results when using SentencePiece BPE, therefore further research includes implementing an ablation study on the vocabulary size for the different language models.

Original: Only a small subset of vocabulary sizes were tested, therefore more research will assist in determining/conﬁrming the optimal size for each language.
Revised: Only a small subset of vocabulary sizes were tested, therefore more research will assist in determining/conﬁrming the optimal size for each language.

Original: An ablation study will also help determine the relationship between the size of the vocabulary, size of the dataset and the linguistics of the language.
Revised: An ablation study will also help determine the relationship between the size of the vocabulary, size of the dataset and the linguistics of the language.

Original:  I MPACT S TATEMENT
Revised:  I MPACT S TATEMENT

Original: This work could assist in the accurate creation of NMT models for English to Southern African languages.
Revised: This work could assist in the accurate creation of NMT models for English to Southern African languages.

Original: An increase in accurate translations could assist in bringing Africa into the scientiﬁc conversion and reduce the access to information gap, particularly in the online setting (Martinus & Abbott, 2019).
Revised: An increase in accurate translations could assist in bringing Africa into the scientiﬁc conversion and reduce the access to information gap, particularly in the online setting (Martinus & Abbott, 2019).

Original: Advances in low-resource language translation has great beneﬁts for education, providing information to people in their native language.
Revised: Advances in low-resource language translation has great beneﬁts for education, providing information to people in their native language.

Original: Additionally it could assist in the accurate detection of hate speech and violence incitement, which often goes unnoticed in current large-scale
Revised: Additionally it could assist in the accurate detection of hate speech and violence incitement, which often goes unnoticed in current large-scale

Original: NMT systems which do not perform adequately on low-resourced languages (Bender et al., 2021)
Revised: NMT systems which do not perform adequately on low-resourced languages (Bender et al., 2021)

Original: It is noted that the dataset used is taken only from South African government data and if used in production could misrepresent current social movements and will align to the existing political regime at the time the data was collected (Bender et al., 2021).
Revised: It is noted that the dataset used is taken only from South African government data and if used in production could misrepresent current social movements and will align to the existing political regime at the time the data was collected (Bender et al., 2021).

Original: Additionally the data has not been adequately curated to determine the biases that exist.
Revised: Additionally the data has not been adequately curated to determine the biases that exist.

Original: If any bias exists it will be encoded into the model, which could result in harm towards speciﬁc groups, if used in production for text generation or classiﬁcation (Bender et al., 2021).
Revised: If any bias exists it will be encoded into the model, which could result in harm towards speciﬁc groups, if used in production for text generation or classiﬁcation (Bender et al., 2021).

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: The effect of two byte pair encoding tokenisation algorithms, subword nmt and SentencePiece, with various parameters were compared for use in NMT models.
Revised: The effect of two byte pair encoding tokenisation algorithms, subword nmt and SentencePiece, with various parameters were compared for use in NMT models.

Original: The research builds upon previous research in the ﬁeld, using an optimised transformer architecture and pre-cleaned data to translate English to Northern Sotho, Setswana, Xitsonga and isiZulu.
Revised: The research builds upon previous research in the ﬁeld, using an optimised transformer architecture and pre-cleaned data to translate English to Northern Sotho, Setswana, Xitsonga and isiZulu.

Original: The results obtained show improvements in the previous BLEU scores obtained for Setswana and isiZulu when using the SentencePiece  tokenisation algorithm.
Revised: The results obtained show improvements in the previous BLEU scores obtained for Setswana and isiZulu when using the SentencePiece BPE tokenisation algorithm.

Original: It is noted that a qualitative analysis of the model results still needs to be done, to determine if the high BLEU scores achieved correspond to accurate translations.
Revised: It is noted that a qualitative analysis of the model results still needs to be done, to determine if the high BLEU scores achieved correspond to accurate translations.

Original: Additionally further work includes implementing an ablation study on the SentencePiece vocabulary size, to deﬁne the relationship between the size of the vocabulary, size of the dataset and the linguistics of the language.
Revised: Additionally further work includes implementing an ablation study on the SentencePiece vocabulary size, to deﬁne the relationship between the size of the vocabulary, size of the dataset and the linguistics of the language.

Original: This work provides a basis for further investigation into using SentencePiece tokenisation for NMT, with optimised vocabulary size, for agglutinative African Languages to achieve better performance. 
Revised: This work provides a basis for further investigation into using SentencePiece tokenisation for NMT, with optimised vocabulary size, for agglutinative African Languages to achieve better performance. A CKNOWLEDGEMENTS

Revised: The author would like to thank Jade Abbott for invaluable insights.

Revised: Special thanks are also given to the anonymous AfricaNLP reviewers for their helpful feedback and consideration.

Original: A A PPENDIX : T RANSLATION E XAMPLES
Revised: A A PPENDIX : T RANSLATION E XAMPLES

Original: Description
Revised: Description

Original: Text
Revised: Text

Original: Example 1:
Revised: Example 1:

Original: Source (en)
Revised: Source (en)

Original: The selection of the cultivar is of utmost importance.
Revised: The selection of the cultivar is of utmost importance.

Original: Reference (tn)
Revised: Reference (tn)

Original: Tlhopo ya mofuta
Revised: Tlhopo ya mofuta

Original: o o jwalwang ke kgang e e leng botlhokwa thata.
Revised: o o jwalwang ke kgang e e leng botlhokwa thata.

Original: Hypothesis (tn)
Revised: Hypothesis (tn)

Original: Go tlhopha mofuta
Revised: Go tlhopha mofuta

Original: o botlhokwa thata.
Revised: o botlhokwa thata.

Original: Example 2:
Revised: Example 2:

Original: Source (en)
Revised: Source (en)

Original: For example , researchers should earn much more for an article published in an international journal than for those published nationally.
Revised: For example , researchers should earn much more for an article published in an international journal than for those published nationally.

Original: Reference (tn)
Revised: Reference (tn)

Original: Ka sekai , babatlisisi ba tshwanetse go amogela go le gontsinyana fa ba gatisitse athikele mo lekwalopakeng la bodit ˇ shabat ˇ shaba go feta fa ba e gatisitse mo go la naga ya rona.
Revised: Ka sekai , babatlisisi ba tshwanetse go amogela go le gontsinyana fa ba gatisitse athikele mo lekwalopakeng la bodit ˇ shabat ˇ shaba go feta fa ba e gatisitse mo go la naga ya rona.

Original: Hypothesis (tn)
Revised: Hypothesis (tn)

Original: Sekao , babatlisisi ba tshwanetse go amogela athikele
Revised: Sekao , babatlisisi ba tshwanetse go amogela athikele

Original: e e phasaladitsweng mo lekwalopakeng la boditˇshabatˇshaba go gaisa bao ba phasaladitsweng mo nageng yotlhe .
Revised: e e phasaladitsweng mo lekwalopakeng la boditˇshabatˇshaba go gaisa bao ba phasaladitsweng mo nageng yotlhe .

Original: Example 3:
Revised: Example 3:

Original: Source (en)
Revised: Source (en)

Original: Domestic visitor
Revised: Domestic visitor

Original: Reference (tn)
Revised: Reference (tn)

Original: Moengselegae
Revised: Moengselegae

Original: Hypothesis (tn)
Revised: Hypothesis (tn)

Original: Baeng ba selegae
Revised: Baeng ba selegae

Original: [Table]
Revised: [Table]

Original: Example 1:
Revised: Example 1:

Original: Source (en)
Revised: Source (en)

Original: They obligingly sang Shosholoza when we requested the customary Venetian accompaniment to our cruise.
Revised: They obligingly sang Shosholoza when we requested the customary Venetian accompaniment to our cruise.

Original: Reference (zu)
Revised: Reference (zu)

Original: Bakuthokozela ukucula
Revised: Bakuthokozela ukucula

Original: uShosholoza ngesikhathi sicela ukuba kudlalwe umculo ohambisana nesiko laseVenisi ohambeni lwethu.
Revised: uShosholoza ngesikhathi sicela ukuba kudlalwe umculo ohambisana nesiko laseVenisi ohambeni lwethu.

Original: Hypothesis (zu)
Revised: Hypothesis (zu)

Original: Bathola iShosholoza ngesikhathi sicela ukuthi amasiko aseVenetian aphelekezela ukuhamba kwethu
Revised: Bathola iShosholoza ngesikhathi sicela ukuthi amasiko aseVenetian aphelekezela ukuhamba kwethu

Original: Example 2:
Revised: Example 2:

Original: Source (en)
Revised: Source (en)

Original: There were actually two Smits, one was the driver and the other one was in the crate with me.
Revised: There were actually two Smits, one was the driver and the other one was in the crate with me.

Original: Reference (zu)
Revised: Reference (zu)

Original: Empeleni babebabili oSmits, omunye wayengumshayeli kanti lona omunye ubesebenza nami emakredini.
Revised: Empeleni babebabili oSmits, omunye wayengumshayeli kanti lona omunye ubesebenza nami emakredini.

Original: Hypothesis (zu)
Revised: Hypothesis (zu)

Original: Kwakunezinkomba ezimbili zoSmit, omunye wabashayeli kanye namanye alowo oyedwa.
Revised: Kwakunezinkomba ezimbili zoSmit, omunye wabashayeli kanye namanye alowo oyedwa.

Original: Example 3:
Revised: Example 3:

Original: Source (en)
Revised: Source (en)

Original: the foreign vessel will ﬁsh under South African regulations and permit conditions
Revised: the foreign vessel will ﬁsh under South African regulations and permit conditions

Original: Reference (zu) lomkhumbi wangaphandle uzothobela yonke imithetho yokudoba nesimiso sephomedi sase South Africa.
Revised: Reference (zu) lomkhumbi wangaphandle uzothobela yonke imithetho yokudoba nesimiso sephomedi sase South Africa.

Original: Hypothesis (zu) umkhumbi wangaphandle uzodoba ngaphansi kwezimo ezithengisa izakhamuzi zaseNingizimu Afrika.
Revised: Hypothesis (zu) umkhumbi wangaphandle uzodoba ngaphansi kwezimo ezithengisa izakhamuzi zaseNingizimu Afrika.