CHEF in the Language Kitchen: A Generative Data Augmentation Leveraging Korean Morpheme Ingredients
Authors: Jaehyung Seo
Hyeonseok Moon
Jaewook Lee
Sugyeong Eo
Chanjun Park
Heuiseok Lim
Venue: EMNLP 2023 Main (2023)
Keywords: Data Augmentation
Morpheme Blender
Label Discriminator
Contrastive Learning
Korean Language
TL;DR: CHEF: A generative data augmentation with morpheme blender and label discriminator for Korean datasets

Abstract: Korean morphological variations present unique opportunities and challenges in natural language processing (NLP), necessitating an advanced understanding of morpheme-based sentence construction. The complexity of morphological variations allows for diverse sentence forms based on the syntactic-semantic integration of functional morphemes (i.e., affixes) to lexical morphemes (i.e., roots). With this in mind, we propose a method - CHEF, replicating the morphological transformations inherent in sentences based on lexical and functional morpheme combinations through generative data augmentation. CHEF operates using a morpheme blender and a label discriminator, thereby enhancing the diversity of Korean sentence forms by capturing the properties of agglutination while maintaining label consistency. We conduct experiments on Korean multiple classification datasets, improving model performance in full- and few-shot settings. Our proposed method boosts performance beyond the preceding data augmentation methods without incurring external data usage. We demonstrate that our approach achieves comparable results yielded by augmentation techniques that use large language models (LLMs).

CHEF in the Language Kitchen: A Generative Data Augmentation Leveraging Korean Morpheme Ingredients
Jachyung Seo', Hyeonseok Moon', Jaewook Lee', Sugyeong Eo' Chanjun Park2 and Heuiseok Lim
Depaarmeet of Computer Science and Engineering, Korea University UUsstage
(iseojae777,geeee0 jaewook 33, djtnrud, Jimhseokjekorea..ek cchanjun. park@ustaage. ai
Abstract
Original
Korean morphological variations present unique opportunities and challenges in natural language processing (NLP), necessitating advanced understanding of morpheme- bassed sentence construction. The complexity of mor- phological variations allows for diverse sen- tence forms based on the syntactic-semanti integration of functional morphemes (i.e., af- fixes) to lexical morphemes (i.e., roots). With this in mind, we propose a method CHEF, replicating the morphological transformations inherent in sentences based on lexical and fune- tional morpheme combinations through genera- tive data augmentation. CHEF operates using a morpheme blender and a label discriminator, thereby enhancing the diversity of Korean sen- tence forms by capturing the properties of ag- glutination while maintaining label consistency. We conduct experiments Korean multiple classification datasets, improving model perfor- mance full- and few-shot settings. Our pro- posed method boosts performance beyond the preceding data augmentation methods without incurring external data usage. We demonstrate that our approach achieves comparable results yielded by augmentation techniques that use large language models (LLMs).
Lasical Morphames
"O" "Ml-")
Peetive
Becic
Reege
x
RecieN
CHEF
Peslóve
Pestive
Peetive
Figure 1: CHEF for data augmentation. The Korean verbs *2/1"" 'H "yo and "=/ have equivalent meaning of 'eat'. The color variations in the shapes indicate morphological changes resulting from combinations with functional morphemes. The 'Recipe N' denotes morpheme set that can compose various sentences. 'Positive' represents the example of label attached the original sentence.
Consequently, the incorporation of morphological alterations following combinatory rules enables significant diversity of linguistic forms be gener- ated from given lexical morphemes.
1 Introduction
As an agglutinative language, Korean encompasses rich array of functional morphemes (Song, 2006; Park et al., 2018). Deep learning-basc research in Korean NLP aims to enhance linguistic construc- tion efficiency through morpheme segmentation. Previous studies have explored morphological anal- ysis to improve model performance (Song and Park, 2019; Lee et al., 2020; Kim and Colineau, 2020; Kim al., 2022). The Korean language presents distinct challenges and opportunities with respect transformations contingent upon the combina- tory patterns between lexical and functional mor- phemes (Matteson et al., 2018 8; Seo et al., 2022).
In this sense, Korean datasets are constrained in capturing the intricate rules governing the interac- tions between lexical and functional morphemes. As a result, the representations derived from these datasets only contain a small fraction of the poten- tial linguistic forms. For instance, Figure demon- strates multiple approaches to represent the concept of 'cat' by employing different ending forms. Ad- ditionally, previous data augmentation approaches are rarely designed to generate synthetic data that considers the morphological characteristic the Korean language, resulting in limited effectiveness in data augmentation. To address these limitations,
Corresponding author

---

it is crucial to develop data augmentation methods that explicitly consider morphological diversity. By incorporating such properties into the augmenta- tion process, we can contribute to the robustness of models and facilitate performance enhancements.
CHEF unlocks the morphological diversity in the training data without using any additional external data and approximates or even out- performs the LLM-based data augmentations.
CHEF exhibits effectiveness even with small amounts of data augmentation.
In this paper, we propose CHEF, data augmen- tation method designed to construct new synthetic data in alignment with the combinations of lexical and functional morphemes. CHEF is composed of a morpheme blender' and label discriminator. For the initial phase, we prepare Korean lexical morphemes from the training dataset as ingredients. The morpheme blender employs the lexical mor- phemes and generates the new synthetic sentence by leveraging the knowledge of derivational and inflectional rules in conjunction with functional morphemes obtained from pre-trained generative model. Morpheme blender can generate diverse sentence expressions by incorporating lexical and functional morphemes, yet does not guarantee the conservation of label information. To adjust the unintended blending, the label discriminator con- trols the synthetic data generation process of the morpheme blender. Its primary objective is to pre- vent substantial shifts in meaning and ensure align- ment with the original labels. The teacher forcing of the morpheme blender incorporates global infor- mation about synthetic data completed by combi- nations of lexical and functional morphemes. This is achieved through contrastive learning with the label discriminator. Integrating these two modules, CHEF ensures that synthetic data maintains label consistency while blending new morpheme combi- nations.
Related Work
Data augmentation widely recognized in deep learning research as a valuable approach to address- ing the scarcity of annotated data. It aids in ensur- ing that the distribution of the training data main- tains robustness even in unseen tests. Over time, several methods have been developed to augment textual content, expanding the diversity and quan- tity of available training data.
Zhang et al. (2015) proposed a method of word substitution using a synonym thesaurus based on rules, which is followed by augmentation meth- ods utilizing thesaurus in the studies of (Dai and Adel, 2020; Daval-Frerot and Weis, 2020). Wei and Zou (2019) suggested an easy data augmenta- tion (EDA) method using word-level replacement, random insertion, random deletion, and random swap. Karimi et al. (2021) introduced an easier data augmentation (AEDA) technique, improving text classification performance by randomly insert- ing six pre-defined punctuation marks. Another strategy involves Mixup-based data augmentation (Zhang al., 2018), combining two sample data to create new training data. Mixup augmentation was initially applied to image data and extended text-based deep neural networks (Guo, 2020). Sub- sequently, it has also been applied to textual data methods, augmentation using Transformer-bast serving as a technique to mitigate model overfitting and enhance generalization capabilities (Sun et al., 2020; Yoon et al., 2021; Kong al., 2022).
As illustrated in Figure 1, CHEF augmentation enriches text data by attaching various functional morphemes to given set of lexical morphemes from the original sentence while preserving the label information. The main contributions of our method are summarized as follows:
Sennrich et al. (2016) proposed back transla- tion method for augmenting data by performing a round-trip translation on data written the orig- inal language, using a neural machine translation system trained on human-annotated data. Data aug- mentation based on back translation has evolved as one of the crucial techniques since it allows for rewriting the entire sentence, rather than just word- level alterations (Fabbri al., 2021; Lowell et al., 2021; Park et al., 2021a).
Despite difficulties enhancing performance in full-shot settings through data augmentation, CHEF demonstrates effectiveness and robust- ness through in-depth analysis.
We observe that employing contrastive learn- ing between label discriminator and a mor- pheme blender is suitable data augmentation approach for maintaining label consistency.
Recently, research on data augmentation using pre- pre-trained language models has been actively pur- sued (Kumar et al., 2020; Du et al., 2021; Schick
Morphhmee blender takes morphemes and generates a sentence that contains given morphemes, In this sense, we denote it as a blender.

---

Morpheme Blender Mg
Label Discriminator MD
Pre-trained
Leons
Frozen g
Gradient
Predict
Pooling
Gradient
Pooling
EnGG
Decg
Mp
Enzoder imput
Is Optional
Premise
Hypothesis
Decodar input
morpheme set in
(optional)
Discriminator imput
Figure 2: Training process of CHEF with Morpheme Blender Mg and Label Discriminator M
and Schütze, 2021; Zhou et al., 2022; Mekala et al.,
set C (ci, to make a target sentence based on the relation within C. We perceive par- allel between this process and combining ingredi- ents to create complete dish. In particular, Seo et al. (2022) have shown that combining lexical morphemes yields optimal parsing for generating Korean sentences. Consequently, on harnsssing the sentence augmentation effect of gequence-t-ssqquee models by assembling ingre- dients of morpheme sets.
2022). Among them, the data augmentation tech- nique through counterfactual is particularly note- worthy, demonstrating improvements to the state- of-the-art performance levels across various bench- mark datasets (Liu et al., 2021; Joshi and He, 2022; Ou et al., 2022; Wen al., 2022). With the advent of LLMs, considerable progress has been made in overcoming augmentation constraints through generative AI (Yoo et al., 2021). Thus, we proceed assuming that it is feasible to secure textual content with LLMs. Nevertheless, it is still necessary to expend financial and temporal resources to iden- tify the optimal direction for augmentation. As result, focus determining which augmenta- tion method could benefit model learning from Ko- rean data. Given the diverse derivational and inflec- tional forms inherent in the agglutinative Korean language, we propose CHEF an augmentation method that considers these linguistic properties.
concentrate
Preliminary
The primary objective of CHEF is to augment dataset D = {(inp', [ where inp' (M' x²) denotes textual input and 1 is its corresponding la- bel. Considering multiple sentence settings such as NLI and STS, regard h hypothesis and define auxiliary input p' as a premise. If the input comprises a single sentence (e.g., YNAT), we re- gard ht as the same text as inp', and p' is conceived as an empty string.
3 CHEF
The structure of CHEF is illustrated in Figure 2. It consists of two components: the label discrimina- tor module (M,) that helps maintain label consis- tency and the morpheme blender (Mc) that aug- ments data by referring each morpheme this framework, the label discriminator module is presented as an encoder architccture, such as BERT (Devlin et al., 2018), and the morpheme blender module possesses an encoder-decoder ar- chitecture, such as BART (Lewis et al., 2020)
Taking inspiration from CommonGen (Lin et al., 2020) and Korean CommonGen (Seo et al., 2022), we focus on the data-to-text generation capabil- ities of sequence-to-sqqeemee models. Common- Gen and Korean CommonGen necessitate genera- tive commonsense reasoning based on given con- cept sets, where given concept sets are used to determine the feasibility of constructing plausible sentences. Both conditional generation tasks are to make generative language models learn a func- tion f : + that maps a set of input concept
In
We adopt Korean pre-trained language models: KoBART (https: thub. con/SK con/SKT-AL AI/KBBRTT) and BLUTTBERTT (https://gil http://ggtttb...// con/KLUE wom/KLLE-bencharkk////

---

Both modules are trained generate synthetic data according to the task and label to be augmented. In the following sections, we describe each module in detail.
Teacher Forcing For attaining ingredient synthe- sizing capacity, M2 is trained generate h' by referring to the concatenated sequence of m', p', and v'. The encoder framework of Mc, denoted as Encg, receives a concatenated sequence and gen- crates a contextualized representation by capturing the bidirectional interactions of the words in the imput. Subsequently, Mc produces sentences by uti- lizing the contextual representation acquired by the encoder. We can define the loss function Lce for training sequence-to to-sequence generation of Mc as in equation (1). For clarity in expression, we denote [..-] I as a scquentialized concatenation all the elements in it, and define the concatenated input seq' = [m², p², v.).
3.2 Label Discriminator
Label discriminator module (Mp) helps retain the intended label consistency during the data augmen- tation process. Mp is designed to take inp' and offer probabilities to be classified into each label. Mp encode inp into the vector space by applying a linear pooling layer followed by the softmax, to the [BOS] position of the last hidden state. We de- note the processed output as Mp(inp') < Rlxme, where ne indicates the number of classes for the task dealt with D. Mp is supervised to maximize the probability of each inp' to be classified into Concisely, Ma is task modulc trained with 'D that provides the label expectation probability obtained by the full contextualized sentence inp', and is utilized as label instructor of the morpheme blender, described in the latter sections.
Lee - Σ Σ log Psaalh) sest) (1)
Through the -e-suueeece training, Mc can generate a whole sentence that covers a given morpheme set with weakly reflecting the label in- formation granted by o² in seq'.
3.3 Morpheme Blender
Label Consistency Supervision Even if label in- formation is reflected in the generation process by feeding it input, we find that generation out- puts from M3 still suffer from label inconsistency. To alleviate this, we exploit the auxiliary training objective utilizing pre-trained discriminator Mg consider label consistency.
Ingredients Preparation The main objective of the morpheme blender (Ms) is to synthesize the sentence by given morpheme ingredients and its targeting label. For training Mc. we extract lexical morpheme ingredients m² = {mi..... m,,)) from the h², where n² denotes the number of lexical mor- phemes in h². In preparing m', we use a Korean morphological analyzer, mecab-ko to make the lexical morpheme set for h'. We leverage that the diversity of Korean sentences that can be combined for the same lexical morpheme ingredients is poten- tially high. Therefore, we construct the morpheme set based on substantial lexical morphemes, allow- ing the morphemes within the model-awar -wwre recipe to be combined with various endings, functional morphemes, and particles.
We argue that by aligning the encoded outputs of Mp and Enc,, we can make Enc; better embed contextual relations with the corresponding mor- pheme sets and task-specific labels and promote label maintenance. This can be distilled to the train- ing objective that maximizes the similarity sim' defined the following equations:
r = SoftMax SoftMaxx(Enccssee -Wo) We)
(2)
r Mo(inp))
We also define label mapping function vi : + o that maps label 1 into the label word of that has form of natural language. For instance, in the case NLI, we adopt "contradiction" for the label "-1", "neutral" for "0", and "entailment" for "1".
sim' (inp)
(3)
|| Mp(inp)]]
Note that through the pre-training of Mp. the encoded output of Mp represents the label infor- mation of the whole sentence h that M; should generate. The auxiliary training objective aims to make the label representation encoded through the of ingredients seq'. be aligned with the label representation yielded by h².
In gathering these, we establish the blender that takes morpheme ingredients mf (optionally with p¹) and label word n² inputs, and returns newly synthesized sentence h Detailed training proce- dures are described in the following section.
For the direct supervision, we define contrastive sample set Ci C D for each (inp', (t) ) E D. and

---

contrastive label set I 3 for each label P. where denotes the set of all possible labels considered in D. In comprising C³, we randomly extract a single sample from D for each label in P. Then the loss function Leont for learning label consistency is defined as equation (4)
The KLUE-NLI dataset (Park al., 2021b) has explicitly been curated for the natu- ral language inference (NLI) task (Bowman et al., 2015). The training, validation, and test data com- prises 24,998, 3,000, and 3,000 sentence pairs. this task, models are required to process pairs of sentences, referred to as the premise and hypothe- sis, and deduce the underlying relationship, which could be entailment, contradiction, or neutral.
KLUE-NLI
exp(sim((inp')) (inp')/ //7)
Ciont - -log E (Imp² Wyyer exp(sim' (inp')/T)
(4)
The KorNLI dataset (Ham al., 2020) also designed for Korean natural language in- ference. It is generated by translating the En- glish Standard NLI (SNLI) and Multi-Genre NLI (MNLI) datasets, as well as the Cross-lingual NLI (XNLI) dataset into Korean. The training data of the KorNLI dataset consists of 942,854 sentence pairs, machine-translate from the SNLI and MNLI datasets, while the evaluation data comprise 7,500 translated sentence pairs from the XNLI dataset.
KorNLI
W6 denotes the linear pooling layer that maps the encoded representation on the [BOS] position into the label classification probability. The tem- perature parameter controls the sharpness of the softmax distribution with larger values leading to a smoother distribution and smaller values leading to more peaked distribution.
In summing these, Mc is trained with Liotal de- fined as the following equation:
KLUE-STS The KLUE-STS dataset (Park et al., 2021b) has been meticulously assembled for the se- mantic textual similarity (STS) task, encompassing 1 1, 1,668 sentence pairs for training, 519 for valida- tion, and 1,037 for testing. In this task, models as- sess pairs of sentences and determine their degree of semantic similarity.
Ceont
Σ C coms
(5)
[D]
Crotal = (1 11--i Cee + XCcont
(6)
In equation (5) and (6), À is the balance parame- ter between the two losses.
3.4 Augmentation Pipeline
KLUE-YNAT KLUE-YNAT dataset (Park et al., 2021b) has been designed for the topic classifica- tion task. The dataset includes training, validation, and test data composed of 45,678, 9,107, and 9,107 samples. In this task, models are tasked with pro- cessing sentences and assigning them to predefined news categories based the underlying topic.
In utilizing Mc, we generate single data for each (inp', in D. The pipeline is as follows:
1. Extract morpheme set m of h in inp
2. Generate Naug = Ma(mm', pt, of)), where of is the label word of l'.
3. Kang is regarded as an augmented data for D, which label is 75.
The NSMC dataser' has been constructed for the sentiment analysis task Korean. It is de- rived from movic reviews and their respective rat- ings from the NAVER platform. The training data of the NSMC dataset comprises 150,000 reviews, and the test set consists of 50,000 reviews. In this task, models are required to process individual sen- tences and determine their underlying sentiment, which can be positive or negative.
NSMC
Considering that excessive augmentation may lead to error accumulation and the following label confusion, we apply CHEF to the small fraction of D in implementing full-shot learning.
4 Experimental Settings
We introduce experimental settings used for the experiments. More details are Appendix A
4.2 Models
The morpheme blender employs an encoder- decoder structure and uses KoBART (Lewis et al.. 2020), pre-trained generative language model for
4.1 Datasets
We adopt Korean multiple classification benchmark datasets. Each dataset used for training and eval- uation according to the proposed task format.
'https://githuu httts://ggitbbb.cmo/tt////mm

---

KLUR-NLI
KoINLI-MNII
KOrNLESNII
KLUEESTS
KLAIE-YAA
NSMC
Model
(Aco.)
(Acc.)
(Paaronr)
[Acc:]
BEXT" TUM
81.53
30.63
71.56
90125
K5.73
90.30
+KoEDA
81,30
80.12
69.78
90.30
86,01
90.42
+AIDA
81.47
S0.76
69.52
90.36
56.35
90.35
+BT
81.30
$0.08
70.71
90.65
85.15
90.35
+Mixup
82.10
30.14
71.74
90.97
86.28
90:10
+GPT-3.5CC
78.90
79.87
68.52
89.49
84.36
89,98
S4PRR
91
90:45
+GPT-4CA
80.73
80
65.3 26
90.92
85,50
90.45
+GPT41PARA
68.
86.14
90:43
+CHRF
82.63
81.07
72.06
91
86.68
90.52
ROBERTE
84.83
8084
71.84
92.50
85.07
90.08
KOFDDA
92
90:71
+AEDA
85.57
$0.56
71.70
92.50
85.19
90.63
90:65
-Mixup
85.83
$0.60
70.85
92.77
85.55
90.00
S-RRAA
$4.60
90.58
70.12
92.95
85.11
90.87
62.15
84.73
-GPT--PPARA
85.03
81.23
70.08
92. 99
85.47
90.94
72.22
93.05
86.18
ROBERTE Lare
81.73
73.26
93.35
85.69
91.28
KOHDA
89.63
82 22.24
72.52
9/3
K5.32
91.12
LARDA
72.26
+BT
33.33
90.36
72.64
92.89
$5.24
91.05
90.83
81.95
72.39
93.43
85.99
90.33
+GPT-3 S4CA
88.03
33.33
90.73
84.23
33.72
G.PT--.5-PARR
88.33
83.12
71.86
93.28
85.88
91.31
+GPT-HCA
89.10
83.48
68.94
93.20
85.46
67.97
+GPT-4-PARA
90.13
82.88
72.87
93.51
86.19
91.21
Table 1: Full-shot evaluation results multiple classification datasets. The best performance in each dataset is bolded. Underlines indicate that CHEF has outperformed the baselines. Scores in red denote training failure due to poor quality. Accuracy (Acc.), Fi-score (F1), and Pearson correlation (Pearsonr) are used evaluation metrics.
Korean, as its backbone. We choose KoBART be- cause it exhibits acceptable Korean text genera- tion abilities even in its small model parameters (124M) (Seo et al., 2022). KoBART takes in- put sequence consisting of lexical morpheme set, label word, and an optional premise to gen- erate synthetic data. The label discriminator fea- tures an encoder architecture and utilizes KLUE- BERT-base (Devlin et al., 2018) 8), a pre-trained lan- guage model for Korean, as its backbone. KLUE- BERT-base conveys the original sentence's label to KoBART through contrastive loss. We opt for the KLUE-BERRTbase as it is the most compact model among the evaluated alternatives, mitigating the possible distillation effects that could emerge during the discrimination process due to model sizes (Park et al., 2021b). To validate the efficacy of CHEF, we select the following Korean language understanding models: BERT
periments using Back Translaanntoon (BT) (Sennrich et al., 2016), Korean-EDA (KoEDA) (Wei and Zou, 2019), AEDA (Karimi et al., 2021), and Mixup (Zhang et al., 2018; Sun et al., 2020). We also employ GPT-3.5 (Ouyang et al., 2022; Brown et al., 2020) and GPT-4 (OpenAI, 2023) as back- bone augmentation models. Our LLM-based ap- proaches included semantic-based paraphrasing (PARA) (Fadaee et al., 2017: Kobayashi, 2018) and counterfactual augmentation (CA) (Liu et al., 2021; Ou et al., 2022).
5 Experimental Results
In this section, we evaluate a set of multiple classi- fication datasets with our proposed method.
5.1 Full-shot Learning
As shown in Table 1, we compare the effectiveness and extent of performance improvement of CHEF against other data augmentation methods in the full-shot settings. We proportionally augmented the data with respect to the size of the training dataset. ensuring that an amount corresponding to 1% the training data is added to each label.
Base KKLUE-BERS base), RoBERTa Base KKLUE-RoBERTA base), and RoBERTaK Large IKLUE-ROBERTTan
4.3 Compared Methods
The experiments are conducted based on full- and few-shot learning. We conduct comparative ex-
htttps: thub. com/ com/torii Somm/orrivigg/Koer.

---

AON
Fode
POdK
PSOS0 AuCCTT
No337 Maacerarg
Ne0377 maog
SOORT KOLNLE-NNI
O00 SNLL
DER KLUE-YNAT
ANCWEROR STC
KLU
KLUE-STS
NSMC
Figure 3: Performance of the language models trained multiple classification datasets by adjusting the augmenta- tion ratio of CHEF. The first column of each plot is the baseline performance without data augmentation.
Augmentation methods employing LLMs hibit the generation of sentences with high diversity and superior qualitative properties. However, they occasionally demonstrate decreases in performance up to 2%. These declines can be attributed to significant alteration of the overall data distribution, which makes it challenging to fully account for the underlying labeling scheme intrinsic to the task. In the case of counterfactual augmentation, the in- stability in label transformations amplified, even when provided with thain-offthoougee prompts (Wei al., 2022). Original sentences and labels undergo counterfactual changes without any review for con- formance, resulting in increased variability that hin- ders the effectiveness of the augmentation process.
strates the efficacy of data augmentation for full- shot learning.
5.2 Changes in Augmentation Size
Figure 3 presents the performance changes accord- ing to each dataset's augmentation ratio of CHEF. As the amount synthetic data increases, the probability of the involvement of data that renders negative noise accumulations. In the full-shot set- ting, simply expanding the number of synthetic data samples does not guarantee a performance improvement. However, CHEF maintains the effec- tiveness of data augmentation and shows superior enhancements compared the baseline in most cases where data augmentation is limited within 10% (More details in Appendix 7).
BT cannot guarantee the preservation of the same labels, resulting in performance decrease cases. AEDA does not show the same level of im- provements for all tasks reported in (Karimi al., 2021) when applied to pre-trained Korean language models. Furthermore, Mixup exhibits no- table performance improvements KLUE-NLI; however, these enhancements do not consistently ensure improved results across other tasks. KoEDA utilizes morphological analyzer to segment the data and leverages thesaurus for Korean. This ap- proach shows modest performance improvements, validating the effectiveness of morpheme- based data augmentation in line with the characteristics the Korean language. However, its performance enhancements are limited due to challenges en- suring label consistency and the absence of contex- tual understanding. We observe that CHEF exhibits the most significant performance improvements. By leveraging comprehensive understanding of derivation and inflectional properties in conjunc- tion with functional morphemes, CHEF effectively enhances models across various scenarios. As a re- sult, CHEF overcomes these challenges and demon-
5.3 Few-shot and Synthetic-only
We further probe the efficacy CHEF in the NLI task, which is relatively capricious data augmen- tation effects in Section $5.2. We conduct com- parative experiments in few-shot setting with 32 samples and add 32 augmented sentences for each label. Table shows the results averaged and max- imum value over three different seeds. The data augmentation methods using the LLMs are more fective in low-resource regimes than in full-shot set- tings. CHEF significantly boosts the model's 's perfor- mance, even in situations with limited data. Further- more, we evaluate the effectiveness of solely using the synthetic data generated by CHEF (i.c.. CHEF- SynOnly) to train the models. The outcomes reveal spectrum of enhancements in performance and demonstrate the quality of the CHEF augmentation.
5.4 Ablation Study
To precisely evaluate the importance of labeling consistency, we conduct ablation studies by sys- tematically altering the label discriminator com-

---

KLUE-NII
KocNLL-MNLI
KorNLI-NNII
KLUE-NLI
KLUE-STS KLUE-YNAT
Model
Model
(Acc.
(Acc.)
(Acc.)
(Acc.)
(Pearsonr)
(FI)
BERT Hoss
37.6/40.1
40.5/41.7
BERT Base
81.53
90.85
85.73
+KoEDA
43,0/5.0
43.5/44,9
36.8/38.6
+Mg +M,+Mo 82.54
81.83
90.62
86.02
+AEDA
43.1/45.5
1/43,
38.0/39.8
91.05
86.34
41.1/43.0
+Mixup
40,6/41.5
42.8/44.2
38.9/39.2
+Mc+Mo+PT 82.63
91.26
86.68
+GPT-3.5CCA
37.1/57.7
41.6/44.6
38,2/39.5
Table 3: Results of the ablation study the CHEF. Mc and Mp respectively denote the morpheme blender and discriminator. PT refers the pre-training of Mp. The values in bold represent the highest performance.
+GPT-++CA
41,7/43.0
46.3/48.5
40.5/41.8
+GPT-H+PARA
44,2/4611
43.3/43.7
37,7/40.7
+CHEF
46.3/47.6
47.7/49.2
41.1/42.3
+CHRF-SymOnly
41,3/4..5
46 8/47,4
35.1/57.1
ROBERTA Nows
37.6/40.1
36.5/36.7
36.1/36.7
+KOEDA
41.2/44.4
47397
36.3/36.5
TRAIN TEST
+AEDA
42,4/48.0
39.5/42.9
36,8/37.2
Model
+BT
39.9/42.2
3..
36.2/37.1
KLUE-NLI Ker-NLI Kor-NLI - KLUENNII
+Mixup
38.4/39.5
BERT Bac(Full)
61.14
71.37
AGPT-3,54A
36.1/36.9
39. 1/42.6
36.1737.1
+CIIEF
60.09
72.43
PAARAS..PAAR
39.8/41.8
39 4/40/
36, 1/36.7
BERT (EIRT"R.((Few) (Fe(Few)
41.01
36.06
+GPT-H-CA
99.6/414
-CROSS" +CHEF
39.04
38.93
+GPT-4-PARA
43,3/45.8
39.4/4177
39,4/4166
49.66
40.93
+CHEF
41,2/43.3
43.9/45.8
40.1/44.7
+CHRF-SymOnly
38.739..5
13.045.1
33.9/34.8
Table 4: Results cross-domain analysis. + indi- cates that the training and test data belong different domains. -CROSS' denotes the results without cross- domain settings. best performance bolded.
ROBERTE Lams
47.3/5177
4044/44
37.5/39.4
+KOEDA
51,2/52.6
38.9440.9
+AEDA
54.6/56.1
45.0/48.6
38.7/40.3
+BT
$4,4/56.3
43.5/48.7
38.038..8
+Mixup
$7.1/7.7
5
40.2/40.5
+GPT-3.54CA
41,4/43.9
41.7/45.9
38.5/40.6
+GPT3.5+PARA
52.6/53.7
43.8/47.1
38.5/39.8
+GPT-4+CA
45.6/510
46.1/48.8
41,4/4..3
evaluation of the model trained on data augmented using CHEF. Table 4 presents the observed per- formance improvements in three out of four cases, compared to the baselines without augmentation. The cross domain effectiveness of CHEF is more pronounced in the few-shot setting. Furthermore, few-shot CHEF outperforms models trained and evaluated solely on single domain. These results alleviate concerns regarding the degradation of the model's generalization ability due the augmen- tation effects, reaffirming its robust performance across different domains.
+GPT-4-PARA
56,9/89.5
45.3/48.2
38.6/41.2
+CHEF
56.5/57.2
51.5154.5
44.9147.6
SCHEF-SynOnly
50.0/57.1
49.8/54.5
33.8/34.2
Table Few-shot evaluation results. scores on left side of the 'I' represent the averages, and those on the right indicate the maximums. The best perfor- mance in each dataset bolded. Underlines indicate that CHEF has outperformed the baselines.
ponent within the CHEF, The variations in perfor- mance resulting from the exclusion or absence of pre-training in the discriminator are presented in Table 3. Upon removal of the label discriminator, observe decline in the augmentation effective- ness in the full-shot setting. Notably, even without the pre-training phase of the discriminator, CHEF maintains capacity for enhancing performance. By integrating the morpheme blender and the pre- trained label discriminator using contrastive loss, CHEF achieves the highest level of performance. These empirical findings provide compelling ev- idence for the effectiveness of incorporating the discriminator to ensure label consistency.
5.6 Larger and Multilingual Blender
To comprehend the implications of employing a larger generative model for CHEF, we evaluate the performance leveraging the mBART-large (Liu et al., 2020). To avoid unintended knowledge dis- tillation effects resulting from using larger discrim- inator modules, we maintain the same discrimina- tor. As presented in Table 5, the results present that using a larger generative model as the blender does not necessarily ensure higher performance, but comparable effects can be achieved as well. We further find that employing a multilingual model as the blender yiclds similar augmentation effects.
5.5 Cross-domain Analysis
We conduct cross-domain experiments in the NLI task. As described 14.1, KLUE-NLI and Kor- MNLI/SNII are datasets derived from different sources. To assess performance, we employ cross-
5.7 Morpheme Ingredients Variants
Table 6 shows the CHEF's performance employ- ing different morpheme ingredients. We modify

---

KLUE-NLI
KLUE-STS KLUE-YNAT
language. However, the method of this study pri- marily focuses on Korean and does sufficiently consider other languages. This area can further explored and improved in future research. Also, due to the performance limitations of the off-the- shelf pre-trained generative model, unnecessary word duplication occurs in sentences augmented using CHEF. This issue can be addressed by in- troducing more optimized decoding strategy employing more advanced generative model, ex- pected to produce higher-qualie results. Moreover, The mecab-ko analyzer, which we used for con- structing the set of morphemes, can have an error rate depending on the domain and data it is applied to. The proportion of morpheme combination rules in CHEF may differ based on which morpheme an- alyzer is used to construct the morpheme set. This presents potential risk: if a less proficient mor- pheme analyzer is used, may fail to recover the original morphemes accurately and be susceptible to errors when processing data from untrained do- mains. Therefore, leveraging a more advanced mor- pheme analyzer could enable data augmentation that more accurately reflects the linguistic charac- teristics of the Korean language.
Model
(Acc.)
(Pearsonr)
(FI)
BERT Base
81.53
90.85
85.73
+CHEF(BART)
82.63
91.26
86.68
+CHEF(mBART)
82.00
91.26
87.11
Table 5: Evaluation results on larger morpheme blender. The best performance in each dataset is bolded.
Model
KLUE-NLI
BERT
81.53
CHEFF Sy momeym
80.22
CHEF Antonyan 79.84
Table 6: Evaluation results on morpheme ingredients variants. CHEF Synonym and CHEF Amonym denote the methods of replacing morphemes and labels.
the lexical morphemes by substituting them with synonyms or altering them to antonyms consider- ing labels. Replacing lexical morphemes based on antonyms synonyms is not guaranteed to pre- serve the compositionality across other lexical mor- phemes. The involvement of con-contextualizze morphemes leads to diminished generative capa- bilities and the generation of low-quality synthetic data, resulting in a decline in performance.
Ethics Statement
6 Conclusion
We employed Korean multiple classification bench- mark datasets in our experiments. Data augmenta- tion was conducted by altering the morphological composition of sentences present in the training datasets. Excluding the NSMC dataset, each bench- mark dataset has been officially released and has undergone validation to ensure ethical considera- tions using human annotators. The NSMC dataset may contain some unethical expressions among negative reviews. Furthermore, we acknowledge that the pre-trained language models (PLMs), used as the backbone for the morpheme blender, could have been exposed to toxic data during the pre- training process, thereby possessing the potential to generate inappropriate synthetic data.
We introduce CHEF, a novel data augmentation method designed for the Korean language, which is inherently agglutinative and rich in morpholog- ical variations. CHEF leverages the combinatory properties of lexical and functional morphemes in Korean construct linguistically diverse and label- consistent synthetic data. By incorporating a mor- pheme blender and a label discriminator module, CHEF ensures that the generated synthetic data pre- serves the label information of the original dataset while introducing new linguistic forms through morphological alterations. Our experiments demon- strate the effectiveness and robustness of CHEF across various scenarios. In future work, we plan explore the adaptability of the CHEF architec- ture other morphologically rich languages and further optimize the interaction between the mor- pheme blender and label discriminator.
A Acknowledgments
This research was supported by the MSIT (Min- istry of Science and ICT), Korea, under the ITRC (Information Technology Research Center) support program (LTTP-2033-0118-00) the IITP (Institute for Information Communi- cations Technology Planning & Evaluation). This work was supported by Institute of Information &
Limitations
supervised by
This study proposes an effective augmentation method suitable for Korean datasets, considering the unique linguistic characteristics of the Korean

---

communications Technology Planning & Evalua- tion (IITP) grant funded by the Korea government (MSIT) (No. 2020-0-00368, A Neural-Symbolic Model for Knowledge Acquisition and Inference Techniques). This research was supported by the MSIT (Ministry of Science and ICT), Korca, under the ICT Creative Consilience program (ITTP-2023- 2020-0-01819) supervised by the IITP(Institute for Information & communications Technology Plan- ning & Evaluation).
Alexander Fabbri, Simeng Han, Haoyuan Li, Haoran Li, Marjan Ghazvininejad, Shafiq Joty, Dragomir Radev, and Yashar Mehdad. 2021. Improving zero and few- shot abstractive summarization with intermediate fine-tuning and data augmentation. In Proceedings of the 2021 Conference of the North American Chupter of the Association for Computational Linguistics: Hu- man Language Technologies, pages 704-717, Online. Association for Computational Linguisties.
Marzich Fadace, Arianna Bisazza, and Christof Monz. 2017. Data augmentation for low- www-resource neural machine translation. In Proceedings of the 55th An- nual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 567-573
References
Daniel M Bikel. 2000, Automatic wordnet mapping using word sense disambiguation. In 2000 Joint SIG- DAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora, pages 142-147.
Angela Fan, Shruti Bhosale, Holger Schwenk, Zhiyi Ma. Ahmed EI- El-Kishky. Siddharth Goyal, Mandeep Baines, Onur Celebi, Guillaume Wenzek, Vishrav Chaudhary, et al. 2021. Beyond english-centrie multi- lingual machine translation. The Journal of Machine Learning Research, 22(1):4839 4886.
Samuel Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. 2015. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference Empirical Methods in Natural Language Processing, pages 632- 642.
Hongyu Guo. 2020, Nonlinear mixup: Out-of-manifold data augmentation for text classification. In Proceed- ings of the AAAI Conference on Artificial Intelligence, pages 4044-4051.
Jiyeon Ham, Yo Joong Choe, Kyubyong Park, Iiji Choi, and Hyungjoon Soh. 2020. Kornli and korsts: New benehmark datasets for korean natural language derstanding. In Findings of the Association for Com- putational Linguistics: EMNLP 2020, pages 422- 430.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33: 1377-1919....
Nitish Joshi and He He. 2022. An investigation of the (in) effectiveness of counterfactually augmented data. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Pupers), pages 3668-3681
Xiang
Dai and Heike Adel. 2020. An analysis of simple data augmentation for named entity recog- nition. In Proceedings of the 28th International Con-
ference on Computational Linguistics, pages 3861- 3867, Barcelona, Spain (Online). International Com- mittee on Computational Linguistics.
Akbar Karimi, Leonardo Rossi, and Andrea Prati. 2021. Aeda: An easier data augmentation technique for text classification. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 2748 -254.
Guillaume Daval-Frerot and Yannick Weis. 2020. WMD at SemEval-2020 tasks 7 and 11: Assess- ing humor and propaganda using unsupervised data augmentation. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 1865- 1874, Barcelona (online). International Committee for Computational Linguistics.
Myung Hee Kim and Nathalic Colineau. 2020. An enhanced mapping scheme of the universal part-of- speech for korean. In Proceedings of the 12th Lan- guage Resources and Exaluation Conference, pages 2823-
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understand- ing. arxiv preprint 181.vv1810.04805.
Nayeon Kim, Jun-Hyung Park, Joon- Joon-Young Choi, Bojin Jeon, Youjin Kang, and SangKeun Lee. 2022. Break it down into bts: Basic, tiniest subword units for ko- rean. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 7007-7024.
Jingfei Du, Edouard Grave, Beliz Gunel, Vishrav Chaud- hary, Onur Celebi, Michael Auli, Veselin Stoyanov, and Alexis Conneau. 2021. Self-training improves pre-training for natural language understanding. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pages 5408 5418. Online, Association for Computa- tional Linguistics.
Sosuke Kobayashi. 2018 Contextual augmentation: Data augmentation by words with paradigmatic re- lations. In Proceedings of NAACL-HLT, pages 452- 457.

---

Fanshuang Kong, Richong Zhang. Xiaohui Guo, Samuel Mensah, and Yongyi Mao. 2022. Dropmix: A textual data augmentation combining dropout with mixup. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language P Processing, pages 890- 899.
Empirical Methods in Natural Language Processing, pages 4992-5001, Online and Punta Cana, Domini- can Republic. Association for Computational Lin- guistics.
Andrew Matteson, Chanhee Lec, Youngbum Kim, and Heuiseok Lim. 2018. Rich character-leve informa- tion for Korean morphological analysis and part-of- speech tagging. In Proceedings of the 27th Inter- national Conference on Computational Linguistics, pages 2482-2492, Santa Fe, New Mexico, USA. As- sociation for Computational Linguistics.
Varun Kumar, Ashutosh Choudhary, and Eunah Cho. 2020. Data augmentation using pre-trained trans- former models. In Proceedings of the 2nd Workshop on Life-long Learning for Spoken Language Systems, pages 18-26.
Young-Jun Lee, Chac-Gyun Lim, and Ho-Jin Choi. 2020. Korean-specific emotion annotation procedure using n-gram-basee distant supervision and korean- specific-feature-aased distant supervision. In Pro- ceedings of the Twelfth Language Resources and Evaluation Conference, pages 1603-1610.
Dheeraj Mekala, Tu Vu, Timo Sehick, and Jingho Shang- 2022. Leveraging qa datasets to improve generative data augmentation. In Proceedings of the 2022 Con- ference on Empirical Methods in Natural Language Processing, pages 9737-9750.
OpenAI. 2023. Gpt-4 technical report.
Mike Lewis. Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy. Veselin Stoyanox, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequenee pre-training for natural language generation, translation, and com- prehension. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, pages 7871-7880. Online. Association for Computa- tional Linguistics.
Jiao Ou, Jinchao Zhang, Yang Feng, and Jie Zhou. 2022. Counterfactual data augmentation via perspective transition for open-domain dialogues. In Proceed- ings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1635-1648, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instruc- tions with human feedback. Advances in Neural Information Processing Systems, 35:27730-27744,
Bill Yuchen Lin, Wangchunshu Zhou, Ming Shen, Pei Zhou, Chandra Bhagavatula, Yejin Choi, and Xiang Ren. 2020. Commongen: A constrained text gen- eration challenge for generative commonsense rea- soning. In Findings of the Association for Computa- tional Linguistics: EMNLP 2020, pages 1823-1840 1840.
Chanjun Park, Jaehyung Seo, Seolhwa Lee, Chanhee Lee, Hyeonseok Moon, Sugyeong Eo, and Heuiseok Lim. 2021a. BTS: Back TranScription for speech-to- text post-pro processoo using text-to-speech-t--cee Proceedings of the 8th Workshop on Asian Transla- tion (WAT2021), pages 106-116, Online. Association for Computational Linguistics.
Qi Liu, Matt Kusner, and Phil Blunsom. 2021. Counter- factual data augmentation for neural machine trans- lation. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, pages 197-
In
Sungjoon Park, Jeongmin Byun, Sion Baek, Yongseok Cho, and Alice Oh. 2018. Subword-level word vec- tor representations for korean. In Proceedings of the 56th Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers), pages 2429-2438.
Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer. 2020. Multilingual denoising pre- training for neural machine translation. Transac- tions of the Association Computational Linguis- rics, 8:726-742.
Sungjoon Park, Jihyung Moon, Sungdong Kim, Won Ik Cho. Jiyoon Han. Jangwon Park, Chisung Song, Jun- seong Kim, Yongsook Song, Tachwan Oh, et al. 2021b. Klue: Korean language understanding evalua- tion. arXiv preprint arXiv:2105.0066
Yinhan Liu, Myle Oit, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining ap- proach. arXiv preprint arxiv
Timo Schick and Hinrich Schütze. 2021. Generating datasets with pretrained language models. In Pro- ceedings of the 2021 Conference on Empirical Meth- ods in Natural Language Processing, pages 6943- 6951.
llya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In International Confer- ence on Learning Representations.
David Lowell, Brian Howard, Zachary C. Lipton, and Byron Wallace. 2021. Unsupervised data augmenta- tion with naive augmentation and without unlabeled data. In Proceedings of the 2021 Conference on
Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Improving neural machine translation models with monolingual data, In Proceedings of the 54th

---

Annual Meeting of the Association for Computational Linguistics ( (Volume 1: Long Papers), pages 86-96.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. 2018. mixup: Beyond empirical risk minimization. In International Conference on Learning Representations.
Jachyung Sco, Scounghoon Lee, Chanjun Park, Yoonna Jang, Hyeonseok Moon, Sugyeong Eo, Sconmin Koo, and Heui-Seok Lim. 2022. A dog is passing over the jet? text-generation dataset for korean common- sense reasoning and evaluation. In Findings of the Association for Computational Linguisties: NAACL 2022, pages 2233-222 2249..
Xiang Zhang. Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text clas- sification. In Advances in Neural Information Pro- cessing Systems, volume 28. Curran Associates, Inc.
Jing Zhou, Yanan Zheng, Jie Tang, Jian, and Zhilin Yang. 2022. Flipda: Effective and robust data aug- mentation for few-shot learning. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics ( (Volume 1: Long Papers), pages 8646-8665.
Hyun-Je Song and Seong- Bae Park. 2019. Korean mor- phological analysis with tied pruuecceto-sequenee multi-task model. In Proceedings of the 2019 Confer- ence on Empirical Methods in Natural Language Pro- cessing and the 9th International Joint Conference on Natural Laguuage Processing (EMNLP-IJCNIP)) pages
A Experimental Details
Jae Jung Song. 2006. The Korean language: Structure, use and context. Routledge.
We trained our models on single NVIDIA A6000 GPU (48GB) and AMD EPYC 7513 32-Core Pro- cessor CPUs.
Lichao Sun, Congying Xia, Wenpeng Yin. Tingting Liang, S Yu Philip, and Lifang He. 2020. Mixup- transformer: Dynamic data augmentation for nip tasks. Proceedings of the 28th International Con- ference on Computational Linguistics, pages 3436- 3440.
CHEF Modules As a discriminator, CHEF uses et al., 2018; Park et al., 2021a), which has 768 embedding sizes, 768 hid- den sizes, 12 layers, and 12 attention heads. As morpheme blender, CHEF employs KoBART- base? (Lewis et al., 2020), where each encoder and decoder have 768 hidden sizes, 6 layers, and 16 attention heads, respectively.
KLUEE-BERT-bss
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed H Chi, Quoe V Le, Denny Zhou, al. 2022. Chain-of of thought prompting elicits rea- soning in large language models. In Advances in Neurul Information Processing Systems.
Korean Language Understanding Models We used three pre-trained language models for Ko- rean. KLUE-BERT base (Devlin et al., 2018; Park et al., 2021a) has 768 embedding sizes, 768 hidden sizes, 12 layers, and 12 attention heads. KLUE- RoBERTa-base (Liu et al., 2019; Park al., 2021a) also has 768 embedding sizes, 768 hidden sizes, layers, and 12 attention heads. KLUE-ROBERTa- large (Liu et al., 2019; Park et al., 2021a) has 1024 embedding sizes, 1024 hidden sizes, 24 layers, and 16 attention heads.
Jason Wei and Kai Zou. 2019. Eda: Easy data augmenta- tion techniques for boosting performance on text clas- sification tasks. In Proceedings of the 2019 Confer- ence on Empirical Methods in Natural Language Pro- cessing and the 9th International Joint Conference on Natural Language Processing (FMNLP-)JCNIPP, pages 3382-6388.
Jiaxin Wen, Yeshuang Zhu, Jinchao Zhang, Jie Zhou, and Minlie Huang. 2022. AutoCAD: Automatically generate counterfactuals for mitigating shortcut learn- ing. In Findings of the Association for Computa- rional Linguistics: EMNLP 2022, pages 2302-2317, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
EDA & Transaaalo We applied Back- Translation (BT) (Sennrich et al., 2016) and easy data augmentation (EDA) (Wei and Zou, 2019) to the data to generate synthetic data. For BT, we used M2M 100 (Fan et al., 2021) to translate Korean text into Japanese and back into Korean. The model we used for BT is Biccbbook-M21100 (418M), which has 1024 embedding size, 12 layers, and 16 atten- tion heads. We used KoEDA library that utilizes the Korean WordNet (Bikel, 2000) for EDA.
Kang Min Yoo, Dongju Park, Jacwook Kang, Sang- Woo Lee, and Woomyoung Park. 2021. GPT3Mix: Lever- aging large-scale language models for text augmen- tation. In Findings of the Association for Computa- tional Linguistics: EMNLP 2021, pages 2225-22333 2239., Punta Cana, Dominican Republic, Association for Computational Linguistics.
Soyoung Yoon, Gyuwan Kim, and Kyumin Park. 2021, Ssmix: Saliency- based span mixup for text classifi- cation. In Findings of the Association for Computa- tional Linguistics; ACL-IJCNLP 2021. pages 3225- 3234.
httpss//ggttuubb.coouu-eeerereee 2 thttps://githhf wtt.:/coo/////ee Soom//KT-LL/KoBBAR
"https: thwb. wttpe/u..oo//towwi///o///////

---

AEDA & Mixup We used data augmentation with the default settings presented in the AEDA pa- per (Karimi et al., 2021)'. Mixup (Zhang et al., 2018; Sun et al., 2020), which suitable for Transformer-based models, was implemented by randomly shuffling sample indices within batch. Subsequently, the hidden state of each sample and the label scalar were mixed at a ratio of 0.2, follow- ing the default À value, to create synthetic data.
mented output. CHEF augmentations typically take the following form of three types of variations. As described Figure 6, we observe that the out- puts generated by CHEF primarily involve modifi- cations of particles and determiners that are closely associated with nouns. However, no significant changes are observed in the verbs of the sentences.
The combination of lexical morphemes introduces slight variations in the meaning of the generated sentences but does lead to changes in the labels assigned to them.
Large Language Models We included GPT-3.5 (gpt-35-ttubb-0300) (Brown al., 2020; Ouyang al., 2022) and GPT-4 (gpt-4-0314) (OpenAI, 2023) as the large language models. We applied augmentation to the given training data using the OpenAl AP and LangChain",. As depicted in Figure 4 and 5, the prompt consists of an example in one-shot template for the given task and in- structions for the augmentation method. The cost incurred due the OpenAl API calls amounted to $245.17, and the data augmentation approach utiliz- ing LLMs did not show a significant improvement in model performance relative to the cost incurred.
As shown in Figure 7, the augmented output pri- marily consisted of conjugation variations, leading to a diverse range of sentences with different end- ing conjunction rules. However, there is minimal impact on the overall meaning of the sentences themselves. Notably, the augmented hypotheses are labeled as having significantly lower similarity to the corresponding premises regarding semantic similarity evaluation.
As illustrated in Figure 8, the augmentation pro- cess involves single sentences resembling news headlines fitting into the "Life & Culture" sec- tion. Unlike the other benchmark datasets, KLUE- YNAT is characterized by the fact that the original sentences contain relatively fewer functional mor- phemes. Based on this characteristic of the dataset, CHEF was applied to augment the data by chang- ing the order or role of the lexical morphemes the sentence.
Hyperparameters Hyperparameters to train the CHEF are batch size 8, learning rate x 10 AdamW optimizer (Loshchilov and Hutter, 2019) (3; = 0.9, f2 = 0.999,
le - 8), lambda 0.2,
max source length 200, max target length 168, and 5 epochs. To train Korean language understanding models is batch size 32, learning rate 2 x 10-5 AdamW optimizer (3, 0.9, f2 = 0.999, 1e-8), and 20 epochs. In the case few-shot learn- ing, all other hyperparameter settings remained un- changed except for the epoch adjusted to 30, con- sidering the model's overfitting point.
Decoding Strategy Within the framework of CHEF augmentation, we imposed specific con- straints the decoding strategy. We established the following settings: beam size of 10, maxi- mum sequence length 168, minimum sequence length of 5, a repetition penalty of 2, and no- repeat n n-gram size of 3 to penalize the generation duplicate tokens.
Qualitative Analysis
We applied CHEF to several benchmark datasets and conducted qualitative analyses of the aug-
/tnttps:/oopeai.ccom "https: www.:/yttton...ncaaiaaa.a cwm/en///////// mommeen/latss.

---

Model
KLUE--LI
KorNLI-MNII
KorNLI-SNLI
KLUE-STS
KLUE-YNAT
NSMC
BERT
81.53
80.63
71.56
90.85
85.73
90.30
+CHEF 1%
82.63
81.07
72.06
91.26
86.68
90.52
+CHEF 2%
81.47
79.83
71.60
91.12
86.76
90.61
+ +CHEF 3%
81.57
79.85
70.61
91.36
86.63
90.39
+CHEF 5%
82.30
79.81
70.63
91.52
86.51
90.50
+CHEF 10%
81.43
79.85
70.33
91.05
86.90
90.41
RoBERTa K Base
84.83
80.84
71.84
92.50
85.07
90.08
+CHEF 1%
86,00
81.77
72.22
93.05
86.18
90.74
+CHEF 2%
85.47
81.63
71.50
93.16
86.28
90.70
+CHEF 3%
86.20
81.11
71.58
92.71
86.64
90.76
+CHEF 5%
85.77
80.70
71.19
92.35
86.23
90.80
+CHEF 10%
85.53
81.17
69.94
92.83
86.58
90.69
RoBERTa Large
89.17
81.73
73.26
93.35
85.69
91.28
+CHEF I %
90.47
82.76
73.56
93.61
86.12
91.38
+CHEF 2%
90.30
82.98
73.40
93.21
86.63
91.35
+CHEF 3%
90.00
82.44
74.14
93.40
86.30
91.33
+CHEF 5%
90.07
83.08
73.17
93.29
87.02
91.28
+CHEF 10%
89.53
83.12
73.23
92.69
86.64
91.34
Table 7: Performance of language models multiple classification datasets by adjusting CHEF's augmentation rate. The best- settperforming method on each dataset is shown in bold. Underlines indicate that the method outperformed the baselines.
(a) Considering the "premise" and "labels" of the given example, paraphrase the hypothesis that the can augmented.
(b) Use the Following JSON format!
ORIGINAL DATA: (""ypothesiis": " -
"premise": "1a 2 ojat 4 (4
Z 45#j0R 0| 210 #822
2 -
"labels": "entailment")
(c) YOUR ANSWER: ""hyothhesss"" ": " 2|4 7/
88 J 20440
7."
"premise": "/ و 26 / 0 E
0
4 0z 0
"
"labels": "entailment")
Begin!
(d)
ORIGINAL DATA: (input sequence)
YOUR ANSWER: {generated L.M output)
Figure 4: An example of a prompt template for augmenting data with given original data. (a) is a human instruction to paraphrase hypothesis of given data. (b) is an additional directive to limit the output to JSON format. (c) example of paraphrasing. (d) is the format of the given input sequence and the generated LLM output.

---

Considering the "premise" and "labels" of the given example, make the counterfactual hypothesis so that the data can be augmented.
A SET "labels": ["entailment", "contradiction", "neutral").
(a)
You should randomly select one of the labels in A SET OF "labels" and change the "labels" in YOUR ANSWER different from the "labels" the ORIGINAL DATA and paraphrase an appropriate "hypothesis".
(b) Use the Following JSON format!
ORIGINAL DATA: (("hypothesis": orga 78440,,
"premise": " - 98 =
2 4 0 7824
2 "
"labels": "entailment")
(c)
YOUR ((
hypothhsss"" " x o _ 382
"premise": . -
o|4 3 34
ビ o 45 0 - 432
3 J 0 -
"labels": "contradiction")
Begin!
(d)
ORIGINAL DATA: (input sequence)
YOUR ANSWER: (generated LLM output)
Figure 5: An example of prompt template for augmenting data with given original data. (a) is human instruction for generating counterfactual sequence for hypothesis on a given input and instructions how the label of the counterfactual sequence. (b) is an additional directive to limit the output to JSON format. (c) is an example of counterfactual sequence generation. (d) the format of the given input sequence and the generated LLM output.
KLUE-NLI
Contradiction
Premise: 2 B But q 0 2 : (There due the Instagram downtime the moming of the 14ah day)
Hypothesis:
14 44 क 2 / @22 (There were no errors whatsoever throughout the 14th day)
CHEF Augmented
Lexical Morphemes: (149) W8 R 2
CHEF Hypothesis ( 14912 o
2
lxa 922 (No errors occurred on the 14th day)
CHEF Hypothesis 14 L
g (No errors occureed the 14th day well.)
CHEF Hypothesis ( 14 2 2
212 (No errors occurred in the morning of the 14th day)
CHEF Hypothesis 4 2
22 (No emors occurred.)
CHEF Hypothesis 5 14 t 말
2
2 ||
[) (There errors whatsoever on the 14 day)
Figure 6: An example augmentation result from CHEF KLUE-NLI

---

KLUE-STS
Similarity 0.1
Premise: 0 27
_ 24|M # E 2 인ा 5 8 24 A 2 4
(The winteny atmosphere of France, which known for elegant ambiance, along with the scattered lighting and buildings throughout
cnitiiiiiiint blended perfectly)
Hypothesis: da o 80 lll lbय/ A _
(The location, price range, and facilities the accommodations in my travel itinerary were satisfartory.)
CHEF Augmented
Lexical Morphemes: (o 288, MA, Latcc N/2. (I.-)
CHEF Hypothesis ① RIA 2I3 Al SIAJ, 9128 5 aE 2 9244
was satisfied with everything, including the location, price range, facilities, and travel itinerary.
CHEF Hypothesis @ R1l,, 2H2 AlM, 91A), R|d $ 01E8 Mgo c=
satisfied with location, range. facilities, overall travel itinerary, including the locations visited. CHEF Hypothesis 3 RE df 719 AlM NA, AA a e प..
was satisfied with everything, including the location travel accommodations, price faciiities and overall itinerary.)
Figure 7: An example of augmentation result from CHEF in KLUE-STS
KLAT
Life & Culture
Title: 2 2 | य
(Bracing for Strong Winds and Subzero Temperatures... Afternoon Rain the Forecast)
CHEF Augmented
Lexical Morphemes: (2) S. 28/22 HJ, 2. 41, 20
CHEF Hypothesis ① 4 2 712 2 22 23 28
Afternoon... Strong CHEF Hypothesis 2 μ Z 말 @ 2.....44 -
(Temperature Dropping
Subfreezing Temperatures in Afternaon)
(Aftemoon Gale Winds Subfreezing Conditions... Rain Expected from Aftemoon Evening)
Hypothesis 3 4 2 _
2
(Subfreezing Conditions Expected from Tomorrow Afteroon... Strong Winds the Afternoon)
Figure example of augmentation result from CHEF KLUB-YNAY


Reviews:
[{"review_id": "IRR5xVUNu1", "reviewer": "Reviewer_ACJB", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your in-depth points. \n\n> R1-1: *'This paper uses mecab-ko analyzer to extract morpheme ingredients, but the analyzer does not provide linguistically correct Korean morphemes, failing to recover original morphemes in irregular verbs and adjectives. More explanation is needed for this choice and its impact on the performance.'*\n\n**The mecab-ko analyzer is capable of restoring the original form of Korean morphemes**. This analyzer may not always perform perfectly, having some error rate depending on the domain and the data. **However**, the issue raised by the reviewer regarding the *'failing to recover original morphemes in irregular verbs and adjectives'* **is almost negligible when using mecab-ko analyzer with CHEF**. **In cases where the irregular form should be applied, CHEF can correctly apply the irregular change**, and likewise, in cases where the regular form should be applied, CHEF can correctly apply the regular change.\n\nFor example, in the KLUE-STS dataset augmented by CHEF, instances of the 'ㄷ' irregular change and regular change both appeared. \n\nex) Synthetic 1: KOR: … 더 많이 묻는… / ENG: ... asking more … (‘ㄷ' irregular change)\n\nSynthetic 2: KOR: … 더 많이 물어보는… / ENG: ... asking more … (regular change)\n\nThe majority of the synthesized sentences were **mapped according to regular/irregular changes**, with lexical morphemes correctly aligned with functional morphemes.\n\nTo elaborate further, the mecab-ko analyzer has been proven effective in restoring the original form of Korean morphemes in various research and posts. **It is a well-recognized open-source tool that has significantly contributed to Korean morphological analysis research.** We do not believe that the following studies, which employ this analyzer, are using an incorrect tool for Korean morpheme analysis [1] [2] [3] [4].\n\n[1] *Park, K., Lee, J., Jang, S., & Jung, D. (2020, December). An Empirical Study of Tokenization Strategies for Various Korean NLP Tasks. In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing (pp. 133-142).*\n\n[2] *Park, C., Eo, S., Moon, H., & Lim, H. S. (2021, June). Should we find another model?: Improving neural machine translation performance with ONE-piece tokenization method without model modification. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Papers (pp. 97-104).*\n\n[3] *Moon, S., Cho, W. I., Han, H. J., Okazaki, N., & Kim, N. S. (2022, June). OpenKorPOS: Democratizing Korean Tokenization with Voting-Based Open Corpus Annotation. In Proceedings of the Thirteenth Language Resources and Evaluation Conference (pp. 4975-4983).*\n\n[4] *Kim, A., & Kim, J. (2022, May). Vacillating Human Correlation of SacreBLEU in Unprotected Languages. In Proceedings of the 2nd Workshop on Human Evaluation of NLP Systems (HumEval) (pp. 1-15).*\n\nWe also aimed to use the **morpheme analyzer that has previously demonstrated its effectiveness**, taking into account the performance of morphological analyzers can be significantly influenced by the data or domain in which they are used. **The Korean text-generation research by Seo et al. 2022 [5] has demonstrated** that when mecab-ko analyzer is combined with a generative language model, it can **achieve performance close to the upper bound of human-level capability**. This proves that the **mecab-ko analyzer can properly handle both irregular and regular morphological rules** and can effectively reconstruct sentences when paired with a generative language model.\n\n[5] *Seo, J., Lee, S., Park, C., Jang, Y., Moon, H., Eo, S., ... & Lim, H. S. (2022, July). A dog is passing over the jet? a text-generation dataset for korean commonsense reasoning and evaluation. In Findings of the Association for Computational Linguistics: NAACL 2022 (pp. 2233-2249).*\n\n\n> R1-2: *‘This paper does not appropriately discuss the linguistic issues on irregular changes in Korean language generation’*\n\n**We thank you for your high level of expertise in language and the important points you have raised.** The proportion of morpheme combination rules in CHEF can differ depending on which morpheme analyzer is used to construct the morpheme set. As illustrated in Figures 1, 7, 8, and 9, CHEF using the mecab-ko analyzer shows a higher proportion of augmentation through regular changes. \n\nIn response to your comments, **we will supplement the insufficient explanation about irregular changes on Page 4, Line 232, and add to the Limitation section about the morpheme analyzer.**\n\nThe content we intend to add to the Limitation section is as follows:\n\n*'The mecab-ko analyzer, which we used for constructing the set of morphemes, can have an error rate depending on the domain and data it is applied to. The proportion of morpheme combination rules in CHEF may differ based on which morpheme analyzer is used to construct the morpheme set. Therefore, leveraging a more advanced morpheme analyzer could enable data augmentation that more accurately reflects the linguistic characteristics of the Korean language.'*\n\nWe thank you for your insightful comments, which will considerably improve the paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The method showed significant improvement in the KLUE benchmark test even though it uses relatively smaller language model (BERT and BART) than GPT4. The paper is easy to follow and the extensive experiments support the proposed method.", "reasons_to_reject": "This paper does not appropriately discuss the linguistic issues on irregular changes in Korean language generation: this paper uses mecab-ko analyzer to extract morpheme ingredients, but the analyzer does not provide linguistically correct Korean morphemes, failing to recover original morphemes in irregular verbs and adjectives. More explanation is needed for this choice and its impact on the performance.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper proposes an efficient data augmentation method for Korean language by generating diverse sentence forms using lexical and functional morphemes. The architecture is based on a generator-discriminator model for a sentence generation and its label consistency. This method is evaluated with the KLUE benchmark and showed its efficiency.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "JjA1civPfQ", "reviewer": "Reviewer_p68b", "summary": "", "questions": "Please clarify the scope of the morphological blender and the label discriminator (word or sentence level)\n\nPlease clarify the input of the morphological blender w.r.t. lexical and functional morphemes.\n\nPlease clarify the role of KoBART and KLUE-BERT in the morphological blender and the label discriminator respectively.", "limitations": "", "rating": 3, "confidence": 2, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you so much for your comments. Please check the following regarding your concerns.\n\n> R1-1 & Q1: *'Because the focus is on morphology, which is usually word-level, it is not clear that the morpheme blender and label discriminator are sentence-level, and not word level & Please clarify the scope of the morphological blender and the label discriminator (word or sentence level)'*\n\nWe **extract lexical morphemes** with substantive meaning **from the original sentence** **segmented at the morphological level** for data augmentation. The **morpheme blender attaches functional morphemes to these lexical morphemes to create sentences** of different forms but similar meanings. Meanwhile, the label discriminator ensures that the sentence generated by the morpheme blender retains the original sentence's label (e.g., +1 or -1).\n\nThe confusion may arise from the unique feature of Korean, where multiple morphemes within a single word can be detached and recombined. **Our proposed method involves decomposing a single sentence into its morpheme units and then reconstructing it based on lexical morphemes.**\n\nTherefore, there is **no need to distinguish between word-level and sentence-level** in the context of our proposed augmentation.\n\n> R1-2: *'It would be very useful to make this clear from the beginning (if indeed this is the case). It would also be useful to provide some concrete example label (maybe in Figure 1, instead of the abstract \"+1\" label)'*\n\nWe agree that a clearer explanation for Figure 1 would be beneficial. In line with the reviewer's suggestion, **we will add a concrete example to Figure 1** and **clarify the explanation about the label by revising it to include concrete labels (e.g., positive & negative).** Additionally, we will enhance the content of the caption to reflect these changes, ensuring that authors who are not familiar with Korean will not experience misunderstanding.\n\n**We will also make revisions to strengthen the content on page 2, lines 69-71 of the Introduction section. The revised text will be as follows:**\n\n*'The role of the morpheme blender is to take the concatenated sequence of morphemes and returns an intact sentence that contains given morphemes. Label discriminator aids morpheme blender to generate a label-consistent sentence by comparing encoded representations generated by the morpheme set and a whole sentence.'*\n\n> R2-1 & Q2: *'Based on the examples in the appendix, it seems that the morphological blender receives as input only lexical morphemes. Is this the case? If yes, where do the functional morphemes come from? & Please clarify the input of the morphological blender w.r.t. lexical and functional morphemes.'*\n\n**Partially Yes**. As depicted in Figure 2, the morpheme blender requires three (or two) types of inputs for the encoder-decoder architecture: **(1) a morpheme set, (2) a label word, (3) a premise (optional)** for teacher forcing and contrastive learning. **The morpheme set (1) consists only of lexical morphemes**.\n\nIn accordance with the **conditional generation task**, **the morpheme blender uses the sequence-to-sequence model (we used 'KoBART'), which takes lexical morphemes as an input source and attaches functional morphemes to generate a complete sentence**. The rules for attaching functional morphemes are based on the **linguistic prior knowledge of the Korean language** as captured by the pre-trained language model, KoBART.\n\nThe paper on page 4, lines 235 - 239 covers this process, where the rules for attaching functional morphemes applied by the morpheme blender are considered part of **the model-aware recipe**.\n\nFor clearer elucidation, **we will specify in Figure 2 that the morpheme set consists of lexical morphemes**. Additionally, **we will revise the text on page 4, line 227**, from '*denotes the number of morphemes*' to '***denotes the number of lexical morphemes.***'\n\n> R2-2: *'Or the blender is more than a blender, and more a generator? If yes, maybe the term \"morphological blender\" is a bit misleading.'*\n\n**The core of CHEF lies in blending lexical and functional morphemes**. As illustrated in Figure 2, we also utilize contrastive loss to incorporate label information into the blend. To clarify that the output of the morpheme blender results in sentence generation through the decoder, we provided a detailed explanation about **“teacher forcing” in a bold paragraph on page 4, lines 250-269**. Direct explanations regarding **\"sentence generation\" are also given on page 4, lines 251 and 266-269**. Furthermore, through the **“Augmentation Pipeline” in Section 3.4 on page 5**, we indicated that the morpheme blender is involved in the sentence generation process.\n\n**To address the reviewer's question more clearly in the paper, we will add an explanation regarding the term \"morpheme blender.\"** The added text will be as follows:\n\n*'Morpheme blender takes morphemes and generates a sentence that contains given morphemes. In this sense, we denote it as a blender.'*\n\n> R3: *'It would be useful to include a brief description of CommonGen, and that should help provide a more informative contrast with the method proposed here.'*\n\nWe will add an explanation about CommonGen according to your advice. The additional text will be placed on page 3, line 173, as follows:\n\n'*Both conditional generation tasks use a set of input concept set* $\\mathcal{C}$, *where* $\\{ c_1, c_2, \\ldots, c_m \\}$ *to make a target sentence* $\\mathcal{T}$, *where* $\\mathcal{T} = \\{ t_1, t_2, \\ldots, t_n \\}$.\n*Generative language models for those tasks learn a function* $\\mathcal{f}: \\mathcal{X} \\rightarrow \\mathcal{Y}$ *that maps* $\\mathcal{C}$ *to* $\\mathcal{T}$'.\n\n> R4 & Q3: *'Section 4.2. is not very clear. What does KoBART correspond to in the morpheme blender? What does KLUE-BERT correspond to in the label discriminator? & Please clarify the role of KoBART and KLUE-BERT in the morphological blender and the label discriminator, respectively.'*\n\n**As described in Page 3, lines 191-202**, KoBART is the backbone model for the morpheme blender and features an encoder-decoder architecture. On the other hand, KLUE-BERT-base, an auto-encoder architecture, functions as the backbone for the label discriminator. \n\n**We appreciate the reviewer's comments and will clarify the description in Section 4.2 as follows**:\n\n*'The morpheme blender employs an encoder-decoder structure and uses KoBART, a pre-trained generative language model for Korean, as its backbone. KoBART takes an input sequence consisting of a lexical morpheme set, a label word, and an optional premise to generate synthetic data. The label discriminator features an encoder architecture and utilizes KLUE-BERT-base, a pre-trained language model for Korean, as its backbone. KLUE-BERT-base conveys the original sentence's label to KoBART through a contrastive loss.'*", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper addresses issues important for morphologically rich languages, and adds to the work on automatic data augmentation. There is extensive evaluation on a variety of datasets, and with different LLM + data augmentation options, in the full training set, using different augmentation ratios, and few-shot settings.", "reasons_to_reject": "Several issues could be clarified:\n\n- because the focus is on morphology, which is usually word-level, it is not clear that the morpheme blender and label discriminator are sentence-level, and not word level. It would be very useful to make this clear from the beginning (if indeed this is the case). It would also be useful to provide some concrete example label (maybe in Figure 1, instead of the abstract \"+1\" label)\n\n- based on the examples in the appendix, it seems that the morphological blender receives as input only lexical morphemes. Is this the case? If yes, where do the functional morphemes come from? Or the blender is more than a blender, and more a generator? If yes, maybe the term \"morphological blender\" is a bit misleading.\n\n- it would be useful to include a brief description of CommonGen, and that should help provide a more informative contrast with the method proposed here.\n\n- Section 4.2. is not very clear. What does KoBART correspond to in the morpheme blender? What does KLUE-BERT correspond to in the label discriminator?", "questions_to_authors": "Please clarify the scope of the morphological blender and the label discriminator (word or sentence level)\n\nPlease clarify the input of the morphological blender w.r.t. lexical and functional morphemes.\n\nPlease clarify the role of KoBART and KLUE-BERT in the morphological blender and the label discriminator respectively.", "paper_topic_and_main_contributions": "The paper presents a method for data augmentation for Korean based on generating morphological variations while preserving a sentence-level label.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "BtLcqKYIVo", "reviewer": "Reviewer_nopA", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your careful and valuable comments.\n\n> R1: *'There are some other data augmentation methods (Guo, H. 2020, Karimi et al., 2021) that should be included for comparison'*\n\nWe appreciate your suggestion and will certainly include experiments comparing with **MIXUP** and** AEDA** in our evaluation. The results with these two methods in **full-shot learning** using default settings are as follows:\n\n1) KLUE-BERT-base\n\n|Method|KLUE-NLI|Kor-MNLI|Kor-SNLI|KLUE-STS|KLUE-YNAT|NSMC|\n|--- |--- |--- |--- |--- |--- |--- |\n|Baseline|81.53|80.63|71.56|90.85|85.73|90.30|\n|AEDA [5] |81.47|80.76|69.52|90.86|86.35|90.35|\n|MIXUP [2,4]|82.10|80.14|71.74|90.97|86.28|90.40|\n|CHEF|**82.63**|**81.07**|**72.06**|**91.26**|**86.68**|**90.52**|\n\n2) KLUE-RoBERTa-base\n\n|Method|KLUE-NLI|Kor-MNLI|Kor-SNLI|KLUE-STS|KLUE-YNAT|NSMC|\n|--- |--- |--- |--- |--- |--- |--- |\n|Baseline|84.83|80.84|71.84|92.50|85.07|90.08|\n|AEDA [5] |85.57|80.56|71.70|92.80|85.19|90.63|\n|MIXUP [2,4]|85.83|80.60|70.85|92.77|85.55|90.00|\n|CHEF|**86.00**|**81.77**|**72.22**|**93.05**|**86.18**|**90.74**|\n\n\n3) KLUE-RoBERTa-large\n\n|Method|KLUE-NLI|Kor-MNLI|Kor-SNLI|KLUE-STS|KLUE-YNAT|NSMC|\n|--- |--- |--- |--- |--- |--- |--- |\n|Baseline|89.17|81.73|73.26|93.35|85.69|91.28|\n|AEDA [5] |90.17|81.45|72.26|93.47|85.56|91.28|\n|MIXUP [2,4]|**90.83**|81.95|72.39|93.43|85.99|90.33|\n|CHEF|90.47|**82.76**|**73.56**|**93.61**|**86.12**|**91.38**|\n\nHowever, it is worth noting that **Nonlinear Mixup** [1] is primarily suited for **CNN/LSTM**-based neural network models and is known to be challenging to fully apply to Transformer-based pre-trained language models [2] [3]. **Therefore, we have used the more advanced Mixup approach [2] [4], which is compatible with pre-trained language models**, to evaluate the performance.\n\nThe **two newly added baselines** demonstrate good augmentation effects but also **face challenges in improving performance on Natural Language Inference (NLI) tasks**. Furthermore, they also tend to be **unstable** in their performance like other baselines, sometimes completely **failing to converge** depending on the initial seed value, which often necessitates the assignment of a new seed for taking the results.\n\nWe will include the complete experimental results in the final version of the paper.\n\n[1] Hongyu Guo. Nonlinear Mixup: Out-Of-Manifold Data Augmentation for Text Classification. AAAI'20.\n\n[2] Sun, L., Xia, C., Yin, W., Liang, T., Philip, S. Y., & He, L. (2020, December). Mixup-Transformer: Dynamic Data Augmentation for NLP Tasks. In Proceedings of the 28th International Conference on Computational Linguistics (pp. 3436-3440).\n\n[3] Yoon, S., Kim, G., & Park, K. (2021, August). SSMix: Saliency-Based Span Mixup for Text Classification. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 (pp. 3225-3234).\n\n[4] Kong, F., Zhang, R., Guo, X., Mensah, S., & Mao, Y. (2022, December). Dropmix: A textual data augmentation combining dropout with mixup. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (pp. 890-899).\n\n[5] Karimi et al. AEDA: An Easier Data Augmentation Technique for Text Classification. EMNLP'21 Findings.\n\n> R2: *'It would be better if the scope of the paper is larger, e.g., testing the method on Japanese (also an agglutinative language) datasets.'* \n\nThank you for the valuable suggestion. The scope of this research aimed to demonstrate the effectiveness of text augmentation by leveraging the \"agglutinative\" characteristics specific to the “Korean language”. **In this study, we aimed for a vertical analysis** focused on one language. **In future research**, we plan to analyze the effectiveness of \"CHEF\" in other agglutinative languages like Japanese, Turkish, etc., **as you suggested, to broaden the scope of our study horizontally**.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The proposed method is intuitive from the perspective of linguistics.\n- Experiments prove the effectiveness of the proposed method on several tasks.", "reasons_to_reject": "- There are some other data augmentation methods (Guo, H. 2020, Karimi et al., 2021) that should be included for comparison.\n- It would be better if the scope of the paper is larger, e.g., testing the method on Japanese (also an agglutinative language) datasets.\n\n\n1. Hongyu Guo. Nonlinear Mixup: Out-Of-Manifold Data Augmentation for Text Classification. AAAI'20.\n1. Karimi et al. AEDA: An Easier Data Augmentation Technique for Text Classification. EMNLP'21 Findings.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper proposes a data augmentation method for Korean, named CHEF, which can generate augmented samples with more diverse morphemes. Experiments show that the proposed CHEF for data augmentation can achieve the best performance on several Korean downstream tasks and outperform state-of-the-art LLMs (i.e., GPT-3.5, GPT-4).", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a data augmentation method for Korean by blending lexical and functional morphemes together to generate new forms.

The reviewers found the proposed method intuitive from a linguistic point of view and acknowledged its effectiveness on several tasks. In the initial round of reviews, the reviewers identified various weaknesses, ranging from the impact of the chosen morphological analyzer (mecab-ko) over general comprehension issues to lack of comparison with existing methods. The authors were able to engage in discussions with all three reviewers and provided additional results that, if included in the final version, would make the paper significantly stronger. All reviewers increased their scores as a result of the discussion with the authors.
Decision: Accept-Main