## 1 Introduction  
Data imbalance is a common issue in classification tasks having adverse effects on the model's performance. The availability of an equal number of samples per category for a real-case scenario in most application domains is nearly impossible. Often common classes end up with far more samples than the least common ones. Researchers address this issue by utilizing data over-sampling techniques for generating synthetic data from the original training samples. For random numerical data, techniques such as synthetic minority oversampling technique (SMOTE) [1] and AdaSyn [2] works well. For Images, deep learning methods employing generative adversarial networks (GAN) such as CycleGAN [3] have performed reasonably well. However, unlike numerical data and often images, synthetic text suffers greatly from contextual and semantic information loss. The generated text often ends up with poor grammar and text structure, thereby losing its meaning. For example, below are such few generated texts. * teacher next time sir litt time improvement
* well buarning hanse
* sir try concernst subject subject
* method teacher good
* teacher good teacher time properly experienced  
Linguistics communication is a thought process that is encoded by the speaker and decoded by the listener. A message can be coded and decoded differently depending on the people involved in the communicative process. The linguistic utterances resulting from communication are composed of compressed information omitting the apparent details that statistical models lack to process the information, often resulting in a poorly constructed sentence. Therefore, language models can easily go astray, generating text sequences when subjected to training data consisting of metaphors, metonymy, hidden relations, quantifier scope, and lexical ambiguity. For instance, \"Do not worry about [PERSON], he is a rock\" is an example of a metaphor where the context \"he is solid like a rock\" is understood both by the speaker and the listener due to common background knowledge. However, most language models may infer [PERSON] to be a rock and will construct synthetic sentences accordingly. Albeit challenging, many models can be proposed to provide a shared background domain knowledge and lexicon and grammar validation for language understanding. The accuracy achieved by various existing language models is far below the acceptable threshold to be employed successfully in various text sequence generation tasks. Language models can be evaluated subjectively by humans/linguistic experts or objectively using performance evaluation metrics to evaluate a system's performance. Two metrics commonly used to evaluate the quality of generated text for correctness, contextual & semantic meaning are Bilingual Evaluation Understudy (BLEU) [4] and Recall-Oriented Understudy for Gisting Evaluation (ROUGE) [5]. BLEU tries to capture the quality of the generated text by the machine to that of humans. The closer a synthetic text is to a human translation, the better it is. On the other hand, ROUGE is a set of metrics for evaluating text summaries to that of humans. There are different variants of ROUGE available to compare on a unigram (ROUGE-1), bigram (ROUGE-2) to n-gram (ROUGE-N), or even longest common sub-sequences (ROUGE-L) based statics, among others. These metrics are only useful as long as the generated text has many overlapping words to the reference text. On the other hand, the embedding-based metrics may provide better accuracy representation for longer sequences of text generation, as explained in Section 4.4.2. This study aims to evaluate the performance of the proposed LSTM-based text generation algorithm trained for domain-specific applications and subject them to the classification task, in addition to evaluating the recently proposed GPT-2 [6] model.