# Controlled Text Generation of Lung Cancer Descriptions for Small Sample Data
## 2 Literature Review  
### Controlled Text Generation  
Several modeling approaches have surfaced during the current research stage on controlled text generation. Initially, text generation systems were mainly rule-based and employed predefined templates and rules to generate text. However, this approach lacked flexibility and generalization, impeding its ability to adapt to diverse output requirements and complex contexts. Statistical natural language processing techniques have become popular, leading to the widespread use of N-gram language modeling [9]. This method uses statistics to analyze lexical frequencies and contextual relationships in a corpus, generating coherent texts. However, there are limitations inherent to this model that make it challenging to control the results of text generation precisely. Following previous work by [PERSON] et al. [10], a model known as the Generative Adversarial Network (GAN) was proposed. The GAN is a framework consisting of a generator and a discriminator. The generator generates text samples similar to real text data, while the discriminator distinguishes between real and generated text. This process encourages the generator to continually improve the generated text's quality. In 2014, the Sequence-to-Sequence (Seq2 Seq) model was proposed by [PERSON] et al. [11]. The Seq2 Seq has achieved significant success in the generation of text. This model can map input sequences of various lengths, such as sentences or paragraphs, to output sequences of variable length, such as generating text summaries or dialogs. Nonetheless, further improvement is necessary to achieve complete control over text generation and implement additional research to achieve this goal. With the emergence of ChatGPT, many text generation models have been developed. [PERSON] et al. [12] succeeded by introducing Label Logic and Multi-hop Attention (LLaMA), a large-scale language model with four-parameter scales, currently competing among the top language model batches. [PERSON] et al. [13] introduced the Adaptable Language Model (Alpaca), a language model fine-tuned by Stanford University on Meta's open-source LLaMA-7B model and designed to follow instructions. It performs effectively on a wide range of text generation tasks. Moreover, Google has recently launched the Bard [14] language model, which is built on the Language Model for Dialogue Applications (LaMDA) [15]. The model's impact on text generation is noteworthy. Supervised training models can be impractical for small-scale research or organizations lacking computer hardware. This is because these models consume significant hardware resources. In response to this issue, unsupervised text generation models are gaining popularity. In recent years, the Variational Autoencoder (VAE) model, a variant of the ground echo state network first presented by [PERSON] et al. [16], has become widely adopted. However, concerns have been raised regarding the model's output ambiguity and computational complexity. In 2017, [PERSON] et al. [17] from Google introduced the Transformer model, a neural network architecture based on the attention mechanism. This model marked a significant breakthrough in machine translation tasks and has since been widely used in text summarization, dialogue systems, and narrative generation. The Transformer model employs non-sequential models to process sequential data in a groundbreaking development within the field. After this, several transformer-based models have emerged that accurately reflect OpenAI's series of pre-trained language models created using the Transformer-based Generative Pre-Trained Transformer (GPT) [18] and Google's pre-trained language models named Transformer-based Bidirectional Encoder Representations from Transformers (BERT) [19]. The GPT series of models generates excellent text by pre-training on extensive corpus data and refining through text generation tasks.