# Controlled Text Generation of Lung Cancer Descriptions for Small Sample Data
## 4 Results
### Experimental Settings
#### 4.1.3 Experimental Indicators  
To compare the performance of various models, this study introduces indicators for measuring the pros and cons of modeling methods. The study mainly focuses on the model's prediction accuracy and the loss function's value. To evaluate text quality, we use a combination of ROUGE [35], cosine similarity [36], Bert score [37], and manual evaluation [38]. The computation of the ROUGE metric rests on various aspects of n-gram overlap and word order. These features can aid in assessing the quality, fluency, and content similarity of the generated text. The ROUGE score is frequently derived using the F1 value [39] as a measure, whereas ROUGE-N [40] concentrates on local matching and performs better for matching phrases. However, it does not effectively capture the overall coherence of the text. While ROUGE-L's focus on word overlap at both the long- and short-range helps evaluate the performance of generative tasks, it is not as effective in measuring the quality of the generated text using the method proposed in this study. To compare with other models, this study calculates the F1 value of ROUGE-L [41] between the generated text and the original dataset. The formula for calculating ROUGE-L is as follows:  
\[R_{lcs}=\frac{LCS(X,Y)}{m} \tag{7}\]  
\[P_{lcs}=\frac{LCS(X,Y)}{n} \tag{8}\]  
\[F_{lcs}=\frac{\left(1+\beta^{2}\right)R_{lcs}P_{lcs}}{R_{lcs}+\beta^{2}P_{lcs}} \tag{9}\]  
Figure 5: Distribution of public dataset samples. This figure shows the sample distribution of cervical, rectal, liver, gastric, and lung cancers in the Chinese Medical Q&A Dataset. Among these, lung cancer generally has more data than other cancers.  
\(LCS(X,Y)\) represents the length of the longest common subsequence of text \(X\) and text \(Y\) and considers the order, and \(m\) and \(n\) denote the length of the original text and the generated text, respectively. \(R_{lcs}\) and \(P_{lcs}\) denote the recall and accuracy, respectively, and \(F_{lcs}\) is the value of the final ROUGE-L.  
Cosine similarity measures the similarity between two vectors. It is widely used in text generation to compare the degree of similarity between two texts since it is insensitive to changes in text length and focuses on content similarity. The practical significance of cosine similarity in the experiments detailed in this study is notable. The formula for cosine similarity is as follows:  
\[\cos(\theta)=\frac{\sum_{i=1}^{n}(x_{i}\times y_{i})}{\sqrt{\sum_{i=1}^{n}(x_ {i})^{2}}\times\sqrt{\sum_{i=1}^{n}(y_{i})^{2}}} \tag{10}\]  
\(X_{i}\) and \(Y_{i}\) represent the word frequency of a word in the original text and the generated text, respectively, and \(cos(\theta)\) is the final result; the closer to 1, the more similar the generated text is to the original text in terms of content.  
Bert score is used to represent the similarity between the generated text and the reference text by embedding the text using the BERT model and then calculating the precision, recall, and F1 score. Compared to traditional text similarity measures, Bert score uses context-sensitive word embedding to capture deeper semantic information.  
Different lung cancer types vary in importance. Oncologists from a tertiary healthcare facility compiled a scoring table for descriptions of lung cancer (refer to Appendix A). This study randomly selected an equal amount of initial and generated texts based on the scoring table and calculated the average scores to measure the quality of the generated texts.