Topic-Guided Self-Introduction Generation for Social Media Users
Year: 2023

Abstract:
Millions of users are active on social media. To allow users to better showcase themselves and network with others, we explore the autogeneration of social media self-introduction, a short sentence outlining a user’s personal interests. While most prior work profiles users with tags (e.g., ages), we investigate sentencelevel self-introductions to provide a more natural and engaging way for users to know each other. Here we exploit a user’s tweeting history to generate their self-introduction. The task is non-trivial because the history content may be lengthy, noisy, and exhibit various personal interests. To address this challenge, we propose a novel unified topic-guided encoderdecoder (UTGED) framework; it models latent topics to reflect salient user interest, whose topic mixture then guides encoding a user’s history and topic words control decoding their self-introduction. For experiments, we collect a large-scale Twitter dataset, and extensive results show the superiority of our UTGED to the advanced encoder-decoder models without topic modeling. 1

heading: 1 Introduction
text: The irresistible popularity of social media results in an explosive number of users, creating and broadcasting massive amounts of content every day. Although it exhibits rich resources for users to build connections and share content, the sheer quantities of users might hinder one from finding those they want to follow (Matikainen, 2015). To enable users to quickly know each other, many social platforms encourage a user to write a self-introduction, a sentence to overview their personal interests.
A self-introduction is part of a self-described profile, which may else include locations, selfies, user tags, and so forth, and is crucial in online user
∗ Corresponding author 1Our code and dataset are released at https://github.
com/cpaaax/UTGED.
interactions (McCay-Peet and Quan-Haase, 2016). Previous findings (Hutto et al., 2013) indicate users tend to follow those displaying self-introductions because a well-written self-introduction will brief others about a user’s interests and facilitate them to initialize connections. It would benefit users in making like-minded friends and gaining popularity; whereas not all users are skillful in writing a good self-introduction. We are thus interested in how NLP may help and study self-introduction generation, a new application to learn user interests from their historical tweets (henceforth user history) and brief them in a self-introduction.
Despite substantial efforts made in profiling users, most existing work (Li et al., 2014; Farseev et al., 2015; Farnadi et al., 2018; Chen et al., 2019b) focuses on extracting keywords from user history and producing tag-level user attributes (e.g., interests, ages, and personality), which may later charac-
11387
terize personalization and recommendation (Wang et al., 2019a; Liang et al., 2022). However, taglevel attributes profile a user through a fragmented view, while human readers may find it difficult to read. On the contrary, we automate the writing of a sentence-level self-introduction via language generation, providing a more natural and easy-tounderstand way to warm up social interactions. It consequently will enable a better socializing experience and user engagement in social media.
To practically train NLP models with capabilities in self-introduction writing, we collect a large-scale Twitter dataset with 170K public users. Each user presents a self-introduction (manually written by themselves) and previous tweets in their history, corresponding to a total of 10.2M tweets.
For methodology design, we take advantage of cutting-edge practices using pre-trained encoderdecoder for language understanding and generation. However, in real-world practice, users may post numerous tweets exhibiting lengthy content, noisy writings, and diverse interests; these may challenge existing encoder-decoder models in capturing salient personal interests and reflecting them in the brief self-introduction writing.
To illustrate this challenge, Figure 1 shows the self-introduction of a Twitter user U and some sampled tweets from U ’s user history. U exhibits a mixture of interests varying in Delaware, invertebrates, paleontology, museum, and others, scatteredly indicated in multiple noisy tweets. It presents a concrete challenge for models to digest the fragmented information, distill the introduction-worthy points, and condense them into a concise, coherent, and engaging self-introduction for further interactions. Moreover, existing NLP models are ineffective in encoding very long documents (Cao and Wang, 2022), whereas popular users may post numerous tweets, resulting in a lengthy history to encode.
Consequently, we propose a novel unified topicguided encoder-decoder (UTGED) framework for self-introduction generation. First, a neural topic model (Srivastava and Sutton, 2017) clusters words by statistics to learn a mixture of latent topics in characterizing user interests underlying their lengthy history. Then, we inject the latent topics into a BART-based encoder and decoder (Lewis et al., 2020); the encoder employs topic distributions as continuous prompts (Lester et al., 2021; Liu et al., 2021; Li and Liang, 2021) to guide capturing personal interest mixture, and the decoder
adopts topic words to control the writing for personalized self-introduction.
In experimental results, the comparison in both automatic and human evaluation show that UTGED outperforms state-of-the-art encoder-decoder models without topic guidance; and ablation studies indicate the individual contribution from topicguided encoder and decoder. Then, we conduct parameter analyses on topic number and topic prompt length; they are followed by the study on model performance given users varying in historical tweet number, where UTGED consistently performs better. Finally, a case study and an error analysis interpret UTGED’s superiority and limitations.
To the best of our knowledge, we present the first NLP study on self-introduction writing from user tweeting history, where we build the first dataset for its empirical studies and show the benefits from latent topics to the state-of-the-art encoder-decoder paradigm. Below are details of our contributions. • We present a new application to capture personal interests from a user’s tweeting history and generate their self-introductions accordingly. • We approach the application with a novel UTGED (unified topic-guided encoder-decoder) framework, which explores latent topics to represent users’ personal interests and to jointly guide user encoding and self-introduction decoding. • We construct a large-scale Twitter dataset for self-introduction study and extensive experimental results on it show the superiority of UTGED practically and the benefits of latent topics on the task.

heading: 2 Related Work
text: Our work relates to user profiling (by task formulation) and topic modeling (by methodology).
User Profiling. This task aims to characterize user attributes to reflect a personal view. Most previous work focuses on modeling a user’s tweeting history (Li et al., 2014) and social network interactions (Qian et al., 2019; Wang et al., 2019a; Chen et al., 2019b; Wang et al., 2021; Wei et al., 2022) to predict user attribute tags (e.g., ages and interests). However, most existing work focuses on classifying user profiles into fragmented and limited tags. Different from them, we study sentence-level selfintroduction and explore how NLP handles such personalized generation, which initializes the potential to profile a user via self-introduction writing.
Topic Modeling. Topic models are popular unsupervised learning methods to explore corpuslevel word co-occurrence statistics and represent latent topics via clustering topic-related words. Recent work mostly adopts neural architectures based on Variational Autoencoder (VAE) (Kingma and Welling, 2014), enabling easy joint work with other neural modules (Srivastava and Sutton, 2017).
Latent topics have shown beneficial to many NLP writing applications, such as the language generation for dialogue summaries (Zhang et al., 2021), dialogue responses (Zhao et al., 2017, 2018; Chan et al., 2021; Wang et al., 2022), poetries (Chen et al., 2019a; Yi et al., 2020), social media keyphrases (Wang et al., 2019b), quotations (Wang et al., 2020), and stories (Hu et al., 2022). Most existing methods focus on exploiting topics in decoding and injecting latent topic vectors (topic mixture) to assist generation. In contrast to the above work’s scenarios, our application requires digesting much more lengthy and noisy inputs with scattered keypoints; thus, we leverage topics more finely and enable its joint guidance in encoding (by feeding in the topic mixture as topic prompts) and decoding (using topic words to control word-byword generation).
Inspired by the success of pre-trained language models (PLMs), some efforts have been made to incorporate PLMs into VAE to conduct topic modeling (Li et al., 2020; Gupta et al., 2021; Meng et al., 2022). However, PLMs might be suboptimal in modeling user history (formed by numerous noisy tweets), because PLMs tend to be limited in encoding very long documents (Cao and Wang, 2022). Here, we model latent topics by word statistics, allowing better potential to encode long input.

heading: 3 Twitter Self-Introduction Dataset
text: To set up empirical studies for social media selfintroduction, we build a large-scale Twitter dataset.
Data Collection. Following Nguyen et al. (2020), we first downloaded the general Twitter streams
from September 2018 to September 2019. Then, we extracted the user ids therein and removed the duplicated ones. Next, we gathered users’ tweeting history and self-introductions via Twitter API2 and filtered out inactive users with less than 30 published tweets. For users with over 100 published tweets, only the latest 100 ones were kept. At last, we maintained the tweet text in English and removed irrelevant fields, e.g., images and videos.
Data Pre-processing. First, we removed nonEnglish self-introductions and those too short (<7 tokens) or too long (>30 tokens). Second, we employed SimCSE (Gao et al., 2021) (an advanced model for semantic matching) to measure the text similarity between a user’s self-introduction and their tweeting history. Then, for training quality concern, we removed users with self-introductions that exhibit less than 0.4 similarity score 3 on average to the top-30 tweets in history.4 Third, for the remaining 176,199 unique user samples, each corresponds to a pair of user history (source) and self-introduction (target). For model evaluation, we randomly split the user samples into training (80%), validation (10%), and test (10%) sets.
2https://developer.twitter.com 3The details of the tweets with less than 0.4 similarity score are shown in Appendix A.2. 4Users may have less than 30 tweets kept in history (e.g., some tweets without English text were excluded). For these users, we considered all their maintained tweets.
Data Analysis. Encoder-decoder models are widely used in summarization tasks (§2). We then discuss the difference of our task through an empirical lens. The statistics of our dataset and other popular summarization datasets are compared in Table 1. We observe that each of our data sample exhibits a longer source text and a shorter target text compared to other datasets. It indicates the challenge of our self-introduction task, where directly using summarization models may be ineffective.
To further analyze the challenges, Figure 2(a) displays the distribution of SimCSE-measured source-target similarity (averaged over top-30 tweets in user history). It implies that very few tweets are semantically similar to their authors’ self-introductions, making it insufficient to simply “copy” from history tweets. We then analyze and show the tweet number distribution in user history in Figure 2(b). It is noticed that 37% users posted over 90 history tweets, scattering interest points in numerous tweets and hindering models in capturing the essential ones to write a self-introduction.

heading: 4 Our UTGED Framework
text: Here we describe our UTGED (unified topicguided encoder-decoder) framework. Its overview is in Figure 3: latent topics guide the PLMs to encode user history and decode self-introductions.
The data is formulated as source-target pairs {Xi, Y i}Ni=1, where Xi = {xi1, xi2, ..., xim} indicates user history with m tweets published by user ui, Y i represents the user-written description, and
N is the number of pairs. In our task, for user ui, models are fed in their user history tweets Xi and trained to generate their self-introduction Y i.

heading: 4.1 Neural Topic Model
text: To explore users’ interests hidden in their numerous and noisy tweets, we employ a neural topic model (NTM) (Srivastava and Sutton, 2017) to learn latent topics (word clusters). NTM is based on VAE with an encoder and a decoder to reconstruct the input.
For word statistic modeling, the history tweets in Xi are first processed to a one-hot vector Xibow ∈ RVbow in Bag-of-words (BoW), where Vbow indicates NTM’s vocabulary size. Then, similar to VAE, NTM encoder transforms BoW vector Xibow into a K-dimensional latent topic variable zi ∈ RK . Conditioned on zi, NTM decoder produces X̂ibow to reconstruct Xibow. Here presents more details.
NTM Encoder. Given the BoW vector Xibow, NTM encoder attempts to learn the mean µi and standard deviation σi based on the assumption that words in Xi exhibit a Gaussian prior distribution. Its mean and standard deviation, µi and σi will be encoded by the following formula and later be utilized to compute the latent topic vector zi:
µi = fµ(fb(X i bow)); logσ i = fσ(fb(X i bow)) (1)
where f∗(·) indicates a single layer perceptron performing the linear transformation of input vectors.
NTM Decoder. We then reconstruct the BoW in Xi based on the NTM-encoded µi and σi. We hypothesize that a corpus may exist K latent topics, each reflecting a certain user interest and represented by word distribution over the vocabulary Vbow. Besides, user history Xi is represented as a topic mixture θi to reflect ui’s interest combination over K topics. The procedure is as follows: • Draw latent topic vector zi ∼ N (µi, σi) • Topic mixture θi = softmax(fθ(zi)) • For each word w ∈ Xi: Draw w ∼ softmax(fϕ(θi)) where fθ and fϕ are a single layer perceptron. The weight matrix of fϕ indicates topic-word distributions (ϕ1,ϕ2,...,ϕK).
The learned latent topics for Xi will later guide the BART-based self-introduction generation (to be discussed in §4.2). The topic mixture θi will be injected into the BART encoder for capturing salient interests and the top-l words Ai = {ai1, ai2, ..., ail} with highest topic-word probability in ϕc (c indexes the major topic suggested by θi) will go for controlling the writing process of the BART decoder.

heading: 4.2 Topic-Guided Generation Model
text: We have discussed how to model a user ui’s latent interests with NTM and the learned latent topics (θi and topic words Ai) will then guide a BARTbased encoder-decoder model to generate ui’s selfintroduction, Y i. In the following, we first present how we select tweets (for fitting overly long user history into a transformer encoder), followed by our topic-guided design for encoding and decoding.
Tweet Selection. Recall from §3 that user history tends to be very long (Table 1 shows it has 1581.3 tokens on average). However, BART encoder limits its input length. To fit in the input, we go through the following steps to shortlist representative tweets from a user ui’s lengthy tweeting history, Xi.
First, we measure how well a tweet xiu can represent Xi via averaging its similarity to all others:
siu = 1 |m| ∑
xiv∈Xi Sim
( xiu, x i v ) (2)
where Sim(xiu, x i v) represents the x i u − xiv SimCSE-measured cosine similarity. Then, we maintain a shortlist Ri to hold Xi’s representative tweets, which is empty at the beginning and iteratively added with xih obtaining the highest similarity score (Eq. 2). To mitigate redundancy in Ri, once xih is put in Ri, it is removed from X
i, and so are other tweets in Xi whose cosine similarity to xih is over a threshold λ (i.e., 0.8). For easy reading, we summarize the above steps in Algorithm 1.
After that, we further rank the shortlisted tweets in Ri based on their overall similarity in Xi (Eq. 2). The top ones are maintained and concatenated chronologically to form a word sequence Ri = {wi1, wi2, ..., wiM} (M denotes the word number).
Topic Prompt Enhanced Encoder (TPEE). We then discuss how we encode Ri (selected user history tweets) in guidance of the topic mixture θi (featuring latent user interests). The encoding adopts the BART encoder and is trained with θi-based prompt fine-tuning (thereby named as TPEE, short for topic prompt enhanced encoder).
We first obtain the topic prompt as follows:
Bi = MLP(θi) (3)
where MLP is a feedforward neural network. Following Li and Liang (2021), Bi ∈ Rd×L is split into L vectors [bi1, b i 2, ..., b i L]. L indicates the topic prompt length and each vector bij ∈ Rd. To inject the guidance from topic prompts {bi1, bi2, ..., biL} (carrying latent topic features), we
put them side by side with the embeddings of words {wi1, wi2, ..., wiM} (reflecting word semantics of Ri). Then, a BART encoder E represents user ui’s salient interests H iE in its last layer:
HiE = E( [ bi1; b i 2; ...; b i L; e i 1; e i 2; ...; e i M ] ) (4)
where eij ∈ Rd is the BART-encoded word embedding of wij and [; ] is the concatenation operation.
Topic Words Enhanced Decoder (TWED). Recall in §4.1, NTM generates l topic words (Ai) to depict a user ui’s major latent interests. To further reflect such interests in the produced selfintroduction, we employ Ai to control a BART decoder D in its word-by-word generation process through the topic control module.
For easy understanding, we first describe how the original BART decode. At the t-th step, the decoder D is fed in its previous hidden states H iD,t, the BART encoder’s hidden states H iE (Eq. 4), and latest generated word Y it , resulting in hidden step oit+1. Based on that, the next word is generated following the token distribution pit+1. The concrete workflow is shown in the formula as follows:
pit+1 = softmax(Weo i t+1) (5)
oit+1, H i D,t+1 = D(HiE , HiD,t, Y it ) (6)
where H iD,t+1 stores all the previous decoder hidden states till step t+1, We is learnable and to map the latent logit vector oit+1 to the target vocabulary.
Then, we engage topic words Ai to control the above procedure by the topic control module. Inspired by BoW attribute model (Dathathri et al., 2020), we calculate the following log-likelihood loss to weigh the word generation probability pit+1 over each topic word aij ∈ Ai:
log p(Ai|Y it+1) = log ( ∑
j
pit+1[a i j ]) (7)
The gradient from log p(Ai|Y it+1) is further involved in updating all decoder layers (H iD,t) in D:
H̃iD,t = ∆H i D,t +H i D,t (8)
∆HiD,t ← ∆HiD,t+α ∇∆Hi D,t log p(Ai|HiD,t +∆HiD,t)
∥∇∆Hi D,t log p(Ai|HiD,t +∆HiD,t)∥γ (9)
where H̃ iD,t indicates the updated (topic-controlled) decoder’s states, ∆H iD,t means the gradient update to H iD,t, α is the step size, and γ is the normalization value. Furthermore, we adopt the same topic-controlling strategy to update the encoder’s final layer states H iE and derive the updated states H̃ iE based on Eq. 8 and 9. With Eq. 5 and 6, we can
accordingly obtain the final token distribution p̃it+1 based on the topic-controlled encoder and decoder states H̃ iE , H̃ i D,t, and previous predicted word Y i t .

heading: 4.3 Joint Training in a Unified Framework
text: To couple the effects of NTM (described in §4.1) and topic-guided encoder-decoder module for selfintroduction generation (henceforth SIG discussed in §4.2), we explore the two modules in a unified framework and jointly train them for better collaborations. The loss function of the unified framework is hence a weighted sum of NTM and SIG:
L = αLNTM + (1− α)LSIG (10)
where LNTM and LSIG are the loss functions of NTM and SIG. α is the hyper-parameter trading off their effects and is set to 0.01 in our experiments.
For NTM, the learning objective is computed as:
LNTM = DKL(δ(z)||ρ(z|X))− Eρ(z|X)[δ(X|z)] (11)
where DKL(·) indicates the Kullback-Leibler divergence loss and E[·] is reconstruction loss.5 For the SIG, it is trained with the cross-entropy loss:
LSIG = − ∑
i
∑
t
log pit (12)
In practice, we first train the unified framework with Eq.10 and exclude Ai (topic words output of NTM). Then, during inference, we fix UTGED, employ Ai to control the decoding process and generate the final self-introduction with Eq.7~Eq.9.

heading: 5 Experiments and Discussions

heading: 5.1 Experimental Setup
text: Model Settings. We implemented NTM (§4.1) based on (Srivastava and Sutton, 2017) and set its topic number K to 100. Its BoW vocabulary size Vbow is set to 10K and hidden size to 200. The input of NTM is the BoW of original user history Xi while the input of SIG is capped at 1,024 tokens based on the shortlisted tweets in Ri (§4.2).6
The SIG model is based on the BART and built on 6 encoding layers and 6 decoding layers. We adopted AdamW and SGD to optimize the SIG and NTM, respectively. The learning rate is set to 5× 10−5 for SIG and 1× 10−4 for NTM. The topic prompt length L is set to 7. To warm up joint training (Eq.10), we pre-train NTM with Eq.11 for
5We refer readers to more details of NTM in Srivastava and Sutton (2017), which are beyond the scope of this paper.
6We also test NTM with BoW on {Ri}Ni=1 and observe slightly worse results. It is possibly because NTM is based on word statistics and would barely be affected by lengthy input.
100 epochs. During joint training, batch size is set to 8 and the maximum epoch to 5. In topiccontrolled decoding, α is set to 0.25 and γ to 1.5 (Eq.9). Topic word number l is set to 30. Models are trained on a 24GB NVIDIA RTX3090 GPU.
Evaluation Metrics. For automatic evaluation, we adopt ROUGE-1 (R-1), ROUGE-2 (R-2), and ROUGE-L (R-L), which are popular metrics in language generation based on output-reference word overlap and originally for summarization tasks (Lin, 2004). We also conduct a human evaluation on a 5 point Likert scale and over three criteria: fluency of the generated language, consistency of a self-introduction to the user’s history, and informativeness of it to reflect essential user interests.
Baselines and Comparisons. We adopt extractive and abstractive summarization models in comparison. The former extracts user history tweets as the self-introduction by ranking them with: (1) BERTExt (Liu and Lapata, 2019) (based on BERT (Devlin et al., 2019)) (2) TextRank (Mihalcea and Tarau, 2004) (unsupervised graph ranking based on similarity) (3) Consen ( unsupervised ranking with the averaged similarity to others (Eq.2)).
For abstractive competitors, models all follow the encoder-decoder paradigm. We employ T5 (Raffel et al., 2020), BART (Lewis et al., 2020), and PEGASUSU-X (Phang et al., 2022), all based on PLMs and are state-of-the-art abstractive summarizers. We also compare to GSum (Dou et al., 2021), which employs highlighted sentences, keywords, relations, and retrieved summaries.
In addition, we examine the upper-bound tweet selection (shortlist given reference selfintroduction). Here SimCSE first measures the similarity between the reference and each tweet in user history Xi. OracleE then extracts the tweet with the highest similarity score. For OracleA, we rank tweets based on the similarity score and the top ones are fed into BART for a generation. Furthermore, to explore the potential of our topic-guided design over OracleA model, we feed OracleA’s input to our UTGED and name it OracleA+Topic.

heading: 5.2 Main Comparison Results
text: Table 2 shows the main comparison results. We first observe the inferior results from all extractive models, including OracleE . It is because of the non-trivial content gap between users’ history tweets and their self-introductions (also indicated in Figure 2). Directly extracting tweets from user
history is thus infeasible to depict self-introduction, presenting the need to involve language generation. For this reason, abstractive methods exhibit much better performance than extractive baselines.
Among comparisons in abstractive models, UTGED yields the best ROUGE scores and significantly outperforms the previous state-of-the-art summarization models. It shows the effectiveness in engaging guidance of latent topics from lengthy and noisy user history, which may usefully signal the salient interests for writing a self-introduction.
In addition, by comparing OracleA results and model results, we observe a large margin in between. It suggests the challenge and importance of tweet selection for user history encoding, providing insight into future related work. Moreover, interestingly, OracleA+Topic further outperforms OracleA, implying topic-guided design would likewise benefit the upper-bound tweet selection scenarios.
Ablation Study. Here we probe into how UTGED’s different modules work and show the ablation study results in Table 3. All modules (tweet selection (S), TPEE (E), and TWED (D)) contrite positively because they are all designed to guide models in focusing on essential content reflecting
user interests against lengthy input. TPEE may show larger individual gain than the other two, possibly because the topic mixtures directly reflect user interests and are easier for the model to leverage.
Human Evaluation. To further test how useful our output is to human readers, we randomly select 100 samples from test set and train 3 in-house annotators from NLP background to rate the generated self-introductions. As shown in Table 4, UTGED is superior in informativeness and consistency. It implies latent topics can usefully help capture salient interests from lengthy and noisy user history. However, its fluency is lower than that of BART, indicating that topic words slightly perturb the pre-trained decoder (Dathathri et al., 2020).

heading: 5.3 Quantitative Analysis
text: To better study UTGED, we then quantify the topic number, prompt length, and input tweet number to examine how they affect performance. Here only R-L is shown for better display, and similar trends were observed from R-1 and R-2. For the full results, we refer readers to Appendix A.3.
Varying Topic Number. The first parameter analysis concerns the topic number K (NTM’s hyperparameter). As shown in Figure 4(a), the score first increases then decreases with larger K and peaks the results at K = 100. We also observe K = 200 results in much worse performance than other Ks, probably because modeling too fine-grained topics is likely to overfit NTM in user interest modeling, further hindering self-introduction generation.
Varying Prompt length. Likewise, we analyze the effects of prompt length L in Figure 4(b). The best score is observed given L=7, much better than very short or very long prompt length. Longer prompts may allow stronger hints from NTM, helpful to some extent; however, if the hint becomes too strong (given too-long prompt), topic features may overwhelm the encoder in learning specific features for self-introduction writing.
Users w/ Varying Tweet Number. Recall in Figure 2(b), users largely vary tweet number in history (attributed to different active degrees). We then examine how models work given varying tweet numbers in history. BART+S and UTGED are tested, both with tweet selection (S) to allow very long input and Figure 5 shows the results. Both models exhibit growing trends for more active users, benefiting from richer content in their history to infer self-introduction. Comparing the two models, UTGED performs consistently better, showing the gain from NTM’s is robust over varying users.

heading: 5.4 Qualitative Analysis
text: Case Study. Figure 6 shows a user sample interested in “teaching” and “reading”. It can be indicated by topic words like “student”, “book”, and “school” produced by NTM. From BART’s output, we find its errors in “seesaw specialist” further mislead the model in writing more irrelevant content (e.g., “google certified educator” and “google trainer”). It may be caused by the common exposure bias problem in language generation (Ranzato et al., 2016; Zhang et al., 2019). On the contrary, UTGED’s output is on-topic till the end, showing topic guidance may mitigate off-topic writing. 7
Error Analysis. In the main comparison (Table 2), UTGED performs the best in yet also has a
7More topic word cases could be found in Appendix A.4 and longer source tweets are shown in Appendix A.5.
non-trivial gap to OrcationA. Here we probe its limitations and discuss the two major error types in Figure 7. First, the output may contain grammatical mistakes, e.g., “deals, deals", limited by BART’S decoder capability and topic words’ effects. It calls for involving grammar checking in decoding. The second error type is propagated from wrong latent topics. As shown in the error case (second row), the user is a provider of “pet”-style clothes, whereas NTM may cluster it with other “pet lover”-users and further mislead the writing process. Future work may explore a better topic modeling method to mitigate the effects of mistakenly clustering.

heading: 6 Conclusion
text: We have presented a new application to generate personalized self-introduction, where a large-scale Twitter dataset is gathered for experiments. A novel unified topic-guided encoder-decoder framework is proposed to leverage latent topics for distilling essential user interests from the numerous noisy tweets a user has posted. Empirical results show our model outperforms advanced PLM-based models, shedding light on the potential of latent topics in helping PLMs digest lengthy and noisy input.

heading: Acknowledgements
text: This paper is substantially supported by the NSFC Young Scientists Fund (No.62006203, 62106105), a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China (Project No. PolyU/25200821), and the Innovation and Technology Fund (Project No. PRP/047/22FX).
Limitations
First, inference efficiency is one of the main limitations of this work. The BART model takes about 14 minutes to complete the inference on our dataset, while our UTGED needs 92 minutes. The reason for the slow inference is that UTGED requires heavy computation to update the gradient to the encoder’s states and decoder’s states (as shown in Eq.7~Eq.9). Future work may consider how to advance model efficiency further.
Second, the lack of multimodal content in the published tweets would result in another limitation. The images contained in the published tweets are ignored in this work. However, due to the complicated relationships between images and texts in a multimodal tweet, images might provide complementary content and complete the meanings of the message (Vempala and Preotiuc-Pietro, 2019). Therefore, future studies might explore selfintroduction generation using multimodal tweets (images and text) to indicate personal interests.
Ethics Statement
Our paper constructs a large-scale Twitter dataset for a self-introduction generation. The data acquisition procedure follows the standard data collection process regularized by Twitter API. Only the public users and tweets are gathered. The downloaded data is only used for academic research. For our experiments, the data has been anonymized for user privacy protection, e.g., authors’ names are removed, @mention and URL links are changed to common tags. Following Twitter’s policy for content redistribution, we will only release the anonymized data. Additionally, we will require data requestors to sign a declaration form before obtaining the data, ensuring that the dataset will only be reused for the purpose of research, complying with Twitter’s data policy, and not for gathering anything that probably raises ethical issues, such as sensitive personal information. For the human an-
notations, we recruited the annotators as part-time research assistants with 15 USD/hour payment.

heading: A Appendix
text: A.1 Tweet Selection Algorithm
Algorithm 1 Selecting representative tweets
Require: collected tweets pool for user ui: Xi = {xi1, xi2, ..., xim} Ensure: representative tweets shortlist Ri 1: initial Ri = {} 2: repeat 3: calculate overall similarity score between
a tweet and other tweets in Xi; 4: assume the tweet with the highest score is
xih, remove x i h from X i to Ri; 5: calculate the similarity score between xih and remained tweets in Xi; 6: for the tweets whose similarity score is higher than λ, remove it from Xi; 7: until there are no tweets in Xi
A.2 Data filtering
Here we show the original dataset’s distribution before filtering: [-1, 0): 9,680; [0, 0.1): 123,560; [0.1, 0.2): 257,759; [0.2, 0.3): 193,847; [0.3, 0.4): 157,478; [0.4, 0.5): 118,881; [0.5, 0.6): 44,455; [0.6, 0.7): 10,977; [0.7, 0.8): 1,691; [0.8, 0.9): 169; [0.9, 1.0]: 26. We observe that the number of user samples first increases and then decreases, indicating that self-introductions are related to the user’s historical tweets. Otherwise, the data distribution will tend to exhibit a long tail (based on social media characteristics).
Additionally, we tested a sample of 10,000 users with similarity scores fall in the ranges of [0.3,0.4), [0.2,0.3), [0.1, 0.2), [0,0.1) and results of the best model OracleA+Topic on low-similarity data samples are shown in Table 5. The results indicate that low-similarity data samples do impact negatively on the training results.
A.3 Full Experimental Results Varying Topic Number. We show the results from BART+S+E on the left of “/” and those from UTGED on the right.

heading: 50 24.58/24.85 7.85/7.99 21.49/21.76

heading: 100 24.78/24.99 7.95/8.05 21.65/21.84

heading: 150 24.77/24.98 7.93/8.01 21.60/21.78

heading: 200 24.67/24.71 7.86/7.90 21.52/21.59
text: Varying Prompt length. We show the results from BART+S+E on the left of “/” and those from UTGED on the right.

heading: 3 24.36/24.53 7.68/7.78 21.25/21.42

heading: 7 24.78/24.99 7.95/8.05 21.65/21.84

heading: 11 24.48/24.69 7.90/8.01 21.43/21.66

heading: 15 24.53/24.74 7.84/7.89 21.41/21.58

heading: 19 24.34/24.48 7.76/7.81 21.30/21.42
text: Varying Sentence Number. We show the results from BART+S on the left of “/” and those from UTGED on the right.

heading: 100 24.26/24.99 7.68/8.05 21.17/21.84

heading: 80 24.23/24.95 7.70/8.06 21.15/21.82

heading: 60 24.12/24.90 7.63/8.02 21.09/21.78

heading: 40 23.82/24.57 7.45/7.83 20.81/21.50

heading: 20 22.79/23.41 6.90/7.15 19.92/20.49
text: A.4 Topic Words
A.5 Detailed Case Study
ACL 2023 Responsible NLP Checklist

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: Limitation section: line 606

heading: 3 A2. Did you discuss any potential risks of your work?
text: Ethics statement section: line 625

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: Abstraction section: line 001
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Section 4

heading: 3 B1. Did you cite the creators of artifacts you used?
text: Section 4: line 257; line 344; line 379
7 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? We will provide the license after publication.
7 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? We will provide it after publication.

heading: 3 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any
text: information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Ethics statement section: line 625
7 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? We will provide it after publication.
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Section 3: line 217
C 3 Did you run computational experiments? Section 5: line 443

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? Limitation section: line 606
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? Section 5: line 425

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Section 5

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Section 5
D 3 Did you use human annotators (e.g., crowdworkers) or research with human participants? Section 5
7 D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? We will provide it after publication.
7 D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? We will provide it after publication.

heading: 3 D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to
text: crowdworkers explain how the data would be used? Ethics Statement Section

heading: 3 D4. Was the data collection protocol approved (or determined exempt) by an ethics review board?
text: Ethics Statement Section
7 D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? We will provide it after publication.

heading: Future Work
text: Future work may consider how to advance model efficiency further. Second, the lack of multimodal content in the published tweets would result in another limitation. The images contained in the published tweets are ignored in this work. However, due to the complicated relationships between images and texts in a multimodal tweet, images might provide complementary content and complete the meanings of the message (Vempala and Preotiuc-Pietro, 2019). Therefore, future studies might explore selfintroduction generation using multimodal tweets (images and text) to indicate personal interests