Towards Open-Domain Twitter User Profile Inference
Year: 2023

Abstract:
Twitter user profile inference utilizes information from Twitter to predict user attributes (e.g., occupation, location), which is controversial because of its usefulness for downstream applications and its potential to reveal users’ privacy. Therefore, it is important for researchers to determine the extent of profiling in a safe environment to facilitate proper use and make the public aware of the potential risks. Contrary to existing approaches on limited attributes, we explore open-domain Twitter user profile inference. We conduct a case study where we collect publicly available WikiData public figure profiles and use diverse WikiData predicates for profile inference. After removing sensitive attributes, our data contains over 150K public figure profiles from WikiData, over 50 different attribute predicates, and over 700K attribute values. We further propose a promptbased generation method, which can infer values that are implicitly mentioned in the Twitter information. Experimental results show that the generation-based approach can infer more comprehensive user profiles than baseline extraction-based methods, but limitations still remain to be applied for real-world use. We also enclose a detailed ethical statement for our data, potential benefits and risks from this work, and our efforts to mitigate the risks. 1

heading: 1 Introduction
text: Users’ profile information provides invaluable user features. Accurate automatic user profile inference is helpful for downstream applications such as personalized search (Shen et al., 2005; Teevan et al., 2009; Zhu et al., 2008; Yao et al., 2020) and recommendations (Lu et al., 2015; Balog et al., 2019;
∗Equal contribution. 1To prevent potential misuse of our resources, we will only release our resources for research purposes based on individual requests. Please see our ethics statement section for details and contact us directly for resource access.
Guy, 2015), and computational social media analysis (Arunachalam and Sarkar, 2013; Bamman et al., 2014; Tang et al., 2015; Amplayo, 2019). However, there are increasing privacy concerns that conducting profiling without appropriate regulations may reveal people’s private information. Therefore, it is essential to investigate the extent of profiling to promote proper use and make the potential risks clear to public and policy makers.
Previous work on user profile inference has focused on a very limited set of attributes, and models for different attributes employ different strategies. One line of research has formulated it as a classification problem for attributes such as gender (Rao et al., 2011; Liu et al., 2012; Liu and Ruths, 2013; Sakaki et al., 2014), age (Rosenthal and McKeown, 2011; Sap et al., 2014; Chen et al., 2015; Fang et al., 2015; Kim et al., 2017), and political polarity (Rao et al., 2010; Al Zamal et al., 2012; Demszky et al., 2019). In such classification settings, each attribute has it own ontology or label set, which is difficult to generalize to other attributes, especially for attributes that have many possible candidate values (e.g. geo-location, occupation). In addition, some work involves human annotation, which is expensive to be acquired and may raise fairness questions for labeled individuals (Larson, 2017).
Another line of research uses an extraction-based method, such as graph-based (Qian et al., 2017) and unsupervised inference (Huang et al., 2016) for geolocation, distant supervision-based extraction (Li et al., 2014; Qian et al., 2019). However, they still only cover limited attributes that cannot produce comprehensive profiles. Besides, many attribute values are only implicitly mentioned in Twitter context, which cannot be directly extracted.
In this paper, instead of limited attributes, we explore whether open-domain profiles can be effectively inferred. Taking WikiData (Vrandečić and Krötzsch, 2014) as the source of profile information, which provides a much more diverse
3172
predicate set, we find WikiData profiles that have Twitter accounts. We further collect Twitter information for each account, including their recent tweets and Twitter metadata, and build models to infer profiles from collected Twitter information, which is solely based on publicly available information and does not involve any additional human annotation efforts.
We first follow Li et al. (2014) to use profile information to generate distant supervised instances and build a sequence labeling-based profile extraction model, similar to Qian et al. (2019). In order to allow open-domain inference, we propose to use attribute names as prompts (Lester et al., 2021) for input sequences to capture the semantics for attribute predicates instead of involving attribute names into the tag set. However, the extraction approach requires that answers must appear in the Twitter context, which ignores some implicit text clues. Therefore, we further propose a promptbased generation method (Raffel et al., 2020) to infer user profiles, which can additionally produce values that are not straightforwardly mentioned in the Twitter information.
Our statistics show that only a limited number of WikiData attribute values can be directly extracted from Twitter information. Our experiments demonstrate a significant improvement when using the generation-based approach compared to the extraction-based approach, indicating that performing inference instead of pure extraction will be able to obtain more information from tweets. Further analysis shows that the improvement comes mainly from the power of combining extraction and inference on information not explicitly men-
tioned. However, we still find several challenges and limitations for the model to be applied for realworld use, including performances of low-resource attributes, distributional variances between celebrities and normal people, and spurious generation.
Our contributions are summarized as follows:
• To the best of our knowledge, this is the first work to explore open-domain Twitter user profiles. • We create a new dataset for user profile inference from WikiData, providing with rich and accurate off-the-shelf profile information that can facilitate future social analysis research. • We propose a prompt-based generation-based method for user profile inference that provides a unified view to infer different attributes.

heading: 2 Problem Definition and Dataset
text: In this section, we first define the open-domain user profile inference and then describe the dataset collection in detail.

heading: 2.1 Problem Formulation
text: The ultimate goal of user profile inference is to infer certain attribute value given the Twitter information of a user. In Twitter, as shown in Figure 1b, we mainly use the collection of recent Twitter tweets from a user u to represent Twitter information, which we denote as
X tweet, u = [ xtweet,u,1, . . . ,xtweet,u,ntweet,u ] ,
where each xtweet,u,i represents a sequence from a single tweet. In addition, we also concatenate the user’s publicly available Twitter metadata (username, display name, bio and location) into a single sequence as complementary user information
xuser,u. The final input from Twitter is the combination of user metadata and recent tweets
Xu = [X tweet,u; [xuser,u]] .
We then assume that user profiles follow the keyvalue representation
Ru = { (pu,1, vu,1), . . . , (pu,nr,u , vu,nr,u) } ,
where each pair (pu,i, vu,i) represents the predicate and value of an attribute. Figure 1a shows an example key-value profile obtained from WikiData.
The model for open-domain user profile inference is to infer the value v of an attribute p from an user u given their Twitter information and a specific attribute predicate with parameter θ
f(Xu, p; θ) = v.

heading: 2.2 Dataset Creation
text: Our dataset consists of WikiData public figure profiles and corresponding Twitter information. An example of paired WikiData profile and Twitter information is shown in Figure 1. We first discuss the collection of WikiData profiles and then discuss the collection of Twitter information.
WikiData processing. WikiData is a structural knowledge base, which can be easily queried with database such as MongoDB2 using its dump3. It contains rich encyclopedia information, including information for public figures. Each WikiData entity consists of multiple properties and correspond-
2https://www.mongodb.com/ 3https://dumps.wikimedia.org/wikidata
wiki/entities/
ing claims, which can be considered as the predicate value pairs as shown in Figure 1a4.
First, we use WikiData to filter entities that are persons with Twitter accounts. This can be done by checking whether each entity contains the propertyclaim pair “instance of” (P31) “human” (Q5) and then checking whether the entity includes the property “Twitter username” (P2002). Then we extract the account of those filtered persons using the claim (value) of property “Twitter username” (P2002). If there are multiple claims, we use the first only.
Next, for each entity we check all its properties to build the person’s profile. In Figure 1a, as an example, we can see that the property “occupation” is “politician”. For each property and claim, we only consider their text information, and we use English information only. If there are multiple claims for a property, we use the first one. We drop all properties that do not have an English name for either predicate or value, or properties that do not contain any claims.
Since WikiData profiles usually contain many noisy properties that are not suitable (e.g., blood type) for Twitter user profile inference, we clean the data by 1) filtering extremely low-frequency properties; 2) manually selecting some meaningful and discriminative properties and 3) removing sensitive personal information listed in the Twitter Developer Agreement and Policy, such as political affiliation, ethnic group, religion, and sex or gender5. Please refer to Appendix B for the complete list of properties that we use.
Twitter processing. We collect publicly available Twitter information for users that we gather from WikiData, as shown in Figure 1b. The Twitter information consists of the user’s at most 100 recent publicly available tweets, as well as their metadata that includes username, display name, bio (a short description that a user can edit in their profile) and location. We remove all web links and hashtags from those tweets.
Statistics. We collect more than 168k public figures from Wikidata and filter out users whose Twitter accounts are no longer accessible. We obtain about 152K users with 13 million tweets in total. We randomly split the users into train, development
4Please refer to https://www.mediawiki.org/ wiki/Wikibase/DataModel for further details of Wikibase DataModel.
5https://developer.twitter.com/en/dev eloper-terms/agreement-and-policy
and test sets by 7:1:2. The detailed statistics are shown in Table 1. We compare it with previous work such as Li et al. (2014) and Fang et al. (2015), demonstrated in Table 2. We find that our dataset contains much more diverse predicates compared to Li et al. (2014) and Fang et al. (2015). We also have a much larger number of users and attribute values compared to the previous work. Although Li et al. (2014) contains more tweets than ours, they only consider the extraction setting, and most of the tweets in their datasets are negative samples.
Long tail distribution of predicates. As shown in Figure 2, the number of examples per predicate follows a long tail distribution. Only a few predicates have many training examples, while most appear only partially in the user’s entity list. This raises a huge challenge for us to develop a good model to utilize and transfer the knowledge from rich-resource predicates to low-resource predicates. We discuss the details in the following section.

heading: 3 Methods
text: In this section, we discuss our methods for opendomain Twitter user profile inference. First, we introduce an extraction-based method that largely follows the principle from Li et al. (2014) and Qian et al. (2019). Then we discuss our proposed prompt-based generation approach that provides a unified view to infer different attribute values, and can further infer values that do not appear in the Twitter context.

heading: 3.1 Extraction-based Method
text: We follow Li et al. (2014) and Qian et al. (2019) to generate distantly supervised training instances for user profile extraction. Since our problem is open domain, we propose using attribute predicates as prompts in input sequences and perform sequence labeling over them. This method can be divided into three steps: label generation, modeling, and result aggregation.
Label generation. Distant supervised labeling assumes that if a user u’s profile contains attribute value v, we can find mentions in their Twitter information expressing the value.
Specifically, we consider each sequence xi in Xu independently. For each attribute predicatevalue pair (pj , vj) in u’s profile, we construct a tag sequence ti,pj for xi and the predicate pj . For a span [xb, . . . , xe] that matches vj , we make
ti,pj ,b = B,
ti,pj ,b+1 = . . . = ti,pj ,e = I.
If a position k does not match the value, then ti,pj ,k = O. For simplicity, we use exact string matching between vj and spans in the sequence. An example tag sequence is shown in Figure 3.
Modeling. Sequence labeling tasks usually include the label name in the tag set (e.g. B-PER for the beginning of a mention representing a person; Lample et al.,2016). In the open-domain profile inference setting, we have numerous attributes and
many of them have only a few instances as shown in Figure 2, which are not sufficient to be considered as separate tag labels.
Therefore, we propose to use prompt-guided sequence labeling, where we append the attribute predicate p to the front of the sequence as the prompt as follows:
[CLS] p [SEP] xi
Then we perform sequence labeling on the second part of the input xi using the generated labels. We use RoBERTa (Liu et al., 2019) as the backbone encoder, and we denote the last hidden states of xi by H = [h1, . . . ,hn] where n represents the length of xi. The probability of predicted labels is
P (ti,p,k | xi, p) = softmax (W hhk + bh) ∈ R3,
where k represent the position in xi. During training, we randomly drop negative instances that do not contain any B labels to keep the positive-negative sample ratio steady.
Result aggregation. During inference, for each user, we first perform sequence labeling on every sequence predicate pair exhaustively. Then we aggregate sequence-level labeling results into userlevel results. For each attribute predicate, we select the span that has the largest averaged logit as the final answer.

heading: 3.2 Generation-based Method
text: Extraction-based methods suffer from the fact that attribute values must appear in the Twitter context.
Instead with user profile inference, it is very likely that we cannot directly find those values in the context and therefore need to infer them using implicit evidence. To address this issue, we propose to use the conditional generation method, which has been shown to be effective in both extracting input information (Raffel et al., 2020; Li et al., 2021) and performing inference and summarization (See et al., 2017; Alshomary et al., 2020). The overall framework is illustrated in Figure 4.
Modeling. We use T5 (Raffel et al., 2020), a generative transformer based model, to directly generate the answer given the predicate. Similar to the extraction-based method, to address the longtail distribution problem we use the attribute predicate as prompt at the beginning of the input sequence, which can capture rich semantics of those open-domain attribute predicates, especially when the attribute predicate lacks examples in the data. Specifically, the input is the concatenation of prefix predicate (e.g. predicate:occupation), user’s Twitter metadata, and the sequence of tweets that the user has recently published. We train the model to generate the attribute value (y1, . . . , yn) by minimizing the cross-entropy loss:
LCE = − 1
n
n∑
i=1
log p(yi|y<i,x),
where x is the input to the model and n represents the length of the output sequence.
Since we have at most 100 recent tweets of each user whose total length normally exceeds the limit of the model, we use sliding window and divide recent tweets organized in chronological order into different windows where each window can represent information within a time range. Then we train the model on these divided examples separately. Each example contains the same prefix predicate and Twitter metadata but uses different parts of the tweets to infer the attribute value.
Result aggregation. During inference, we use the same sliding window strategy and divide the input into different examples to make predictions independently. Then, similar to the extraction-based method, we aggregate those window-level predictions into a user-level prediction. We count the occurrences of each predicted text for a predicate and then use majority vote to find the aggregated result of that predicate.
Result filtering. The generation-based method aggressively generates output without estimating whether the generated output is spurious. Therefore, it is important to filter those incorrect predictions during inference.
After result aggregation, we first take the product of probability for each generated token as the score for each aggregated prediction, and then use the averaged score over all aggregated predictions as the confidence score for the aggregated result. A low confidence score indicates that the model cannot determine whether the prediction is valid.
For each predicate, we search the best threshold and set predictions with confidence scores lower than threshold as “no prediction”. We consider all predicted confidence scores from the development set as candidate thresholds and choose the threshold that yields the best performance on the development set. The best searched threshold is then directly applied to filter results on the test set.

heading: 4 Experiments
text: In this section, we conduct experiments on our constructed dataset and user profile extraction dataset (Li et al., 2014). Then we provide a qualitative analysis and discuss the remaining challenges.

heading: 4.1 Experimental Setup
text: We use roberta-base6 as the base model for the extraction-based model, as it demonstrates its effectiveness on multiple sequence labeling tasks. We use t5-small7 for the generation-based model, which has much fewer parameters than roberta-base. Please refer to Appendix A for a detailed hyperparameter setup and estimated training and inference time.
Evaluation metric. We choose user-level F1 as our evaluation metric. Specifically, we suppose
6https://huggingface.co/roberta-base 7https://huggingface.co/t5-small
a user profile consists n different attributes. We use C(·) to represent the count of different types of output. C(no prediction) refers to the count of “no predictions” and C(correct prediction) refers to the count of predictions that match the WikiData profile. Then we obtain the user-level F1 as follow:
precision = C(correct prediction) n− C(no prediction)
recall = C(correct prediction)
n
F1 = 2 precision · recall precision + recall
We consider the prediction go be valid when it identically matches the ground truth. We do not use entity-level or tag-level F1 as Qian et al. (2019) because it is not applicable to the generation model. We do not use generation-based metrics (e.g., BLEU) because we observe that most predictions are very short. In addition, compared to no prediction, we want to penalize wrong predictions more. In F1, the basis of precision does not include “no prediction” results from models while it still has a penalty for wrong predictions.

heading: 4.2 Results

heading: 4.2.1 Results on User Profile Inference
text: The main results are shown in Table 3. The random result means that predictions are uniformly
randomly selected and the majority means that predictions are selected with the values that occur most frequently in the training set. We find that both simple methods perform poorly. Overall, we find that our generation-based method significantly outperforms other methods by a large margin. We also find that the extraction-based method cannot even outperform the majority baseline. The reason is that the majority vote can achieve relatively high accuracy on attributes that have a relatively small number of candidates, or one specific candidate takes a large portion of the data, while we cannot find corresponding occurrences of some of those attributes in the Twitter context.
To verify the above claim, we perform another test on a subset of the test set data, for which we can find corresponding occurrences of attribute values in the Twitter context. We find that only 13.56% of the test data can find those value occurrences, which indicates that the majority of the data cannot be directly extracted from Twitter context. The results are shown in Table 4. By comparing the results with overall results, we can find that both extraction and generation systems can get better performance on the subset that we can find occurrences of attribute values. We find that the extraction method performs quite closely to the generation-based method in this setting, though the generation-based method performs better on precision and F1 and the extraction-based method better on recall. This result indicates that when attribute values occur in Twitter context, the extraction model can effectively extract them, while the generation-based method can additionally infer values that are not included in the Twitter content.

heading: 4.2.2 Results on User Profile Extraction
text: We conduct additional experiments on the profile extraction dataset from Li et al. (2014), where we can provide a direct comparison between our generation-based model and previous work. We follow the same preprocessing as Qian et al. (2019)
on EDUCATION and JOB. We make two changes to our generation-based model for this dataset. 1) This dataset does not contain a timestamp for each tweet, so we use each tweet as an independent sample instead of the sliding window strategy. 2) This dataset is designed for extraction, so for tweets from which the answers cannot be extracted we train the generation model to output “no prediction”.
The experiment results are shown in Table 5. We compare with GraphIE (Qian et al., 2019), one of the state-of-the-art model on this dataset. We reproduce the results from their script8 and re-evaluate on user-level with majority vote. We use the averaged results over 5-fold cross validation as Qian et al. (2019). The results show that our model can significantly outperform GraphIE on both EDUCATION and JOB attributes, which indicates that even if the attributes are limited, the generation-based method can still achieve promising performance.

heading: 4.3 Ablation study
text: We conduct an ablation study on two of our components, result filtering and result aggregation, on our profile inference data, as shown in Table 6. We find that result filtering can successfully filter spurious results by improving over 13% on precision, while only dropping about 2% on recall. We also find that result aggregation improves both precision and recall, indicating that we can obtain better
8https://github.com/thomas0809/GraphIE
inference by using a larger Twitter context. Twitter metadata also provides rich information about the user’s background. We train and evaluate another model without Twitter metadata, and find that we see a significant performance drop. But we still find that many attributes inferred by the model are not dependent on those metadata.

heading: 4.4 Qualitative Analysis
text: Figure 5 demonstrates four window-level predictions from generation-based model with relevant input context. The first case shows that the model can directly copy relevant information from context. The second and third cases show that the model can infer the information based on the context. The last case shows an error that the model does not fully utilize the information provided by “wrestle” and generates incorrect information, possibly affected by the other word “show”. This case indicates the importantce of background information for a specific attribute value.

heading: 4.5 Remaining Challenges
text: Although achieving improvement on open-domain attribute inference, we still find that the model’s performance on attributes with low training samples is generally much lower than on attributes with rich samples. It is still under investigation for better generalization on these low-resource attributes.
WikiData provides rich profiles for many Twitter users. However, the distribution of these Twitter
users with WikiData profiles may not align with the need for downstream tasks. For example, most people with WikiData profiles are celebrities, such as politicians and athletes, which lacks information for general occupations such as farm worker.
The granularity of prediction results is also another important directions to investigate. We observe some cases that the prediction and the grountruth are in different levels of granularity. For example, the groundtruth can be "Tokyo” while the prediction may be "Japan”. Therefore, it is also important to address this issue with both better modeling as well as evaluation.
We consider that the model can predict all collected attribute values because we have manually selected meaningful and discriminative properties from WikiData during dataset construction. However, it is still possible that a specific property value cannot be detected well based on Twitter content, leading to spurious generation output. For example, if a user is a medical doctor but did not discuss any medical information on Twitter, the occupation is very hard to predict. It is still important to further investigate this “cannot predict” cases in both dataset construction and model design.

heading: 5 Related Work
text: User Profile Inference. One line of user modeling research focuses on profile inference or extraction. Previous work on user profile inference focuses on some specific attributes such as gender (Rao et al., 2011; Liu et al., 2012; Liu and Ruths, 2013; Sakaki et al., 2014), age (Rosenthal and McKeown, 2011; Sap et al., 2014; Chen et al., 2015; Fang et al., 2015; Kim et al., 2017), and political polarity (Rao et al., 2010; Al Zamal et al., 2012; Demszky et al., 2019). They often consider them as multi-class classification problems. Most of these methods use the context of those social media posts. Alternatively, user name and profile in social media (Liu et al., 2012; Liu and Ruths, 2013), part-of-speech and dependency features (Rosenthal and McKeown, 2011), users’ social circles (Chen et al., 2015) and photos (Fang et al., 2015) have been explored as additional important features for different attribute inference. But those classification settings have a pre-defined ontology or label set, which is difficult to extend to other attributes.
In addition to classification-based methods, there are also graph-based (Qian et al., 2017), distant supervision-based and unsupervised extrac-
tion (Huang et al., 2016). Compared to the classification method, extraction-based methods are capable of identifying attributes with a large ontology. But they rely on entities from the context as candidates, which limits the scope of the attributes that occur frequently in the social media context.
Our open-domain Twitter user profile inference uses a larger predicate set and data than previous work. We further propose the generation-based approach, which addresses the limited scope.
Another line of user modeling research focuses on leveraging behavior signals (Kobsa, 2001; Abel et al., 2013) or building implicit user representations (Islam and Goldwasser, 2021, 2022), which is more distantly related to our problem.
Sociolinguistic variation. The intuition of inferring user attributes from their posts aligns with sociolinguistic variation in which people investigate whether a linguistic variation can be attributed to different social variables (Labov, 1963). Computational efforts to discover these relationships include demographic dialectal variation (Blodgett et al., 2016), geographical variation (Eisenstein et al., 2010; Nguyen and Eisenstein, 2017), syntactic or stylistic variation over age and gender (Johannsen et al., 2015), socio-economic status (Flekova et al., 2016; Basile et al., 2019).

heading: 6 Conclusion
text: In this paper, we first explore open-domain Twitter user profile inference. We use the combination of WikiData and Twitter information to create a largescale dataset. We propose to use a generation-based method with attributes as prompts and compare it with the extraction-based method. The result shows that the generation-based method can significantly outperform the extraction-based method on opendomain profile inference, with the ability to perform both direct extraction and indirect inference. Our further analysis still finds some of the errors and remaining challenges of the generation-based method, such as degraded performances for lowresource attributes and spurious generation, which reveals the limits of our current generation-based user profile inference model.
Limitations
Besides the technical challenges discussed in Section 4.4-4.5, limitations of this work also include the issue of data imbalances that some attributes
may have imbalance distributions. For example, we may find significantly more profiles with the country of citizenship as United States than any other countries, which may have a negative impact on generalization, especially when the distributions of training and inference diverge. Similarly, the distributional variances discussed in Section 4.5 indicate that the prediction results for non-celebrity distributions should be carefully adjudicated. The degraded performances on low-resource attributes also indicate that the prediction results may be unreliable when performing inference on attributes without enough training data.
In this paper, we assume that the attributes are already given. However, many WikiData attributes are not applicable to everyone. For example, attributes such as “position played on team” may be specific to athletes. Therefore, it is also important to investigate how to automatically detect applicable attributes for certain users.
In this work, we use at most 100 recent tweets and aggressively create training and inference examples between each attribute and those tweets. Since we use sliding window on the collected tweets, involving more tweets in training or inference may significantly increase the time cost.
Ethics Statement
The goal of this paper is to extend Twitter user profile inference from limited attributes to the open domain. We hope that this work will help to illustrate how people express their attributes both explicitly but especially also implicitly through their social media posts. We also believe that the NLP community has to produce detailed information about the potential, pitfalls, and basic limitations of profile inference methods so that we can establish standards to facilitate proper use of these technologies, as well as be vigilant and effective at combating nefarious applications.
Data and model biases. To mitigate potential distributional biases, we exhaustively collect entities from WikiData without selecting certain groups of users. However, we acknowledge that the collective information may still contain unintentional social biases. As an example, one of the potential issues is that people who have WikiData profiles are public figures, which may not reflect the actual distribution over general populations (e.g., occupation). Besides, as in Abid et al. (2021), large language models themselves may contain biases.
WikiData is constantly edited by a large number of WikiData contributors and maintainers. Although we try to make our study as representative as possible, it is possible that a statement from WikiData may not reflect the preception from certain groups or individual (Shenoy et al., 2022). We would like stakeholders to be aware of these issues and we urge stakeholders to first investigate the effect of potential issues before drawing any conclusions for any individual or social group using this work.
Proper use v.s. improper use. The major difference between proper use and improper use is whether the use case follows necessary legal and ethical regulations or framework. For example, Williams et al. (2017) propose an ethical framework based on users’ consent to conduct Twitter social research. If the information is not publicly available, one must obtain consent. Opt-out consent can be used when the information is not sensitive, otherwise opt-in consent is required. With proper regulations, this work can be used to enhance personalized user experience, investigate what stakeholders to know to effectively protect personal information.
Sensitivity of personal information. In this work we follow Twitter Developer Agreement and Policy and remove sensitive personal information. But it is still possible to infer sensitive information indirectly. For example, “candidacy in election” may be possibly used to infer political affiliation although the affiliations are generally public for those people. Similarly, personal pronouns, widely present in tweets, may also be used to infer gender. Furthermore, combinations of various sources might allow personal identification (Sweeney, 2000a,b). Even though we do not use private information in our work, based on our results, we speculate that there are unobserved risks of privacy loss for using Twitter. Therefore, We ask that future work should fully comply with regulations and any non-public or private results should be properly protected (Kreuter et al., 2022).
We have set up the following protocol to ensure the proper use and to prevent adverse impact:
• We believe that increasing the transparency of the pipeline can help prevent potential social harm. We plan to release all necessary resources for research reproduction purposes so that others can audit and verify it and prevent overestimation of the model. We also provide a complete list of attributes in Table 7 to increase the transparency.
We are open to all further explorations that can prevent unintended impacts. • Our constructed dataset for profile inference research is drawn solely from publicly available WikiData and Twitter, where the ethical consideration should be similar to other work using encyclopedia resources such as (Sun and Peng, 2021). Furthermore, according to WikiData: Oversight, non-public personal information are monitored and removed by Wikidata. According to WikiData Term of Use, we can freely reuse and build upon on WikiData. According to the Twitter Developer Agreement and Policy, we will only release IDs instead of actual content for noncommercial research purposes from academic institutions. • To ensure the proper use of this work, we will not release the data via a publicly available access point. Instead, we will release the data based on individual request and we will ask for consent that 1) requesters are from research institutions 2) they will follow all the regulations when using our work 3) they will not use the model to infer non-public users unless obtained proper consent from those users.

heading: A Detailed Experiment Setup
text: We use two Nvidia GeForce RTX 3090 GPUs as our computing infrastructure.
Extraction-based method setup. We finetune the model with 10 epochs using AdamW. The learning rate is 5e-5 using the linear scheduler without warmup. The batch size is 128. The hidden size for classification is 768. The positive-negative sample ratio is 1:5. We use tag-level F1 as Qian et al. (2019) to select the best results on the development set efficiently for a single run. The training time is about 16 hours and inference on test set is about 5 hours.
Generation-based method setup. We fine-tune the model on all sliding window examples for 5 epochs using AdamW. The learning rate is 1e-4 using linear scheduler with no warmup. The batch size is 96. We use gradient clipping with max norm 3 to increase stability during training. We use sliding windows with size 512 and stride 128. We use greedy search during inference. We use exact match to select the best results on development set efficiently for a single run. The training time is about 40 hours and inference on test set is about 3 hours.

heading: B Attribute Descriptions
text: We provide the descriptions of each attribute from Wikidata in Table 7 to facilitate the understanding of attributes and mitigate the potential impact from dataset biases.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: 4.5 Limitations and Challenges Ethics Statement

heading: 3 A2. Did you discuss any potential risks of your work?
text: Ethics Statement

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: Abstract 1 Introduction
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts?

heading: 1 Introduction 2 Problem Definition and Dataset

heading: 3 B1. Did you cite the creators of artifacts you used?
text: 1 Introduction

heading: 3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts?
text: 2.2 Dataset Creation Ethics Statement

heading: 3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided
text: that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Ethics Statement

heading: 3 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any
text: information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Ethics Statement

heading: 3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and
text: linguistic phenomena, demographic groups represented, etc.? 2.2 Dataset Creation Ethics Statement
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. 2.2 Dataset Creation
C 3 Did you run computational experiments?

heading: 4 Experiments

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? 4.1 Experiment Setup Appendix A Detailed Experiment Setup
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? 3.2 Generation-based method 4.1 Experiment Setup Appendix A Detailed Experiment Setup

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Appendix A Detailed Experiment Setup

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? 3.2 Generation-based method 4.1 Experiment Setup
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: Therefore, We ask that future work should fully comply with regulations and any non-public or private results should be properly protected (Kreuter et al. , 2022). We have set up the following protocol to ensure the proper use and to prevent adverse impact:
• We believe that increasing the transparency of the pipeline can help prevent potential social harm. We plan to release all necessary resources for research reproduction purposes so that others can audit and verify it and prevent overestimation of the model. We also provide a complete list of attributes in Table 7 to increase the transparency. We are open to all further explorations that can prevent unintended impacts. • Our constructed dataset for profile inference research is drawn solely from publicly available WikiData and Twitter, where the ethical consideration should be similar to other work using encyclopedia resources such as (Sun and Peng, 2021). Furthermore, according to WikiData: Oversight, non-public personal information are monitored and removed by Wikidata. According to WikiData Term of Use, we can freely reuse and build upon on WikiData. According to the Twitter Developer Agreement and Policy, we will only release IDs instead of actual content for noncommercial research purposes from academic institutions. • To ensure the proper use of this work, we will not release the data via a publicly available access point. Instead, we will release the data based on individual request and we will ask for consent that 1) requesters are from research institutions 2) they will follow all the regulations when using our work 3) they will not use the model to infer non-public users unless obtained proper consent from those users