U ser I dentifier: Implicit User Representations for Simple and Effective Personalized Sentiment Analysis (2022)

Abstract: Global models are typically trained to be as generalizable as possible. Invariance to the specific user is considered desirable since models are shared across multitudes of users. However, these models are often unable to produce personalized responses for individual users, based on their data. Contrary to widely-used personalization techniques based on few-shot and meta-learning, we propose UserIdentifier, a novel scheme for training a single shared model for all users. Our approach produces personalized responses by prepending a fixed, user-specific non-trainable string (called “user identifier”) to each user’s input text. Unlike prior work, this method doesn’t need any additional model parameters, any extra rounds of personal few-shot learning or any change made to the vocabulary. We empirically study different types of user identifiers (numeric, alphanumeric, and also randomly generated) and demonstrate that, surprisingly, randomly generated user identifiers outperform the prefix-tuning based state-of-the-art approach by up to 13, on a suite of sentiment analysis datasets.

## introduction personalization arises in applications where different clients need models specifically customized to their environment and user profiles @xcite @xcite . this need for customization stems from the inherent heterogeneity existing in the data and the labels, especially when the task is classification @xcite . fig. 1 shows an example of the sentence "that is just great!". this sentence could carry a positive sentiment, a neutral apathetic sentiment, or even a completely negative sentiment. a non-personalized model cannot correctly predict the label for different users. most techniques for personalization generally involve two phases: first, a shared, global model is activation text ## useridentifier in this section, we first explain how useridentifier operates, then we go over the parameterization and learning procedure. ## results apart from the evaluations here, we have also provided evaluations of applying our method to federated learning in appendix a.1, and applying it to new unseen user samples in 4.4. (consecutive numbers) and def. (default username) scenarios, as detailed in section 4.2, the length of the user identifier sequences depends solely on the tokenization process. for the case of rand. all (randomly sampled from all vocabulary tokens), however, it is shown that the sequence length of 10 tokens provides the best performance through the ablation study, therefore the results are reported for this length. since the default usernames for imdb and yelp datasets are not provided, the corresponding results are not reported here. it is shown that useridentifier with randomly generated identifiers outperforms all baselines, in all tasks. our intuition is that useridentifier outperforms useradapter because of collaborative learning and personalization happening simultaneously, unlike in the case of useradapter where personalization is performed separately for each user. the performance of trainable user embeddings appears inferior to that of useridentifier, which could be attributed to the parameter tying used in useridentifier. this parameter tying couples the learning problems for both domains (user identifier and text) and allows us to jointly learn from the full data, as in @xcite . for the sentiment140 dataset, we can see that increasing the heterogeneity or skew in the dataset boosts the benefits brought about by useridentifier. this shows that the proposed method performs better in setups where personalization is actually needed @xcite . ## conclusion in this work, we present a novel approach for learning global models, producing personalized classification responses. this method which doesn't require model extensions or specialized training algorithms, consists of appending a fixed, non-trainable, unique identifier string to each sample during training and inference.