Learning Interpretable Style Embeddings via Prompting LLMs
Authors: Ajay Patel
Delip Rao
Ansh Kothary
Kathleen McKeown
Chris Callison-Burch
Venue: EMNLP 2023 Findings (2023)
Keywords: style
stylometry
representation learning
embeddings
vectors
interpretability
prompting
llm
TL;DR: In this work, we train interpretable style embedding models using large language models and prompting.

Abstract: Style representation learning builds content-independent representations of author style in text. To date, no large dataset of texts with stylometric annotations on a wide range of style dimensions has been compiled, perhaps because the linguistic expertise to perform such annotation would be prohibitively expensive. Therefore, current style representation approaches make use of unsupervised neural methods to disentangle style from content to create style vectors. These approaches, however, result in uninterpretable representations, complicating their usage in downstream applications like authorship attribution where auditing and explainability is critical. In this work, we use prompting to perform stylometry on a large number of texts to generate a synthetic stylometry dataset. We use this synthetic data to then train human-interpretable style representations we call LISA embeddings. We release our synthetic dataset (StyleGenome) and our interpretable style embedding model (LISA) as resources.

Learning Interpretable Style Embeddings via Prompting LLMs
Ajay Patel
Delip Rao
Ansh Kothary
University of Pennsylvania a ajayp@upenn.cm edu
University of Pennsylvania delip deliprao@gmail.. com
Columbia University
Kathleen McKeown
Chris Callison-Burch University of Pennsylvania ccb upenn. edu
Columbia University kathyêcs. www.wooa...ceeoooooooooo......oe
Abstract
"Really really cool action movie imo. Cassting is greet too, it's like they were born their roie that movie."
Style representation learning builds content- independent representations of author style in text. To date, large dataset of texts with sty- lometric annotations on wide range of style di- mensions been compiled, perhaps because the linguistic expertise perform such annota- tion would be prohibitively expensive. There- fore, current style representation approaches make use of unsupervised neural methods to disentangle style from content to create style vectors. These approaches. however, result in uninterpretable representations, complicating their usage in downstream applications like au- thorship attribution where auditing and explain- ability is critical. In this work, we use prompt- ing to perform stylometry on a large number of texts generate synthetic stylometry dataset. We use this synthetic data to then train human- interpretable style representations
LISA
768 dimensional interpretable style vector
[0.889 ... 0.613, 1.000, ... 0.949
The author repeating words.
The author is complimentary.
Figure I: An example of a 768 -i8ddimensiona interpretable style vector produced by LISA, trained using GPT-3 annotated synthetic stylometery dataset.
call LISa embeddings. We release our synthetic dataset (STYLEGeNOME) and our interpretable style embedding model (LiSa) resources.
2009; Tausczik and Pennebaker, 2010). More mod- ern, neural approaches attempt to learn style rep- resentations in unsupervised fashion through a proxy task like style transfer (Shen al., 2017; Fu et al., 2018: 8; John et al., 2019; Dai al., 2019; Li et al., 2019; Yi al., 2021; Zhu al., 2022) or authorship verification (Boenninghoff et al., 2019; Hay et al., 2020; Zhu and Jurgens, 2021; Wegmann et al., 2022). These stronger neural approaches, unlike simpler frequency based techniques, are uninterpretable. This makes it difficult to effec- tively analyze their representations and their failure modes, and precludes their usage in real-world thorship attribution scenarios because interpretabil- ity and verification is critical for legal admissibility (Tiersma and Solan, 2002).
1 Introduction
Style representation learning aims to represent the stylistic attributes of an authored text. Prior work has treated the style of text as separable from the content. Stylistic attributes have included, but are not limited to, linguistic choices in syntax, gram- mar, spelling, vocabulary, and punctuation (Jafari- tazehjani et al., 2020). Style representations should represent two texts with similar stylistic attributes more closely than texts with different attributes independent of what content present in the texts. Stylometry, the analysis of style, applies foren- sic linguistics to tasks like authorship attribution. Stylometry often relies on semi- manimllnual analysis by forensic linguistic experts (Mosteller and Wallace, 1963; Holmes, 1994; Rosso et al., 2016). Computa- tional stylometry often uses rule-based approaches utilizing count- count-based features like the frequencies function words (Stamatatos, 2009; Koppel et al.,
With this motivation, we propose human- interpretable style representation model M which, for given text t, produces D-dimensional vector M(I [0.0, 1.0) Each dimension corresponds
to one of D style atuributes {ap, 01..... aD). Each element at dimension d of this vector is constrained in the range [0.0, 1.0] to represent the probability the corresponding style attribute ad being present

---

in the text See Figure 1 for visualization of a result from our final trained model with D 768 dimensions. An immediate obstacle to train such a model is that no large dataset of texts with sty- lometric annotations currently exists; annotating a large number of texts on wide variety (D = 768) of stylistic attributes would likely require annota- tors with linguistic expertise and be prohibitively expensive. Given this, we use GPT-3 (Brown et al., 2020), large language model (LLM), and zero- shot prompts to generate synthetic dataset we call STYLEGENOME of inter-intcrprettable stylomet- ric annotations for various texts. Our approach is motivated by recent works showing models trained on synthetic datasets annotated by prompt- ing LLMs can match and sometimes even outper- form models trained on human-labelee datasets (Wang al., 2022: Gilardi al., 2023; Huang al., 2022; Honovich et al., 2022). Training on STYLEGENOME, we develop the Linguistically- Interpretable Style Attribute (LISA) embedding model. We summarize our primary contributions:
punctuation, emoji usage, etc.:
Reddit User #47: forreal though sell that ****
its worth like 6508
Reddit User #205: This was amazing :D
Please, make more!
Reddit User #216: I thought that was Israel...or Saudi Arabia...or Egypt... Venezuela? Reddit User #356: wouldn't watch football game like that either. It's novel, but I think its appeal is limited and again, not the same thing actual VR where things feel tangible and scale.
We use GPT-31 (Brown et al., 2020; Ouyang et al., 2022) to annotate each post in two stages.
Stage
We elicit the model with a zero-shot prompt to generate a description of the style of a given Reddit post:
Write a long paragraph describing the unique grammar style of the following passage without referring to specifics about the topic.
Prompt:
Wow that's 's really cool. I'll have to watch for that next time I watch it.
Passage:
Description:
1. We outline an unsupervised method for pro- ducing interpretable style embeddings using zero-shot prompting and distillation.
The passage reffects conversational style its grammar, with short, simple sentences that easy understand. The language is informal and direct, with the speaker expressing their enthusiasm for the topic in straightforward manner. The speaker also contractions, such as "I'll" to emphasize their excitement. The speaker also uses a casual tone, with the phrase "really cool" to emphasize their interest the topic. The speaker also uses the present tense express their anticipation for the future, with the phrase "next time watch it." This style of grammar is effective in conveying the speaker's 's enthusiasm and interest the topic.
Generation:
2. We generate and release STYL a synthetic stylometry dataset with -5.5M ex- amples, the first large-scale dataset with texts paired with wide range of stylometric annota- tions.
3. We train, evaluate, and release LISA, the first-ever interpretable style embedding model with a wide varicty of linguistic dimensions (D - 768). We find LISA matches the perfor- mance of existing style representations, while allowing for explainability and transparency.
Stage 2
We use another zero-shot prompt standardize the generations from Stage into short, declarative sentences with a uniform structure:
2 Generating STYLE
Prompt: Here's a description of an author's writing style for passage: The passage utilizes a comversatinnal style of grammar, with short...
To create STYLEGENOME for training LISA, we select Reddit data from the Million User Dataset (MUD) (Khan et al., 2021; Andrews and Bishop, 2019) to stylometrically annotate following prior work that uses Reddit to source a diversity of styles from different authors (Wegmann et al., 2022). We sample 10 random posts per author for 1,000 ran- dom authors, resulting in 10,000 total posts se- lected for annotation. We display some of the di- versity of styles captured in the following examples from our Reddit authors. They vary in formality,
Rewrite this description as a long list of short sentences describing the author's writing style where each sentence is in the format of "The author is X." or "The author uses X.".
Output:
'We use text-davi inci-003 with temperature setting of 0.0 and a top_p setting of 1.0.

---

The author is using conversational style of grammar.
Generation:
tone" are counted separately in the raw dataset. Our training procedure in Scction 3.1 is able train directly on these natural language style attributes, obviating a normalization step. Some annotations may be hallucinated resulting in a noisy dataset, but we choose to train on the full synthetic dataset, without manual intervention, to maintain unsu- pervised procedure following prior work (Wang et al., 2022). We hypothesize our model will find signal in the noise, which we evaluate Section 4. The final dataset statistics can be found in Table 1.
The author is using short, simple sentences.
The author is using language that is informal and direct.
The author is expressing enthusiasm for the
topie in a straightforward manner.
The author is using contractions, such "I'Il.. The author is using casual tone.
The author is emphasizing their interest in the topic with the phrase "really cool".
The author is using the present tense to express anticipation for the future.
We run both stages with 93 different Stage prompts for all 10,000 posts. 6 of the 93 total prompts open-ended prompts that elicit descrip- tions of passage on broad dimension of style:
# of Reddit Authors
1,000
# of Reddit Posts
10,000
# of Interpretable Style Attributes
1,255,874
# of (Text, Style Attribute) labeled pairs
5,490,847
Write a long paragraph describing the unique grammar style of the following passage...
Table I: Statistics for the STYLEGENOME dataset.
2. Write long paragraph describing the unique vocabulary style the following passage...
3
Method
3. Write a long paragraph describing the unique punctuation style of the following passage...
We first distill stylometric annotation knowledge from GPT-3 into Style Feature Agreement Model (SFAM). Given a text t and a style attribute a as input, SFAM(t, a) produces an agreement score be- tween 0.0 and 1.0 representing the probability of the style attribute being present in the text. By lecting a set of 1) style attributes (ao, .i.-aa...-.. ...... can use SFAM construct LISA, our inter- pretable style representation model that produces D D-dimensional vectors:
4. see all 6 open endede prompts in Appendix A.I
The remaining 87 prompts target narrow and spe- cific dimensions of style:
1. Write description of whether the author of the following passage has any figurative language ...
2. Write a description of whether the author of the following passage has swear words
3. Write a description of whether the author of the following passage has any repeated words...
4. ... all 87 targeted prompts in Appendix A.2
Musa (t) = (Sfamit, aa), SFAM(L, a.).... SFAM(t,
The 87 targeted prompts are derived from sur- veys of stylometry literature, and they cover all of (Tausczik and Pennebaker, 2010)'s linguistic and psychological categories. See Appendix A.2 for more details. We report the results of an ablation experiment between the two Stage prompt cate- gories in Appendix C. Appendix D details dataset annotation costs.
The Euclidean distance between style vectors for two texts [|MLisa MLISA would not be particularly meaningful. We can multiply a trained weight vector w or a weight matrix W to the style vectors, that act as simple interpretable embedding layers. This operation would make the Euclidean distance more meaningful, for example (t1ssatt) w=122- We call the
MLisa (t2) result of a LISA style vector multiplied by w or W LISA style embedding. We discuss training detail next, leaving hyperparameter and implemen- tation specifics in Appendix E.
The output of Stage is sen- tence t and filtered to keep only sentences beginning with "The author". We refer to these sen- tences as human- interprrrttreet style attributes. Our method annotates the texts with nearly 1.3M style attributes. These style attributes are represented in natural language so "The author creates a conversa- tional tone" and "The author has conversational
STYLEGENOE
3.1 SFAM
We use distillation (Ba and Caruana, 2014) to teach the stylometric annotation capabilities of GPT-3 to EncT5* (Liu et al., 2021; Raffel et al., 2020), smaller, more efficient student model.
We preprocess posts to the first 25 sentences and use the emooi Python package to convert emojis to textual represen- tations better tokenization.
We use the sent eenteeneesppp sppl itter Python package.
"We use the t.5 b-ase model and truncate at 512 tokens.

---

Type
Dataset
Style Attribute
Corvelation (p)
Spearman
Formality
Formality Online Communication
The author informal language.
0.599 0.200)
Grammarly' Answers Formality Corpus
Yelp Reviews
0.788
Sentiment
IMDB Large Movie Review Dalaset
The author uses negutive tone.
0.665
abbreviated for space, see Appendix F for full results
DAIR. Al Emotion (Love vs. Anger)
0.542
DAIR. Al Emotion (oy vs. Sad)
0.531
GoEmotions (Love Anger)
0.769
Emotion
GoEmotions (Joy vs. Salness)
The author expressing [/emotion]]
0.639
GoFmintions (Disgust vs. Desire)
0.630
..bbbeevitted for space, Appendix F for full results
Political Slant
The author Democrat.
0.005
Twitter User Gender Classification
The female.
0.166
Author Profiling
African-American Vernacular English
The author uses A AeeercAmeriiaan Vermac-
0.238
ular English.
Shakespeare (Farly Moderm English)
The author uses Early Modem English.
0.108
Wikipedia Bias
The author
biased point view.
0.014
Harmful
HateSpeech18
The author's 's writing contains speech.
0.229
Speech
Offensive Social Media
The author uses offensive language.
0.401
Text
Simple Wikipedia
The author uses simple language.
0.043
Simplification
ASSET
0.050
Acceptability
Linguistic
CoLA
BLiMP
The author uses incorrect grammar.
0.020
0.077
Average
0.342
Table 2: Correlation of agreement scores produced by SFAM against human judgments texts over wide variety linguistic and authorship dimensions. The natural language style attributes used as input to SFAM when producing the agreement scores for each dataset are also provided.
Sampling Batches We train EncT5 with a binary classifier head on randomly sampled batches of ex- amples (I4,34 34) where each batch contains an equal number of positive (ys- = 1) and negative (Ui = 0) examples. The input z; consists of a style attribute a and author's text / concatenated in a string "(aa) Illl (t))", for example x; "The author is using a positive Yllo got this :)" Labeled pairs from STYLEGENOME are sampled as positive each style attribute is sampled with equal probability. For each positive example, we perform negative sampling and retrieve neg- ative example text where the positive example's 'ss style attribute is likely not present. To do this, find the 10,000 most dissimilar style attributes to the positive example's 's style attribute with SBERTS similarity. We select a text that is positively labeled with randomly selected dissimilar style attribute as the negative example text.
and generalize to unseen style attributes using the semantic information in those natural language de- scriptions. This is especially desirable since some style attributes only have handful of text exam- plcs, while others may have thousands. This setup resembles the multitask mixture trained in Raf- fel et al. (2020), To validate training, we hold-out 50 random style attributes that have between 30- 50 examples each as a validation set. We validate learning during training by measuring the ability of STAM to generalize and produce accurate agrec- ment scores for the unseen style attributes. At infer- ence, we softmax the binary class logits to interpret them probabilities and take the probability of yi as the agreement score. We also study the effect of the size of STYLEGENOME on perfor- mance and find that as the synthetic dataset grows, validation performance improves and SFAM gener- alizes to better predict agreement scores for unseen style attribute and text pairs (see Appendix B).
examples such that
Training and Inference unique style attributes in STYLEGENOME, our training dataset for SFAM is effectively multi- task mixture. Style attributes presented in natural language to the model allows the pre-trained T5 encoder to jointly learn between style attributes
Training over the -1.3M
3.2 LISA Style Vectors
As discussed earlier, SFAM is directly used to pro- duce the LISA interpretable style vectors. We ar- bitrarily choose D 768 in this work, follow- ing the dimensionality of prior style vectors and BERT (Devlin al., 2019). We now detail how we select the style attributes associated with cach dimension { (ap, C1,-..,
*We use the nl i diiistiii i roberta saaberta-bbse--ee base-v2 model (Sanh et al. 2019; 9, Liu et al., 2019; Devlin et al., 2019; Reimers and Gurevych, 2019) for all SBERT usage in this paper.
. 0768) with little manual in-

---

Text
Top 5 LISA Vector Dimensions
1. 1.00
The author is being polite.
2. 1.00
The author is writing in cheerful manner.
Lol right
3. 1,00
The author is using lighthearted tone.
4. 1.00
The author is laughing.
5. 1.00
The author is complimentary.
1. 0.99
The author is simply describing product. *
There is the Toyota GT86 R3 :) htp://www.toyolt
2. 0,99 - The author is providing a visual cue to the reader.
wttppppwwwooooooooooooooooooodea DSF4311.jpg
3. 0.98 The author simply provides information.
4. 0.97 The author is using emoji.
5. 0.97 The author provides information.
1. 1.00 - The author is discussing a television show.
Every time watched this episode as a kid was always like "WTE, JAMES, A POKEBALL ISNT EVEN A POKE- MON?! GET YOUR ACT TOGETHER, SON!!"
2. 0.99 The author has a hint of nostalgia.
3. 0.99 The author is making a humorous comment.
4. 0.99 - The author is animated in their writing.
5. 0.99 The author is laughing.
The author is using an elongated word.
2. The author is using only English words.
13 POT MORDE BABY WO00000000000000
3. 1.00 The author is using a single word.
4. 1.00
The author is using swear words.
5. 1.00
The author uses two exclamation marks.
1. 1.00
The author is scorfful.
2. 1.00
The author is ungenerous.
No wonder everyone resorts to performing a murder spree oventually.
3. 0.99
The author is expressing antisocial hehaviors.
4. 0,99
The author is uncaring.
5. 0.99
The author is dramatic.
1. 0.97
The author uses variety of words to describe
the same concept.
2. 0.97
The author is simply describing product.
Podcast originally refers to an iPod, and before that there was definitely TWIT, which still calls itself a Netcast
3. 0.95
The author uses specific terms related to the topic.
4. 0.94
The author has a deep understanding of the topic.
5. 0.94 The author is using words focusing on the past.
Table 3: The five highest scoring dimensions from the 768-dimensional LISA vector produced on various Reddit texts. The interpretable style attribute corresponding to each dimension is displayed along with the score. We manually inspect the top style attributes and annotate them as reasonable, plausible, or incorrect. Attributes annotated with blur the line between style and content. Error analysis can be found in Section 4.1.
tervention. The first 87 attributes {oo, a1...., ) directly correspond to the features of our 87 tar- geted prompts in the form of "The author is using (targeted_ (rrgeteeeeettrreee The remaining 681 are downselected from the ~1.3M style attributes with filtering heuristics, choosing those that appear for at least 10 authors, but no more than 600 authors (attempting to select for frequent, yet discrimina- tive attributes). Once a style attribute is selected to be part of the 768, we do not select another style attribute with SBERT cosine similarity > 0.8 to largely avoid near-duplicates. We also reject style attributes for selection that are undesirable for in- terpretability." Examples of LISA can be found in Figure and Table 3. With 768 dimensions,
producing single LISA vector would require 768 inferences of SFAM, computationally expensive operation. To address this, we produce the LISA representations for 1,000,000 random Reddit posts from MUD. We then distill into new EncT5 model with 768 regression labels. We hold-out 10,000 ex- amples as a validation set. After distillation to the dedicated model, the 768-dimensional style vec- tor can be produced in a single forward pass with minimal degradation (validation MSE = 0.005).
3.3 LISA Style Embeddings
We experiment with two different simple and inter- pretable embedding layers, a weight vector (w768) and weight matrix (W768x64). We attach these on top of the LISA model and train just the layer us- ing a contrastive learning objective and triplet loss (Khosla et al., 2020; Schroff et al., 2015). We also experiment with two different authorship datasets
We reject style attributes that are > 120 characters, are not pure ASCII, contain "not" or "avoids" (negative state- ments), or contain quotes or the word "mentions" (these at- tributes tend to he more relevant to content than style).

---

Model
Formal
Complex
Numb3r
C'ioon
Avg
Interpretable
Random Baseline
0.50/0.50
0.50/0.50
0.50/0.50
0.50/0.50
0,50/0.50
Content-Aware Representations
SBERT
0.78/0.00
0.54/0.01
0.81/0.04
0.86/0.00 00
0.75/0.0 01
LUAR
0.80/0.14
0.67/0.00
0.74/0.03
0.77/0.00
0.75/0.04
X
Content-Independens Style Representations
LIWC
0.52/
0,52/
0,50/
0.99/
0.63/
Wegmann et al. (2022)
0.84/0.69
0,59/0.26
0,56/0.03
0.96/0.02
0.74/0.25
X
LISA
0.69/0.07
0,57/0.01
0,80/0.03
0.77/0.00
0.7100.03
LISA (Wegmann w)
0.72/0.07
0.61/0.03
0.81/0.08
0.68/0.00
0.71/0.05
LISA (Wegmann W)
0.66/0.03
0,56/0.01
0.70/0.01
0.87/0.00
0.70/0.01
LISA (LUAR + w)
0.73/0.05
0.65/0.00
0.85/0.03
0,92/0.00
0.79/0.02
LISA (LUAR- + W)
0.81/0.07
0.56/0.01
0.74/0.03
0.82/0.00
0.73/0.03
Table 4: Accuracy scores on STELJSTEL-or-Cnntre an evaluation framework for style measures proposed by Wegmann and Nguyen (2021) and Wegmann et al. (2022). "LIWC" results are from Wegmann and Nguyen (2021). "LISA" is the 768-dimensional style vector. "LISa (...) uses LISA embeddings with training dataset and embedding layer type denoted in (...). Gray indicates worse than random baseline performance the adversarially challenging STELLor-Conten task. All approaches underperform on STEL-or-Content but LISA approaches outperform or closely match existing style representation choices on STEL, while providing interpretability.
from prior works to train the embedding layer; we refer to these datasets as the Wegmann dataset (Wegmann et al., 2022) and the LUAR dataset (Rivera-Soto et al., 2021). Like the prior work, we assume an author has consistent style between their different texts. Given some anchor text by an author, we use another text by the same author as a positive example, and text by a different author as a negative example for our triplets. This objective minimizes the distance between two texts by the same author and maximizes the distance between texts by different authors, learning meaningful metric.
of the effectiveness of GPT-3 in annotating these categories, an expected result given prior work has shown language models struggle with identifying these features (Warstadt et al., 2020). Interestingly, SIAM demonstrates some limited ability perform authorship profiling, task adjacent to stylometry. The ability to probe SFAM in an interpretable man- ner helps identify which categories of features it can reliably represent, whereas prior approaches were more opaque. Overall, the Table 2 results demonstrate SFAM's annotations do correlate with human judgments on some important dimensions of style. We hypothesize future research with larger datasets 10,000 posts), more diverse sources of texts, and larger and more performant LLMs may further broaden and improve Icarned stylometric annotation capabilities.
4 Results
We first evaluate to what degree SFAM, which is ul- timately used to build LISA representations, learns useful stylometric annotation capabilities that align with human reviewers. We then evaluate LISA itself on STEL, framework purpose-built for eval- uating the quality of style measures (Wegmann and Nguyen, 2021). All evaluations were completed after the collection of the STYLEGENOME dataset,
STEL In Table 4, provide the results of eval- uating LISA using STEL. The STEL task evalu- ates whether two texts with similar styles can be matched using the distance/similarity metric de- fined by a style representation. We compare with other content-indeppedent style representations, or methods that explicitly limit representation of con- tent in favor of style. LISA explicitly limits the representation of content through the 768 style- focused attributes that act as a bottleneck. Content- aware representations like SBERT, on the other hand, have direct access the text and may be able to represent the content in the text to an extreme degree, representing the usage of a specific rare
Correlation to Human Judgments We conduct broad set of 55 studies across 21 datasets in 7 dis- tinct categories of linguistic style and authorship dimensions in Table 2. We measure the correlation of STAM'S agreement scores to human judgments. SFAM performs stronger on dimensions like formal- ity, sentiment, and emotion than dimensions like linguistic acceptability. This is likely an artifact

---

Text
Style Attribute
Subseribed, Interesting idea. I would like to percentages to different locations on court, For example, having the court broken up into maybe 12 zones and then hitting percentages from cach position to those zones. remember seeing article that did this years ago and I have never been able to find anything online. said recently on this sub that the deep angle shot from the left side right side was the highest percentage shot in volleyball, but
advanced stats hitting
The author is being polite.
not able back up my claim with any sources or anything. Anyways, I am a VB nerd, no doubt. Interested to see what direction you take this. Cheers!
Yeah also work QA. and seeing this kind of stuff gel released maddening. Ahout a year ago working on new platform we were seeing bugs in the hun- dreds each week. we pushed back the release of the product months because basically didn't work. If was up to the devs, they'd have released it on time, because the stuff they'd 'l written code for worked. Thorough doesn't 't even cover the work we go through every 3 months, and Niantic's 's approach seems completely amateur from this side. They's 're putting bandaids on problems and hiding things like the step problem behind curtains without scemingly fixing anything although I have to say their balance tweaks to battling have been big step in the right direction.
The author is using a personal aneodote to illustrate their point.
Thank you. I'd be interested in reading more about your experiences, in addition to the Ameriiaa Wedding" story. Are you watching the stream? wish there was a way to find out how many people in the world are watching it. The music is lovely. huh? God damn. He's got his bunny Fair Isle sweater on, drinking Dunkin' Donuts coffee. would have thought him Starbucks man. :-)
The author is using an emoji.
Table 5: Sentence-level LISA vectors over each sentence from a longer passage of text can help identify and quantity which sentences contribute to overall style attributes scored on the longer passage providing granular interpretabillty.
word or discussion of a specific concept. We pro- vide the results of content-aware representations simply for reference. We find LISA embeddings are able to closely match (and on average slightly outperform) prior style representations on STEL while providing interpretability.
cation task from Wegmann et al. (2022) in Table 6 with LISA (LUAR + W). Further examples and dis- cussion on how the top common and distinct style attributes are ranked can be found in Appendix G.
4.1 Error Analysis
We highlight insights and observations around com- mon failure modes of our technique in this section. We annotate the common failure modes with their percentage rate of occurrence.
Sentence-Level Interpretabilityy In Table 5, we demonstrate how visualizing a dimension of sentence-level LISA vectors can help explain which sentences contribute to a dimension activated on a passage-level LISA vector.
Content vs. Style Attributes (3%) It is unclear whether style and content can truly be separated as some content features are important for style or profiling an author (Jafaritazehjani et al., 2020; Bischoff et al., 2020; Patel al., 2022). Even after filtering, 3% of dimensions of LISA still represent content. For example, "The author is using words related to the game they are discussing". However, while LISA may have the ability to represent that two texts are both discussing the topic of video games, it does not have the direct ability a content- aware approach would of representing which spe- cific video game is being discussed, due to the
Forensic Interpretability Typically for author- ship attribution tasks, content-aware representa- tions that capture both content and style are used to make a determination. Author style, however, is still an important component in determining attri- bution (Rivera-Soto et al., 2021). Offering a clear explanation and presenting supporting evidence is crucial, particularly in the context of forensic anal- ysis, such as when presenting evidence in a court trial. Explainability has often been overlooked in neural approaches to authorship attribution tasks. To motivate this as future research direction using our interpretable stylometric representations and our general approach, we provide an example of explanations on the Contrastive Authorship Verifi-
"W manually inspect small sample set of 2,000 style attribute annotations (the top 20 style attributes for 100 random texts) by LISA.

---

Texts
Top 3 Common/Distinet Style Attributes
Anchor: Devices that use two pronged instead of three pronged plugs are required to meet certain safe design re- quirements. Among other things, if a device has switch, the switched line MUST BE hot, not neutral. The polarized plugs make sure that the right prong/wire is hot. This is why devices that have no switches (primarily wall warts) need not have polarized plugs.
Same Author: Your diaphragm would be trying to contract against the air pressure in your lungs. That's why deep sea diving requires regulators, to match the pressure of the air supply to the pressure surrounding your rib cage. You can breathe against a maximum of about 1/2 PSI, which is not enough pressure to udequately oxygenate your blond.
(0.89, 1.00)
The author is using a scientific approach.
(0.96, 0.98)
The author using combination of
technical terms and everyday language.
(0.91, 0.84)
The author is using formal and professional language.
to be finding its' niche. Now if they could just make Star Wars version of this game, I'd happily swallow that fat learning curve and overcome my frustrations with the combat system.
Different Author: That's great! I'm glad
(0.06, 0.99)
The author is using words related to the
game they are discussing.
(0.00, 0.88)
The author is using an emoji.
(0.02, 0.87) The author uses an emoticon at the end.
Table 6: Example interpretable explanations on the Contrastive Authorship Verification task. The top style attributes in common between the Anchor text and text by the Same Author are shown. The top distinct style attributes between the Anchor text and a text by a Different Author are also shown. The scores of each style attribute against the texts is shown in (+, +/+). Attributes annotated with * blur the line between style and content. Error analysis can be found in Section 4.1. Further examples and details on style attribute ranking can be found in Appendix G.
limited set of 768 features that act as a bottleneck. Our upproach also allows visibility into understand- ing how much of the representation derives from content-related features, while other neural repre- sentations are opaque and may use content-related features in way that cannot be easily assessed.
hallucination and annotation errors by GPT-3. Hal- lucinated generations is a common issue with any LLM-guided approach and we discuss it further in Limitations along with potential future mitigations.
5 Conclusion
Conflating Style Attributes with Content (2%) For some style attributes, LISA conflates the con- tent of text with the presence of the style attribute. For example, "The author is cautious", may have a high agreement score on any text containing the word "caution" " even if the author is not actually expressing caution in the text.
In this work, we propose promising novel ap- proach to learning interpretable style representa- tions. To overcome lack of stylometrically an- notated training data, we use a LLM to generate STYLEGRNOMF, a synthetic stylometry dataset. Our approach distills the stylometric knowledge from STyleGeNome into two models, SFAM and LISA. We find that these models learn style repre- sentations that match the performance of recent di- rect neural approaches and introduce interpretabil- ity grounded in explanations that correlate with human judgments. Our approach builds towards research direction focused on making style rep- resentations more useful for downstream applica- tions where such properties are desirable such as in a forensic analysis context. Future directions that introduce human-in in-the-loop supervised anno- tations or newer, larger, and better aligned LLMs for annotation have the potential to yield further gains in both performance and interpretability.
Spurious Correlations (6%) For other style at- tributes, LISA has learned spurious correlations. For example, "The author uses two exclamation marks", often has a high agreement score on any text that is exclamatory in nature, but does not ac- tually use exclamation marks. An example can be found in Table 3.
Fundamental Errors (10%) LISA sometimes produces high agreement score for text displaying the polar opposite of style attribute or produces high agreement score for attribute that simply is not present in the text. Table demonstrates some of these incorrect examples. Inspecting our dataset, this error happens both due to EncT5's in- ternal representations likely aligning on relatedness instead of similarity (Hill et al., 2015) and due to
Model and Data Release We release our dataset (STYLEGENOME) and our two models (SFAM and LIsa) to further research in author style.

---

Limitations and Broader Impacts
Acknowledgements
This research based upon work supported part by the DARPA KATROS Program (contract PA8750-19-2-1000 the DARPA LwLL Program (contract FA8750-19-2-0201 the IARPA HIATUS Program (contract 0022-22072200005), and the NSF (Award 1928631). Approved for Public Re- lease, Distribution Unlimited. The views and con- clusions contained herein are those of the authors and should not be interpreted as necessarily rep- resenting the official policies, either expressed or implied, of DARPA, IARPA, NSF, or the U.S. Gov- ernment.
Limitations Handcrafted features by forensic linguists typically rely on frequency counts of word usage, usage of unique words or phrases, etc. (Mosteller and Wallace, 1963). The space of these kinds of features is non-enumerable and would not be well-represented with our technique that scores a fixed set of 768 interpretable features. Pure neural approaches may capture these kinds of features, but are non-interpretable and may capture undesirable content-related features. We explicitly trade-off the use of these kinds of features in this work to achieve interpretability. While we demonstrate our synthetic annotations are enough for a model to learn to identify stylistic properties in text in Table 2, they cannot be fully relied on yet for the reasons we discuss in Section 4.1. As large language mod- els scale and improve, however, we believe this work could benefit from increasing coherency and decreasing hallucination in the annotations (Kaplan al., 2020). STYLEGENOME is collected only on 10,000 English Reddit posts, however, larger datasets may improve performance as we show in Figure? and future research in multilingual LLMs may make it feasible to replicate this procedure for other languages.
References
Fernando Alva-Manchego, Louis Martin, Antoine Bor- des, Carolina Scarton, Benoît Sagot, and Lucia Spe- cia. 2020. ASSET: A dataset for tuning and evalua- tion of sentence simplification models with multiple rewriting transformations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguisties, pages 4668 (668,,679, Online, Association for Computational Linguistics.
Inc. Amazon.com 2018. Amazon Customer Reviews Dataset - s3.amazonaws.com com. https: //s3. a ma mazonaws. , com/
www.oooooeeevieeeeeeeeeeeeeeeeee eadme html. [Accessed I 17-May-2023).
Nicholas Andrews and Marcus Bishop. 2019, Learning invariant representations of social media users. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNIP)) pages 1684-1695.
Ethical considerations Style representations useful for text style transfer (Riley et al., 2021) and in manipulating the output of machine gener- ated text to match a user's style, for example, in machine translation (Niu et al., 70 Rabinovich et al.. 2017). While style transfer can be a useful be- nign commercial application of this work, superior style representations may aid the impersonation of authors. We demonstrate how style representations may aid legitimate cases of authorship attribution, a task that is typically done by forensic linguist experts. Our work introduces an interpretable ap- proach, an important step in legitimizing the use of computational models for authorship attribution by providing explanations for predictions that can be audited and verified.
Katherine Atwell, Sabit Hassan, and Malihe Alikhani. 2022. APPDIA: A discourse-aware transformer- based style transfer model for offensive social me- dia conversations. In Proceedings of the 29th Inter- national Conference on Computational Linguistics. pages 6063-6074, Gycongju, Republic of Korea. In- ternational Committee on Computational Linguistics.
Jimmy Ba and Rich Caruana. 2014. Do deep nets really need to be deep? Advances in neural information processing systems, 27.
Sebastian Bischoff, Niklas Deckers, Marcel Schliebs. Ben Thies, Matthias Hagen, Efstathios Stamatatos, Benno Stein, and Martin Potthast. 2020. The im- portance of suppressing domain style in authorship analysis. arXiv preprint arXiv:2005. 14714.
Diversity and inclusion We believe style repre- sentations that capture wider dimensions of style can help aid in analyzing and representing minority writing styles in downstream applications like style transfer.
Benedikt Boenninghoff, Robert M Nickel, Steffen Zeiler, and Dorothea Kolossa. 2019. Similarity learning for authorship verification in social media. In ICASSP 2019-2019 IEEE International Confer- ence on Acoustics. Speech and Signal Processing (ICASSP), pages 2457-2461. IEEE.

---

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Vosss Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Ma- teusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Ad- vances in Neural Information Processing Systems, volume 33, pages 1877-1901. Curran Associates, Inc.
Fabrizio Gilardi, Meysam Alizadeh, and Maël Kubli. 2023. Chatgpt outperforms crowd-worken for text- annotation tasks. arXiv preprint arXiv:2303. 15056.
Giovanni Grano, Andrea Di Sorbo, Francesco Mercaldo. Corrado A. Visaggio, Gerardo Canfora, and Sebas- tiano Panichella. 2017. Android apps and user feed- back: A dataset for software evolution and quality improvement. In Proceedings of the 2nd ACM SIG- SOFT International Workshop on App Market Analyt- ies, WAMA 2017, page 8-11, New York. NY, USA. Association for Computing Machinery.
Sophie Groenwold, Lily Ou, Aesha Parekh, Samhita Honnavalli, Sharon Levy, Diba Mirza, and William Yang Wang. 2020. Investigating African- American Vernacular English in transformer-based text generation. In Proceedings of the 2020 Con- ference on Empirical Methods in Natural Language Processing (EMNLP), pages 5877-5883, Online. As- sociation for Computational I Linguistics.
William Coster and David Kauchak. 2011. Simple En- glish Wikipedia: A new text simplification task. In Proceedings of the 49th Annual Meeting of the Asso- ciation for Computational Linguistics: Human Lan- guage Technologies, pages 665-669, Portland, Ore- gon, USA. Association for Computational Linguis- tics.
Julien Hay, Bich-Lien Doan, Fabrice Popineau, and Ouassim Ait Elhara. 2020. Representation learning of writing style. In Proceedings of the Sixth Work- shop on Noisy User-generuted Text (W-NUT 2020). pages 232-243.
CrowdFlower. 2017. Twitter User Gender Classification kaggle.com. https:/// / /www. wonggee.oo////mm d atasets crowd ower wtw.e/ttiii.ee tter-user-g onder-classe wwwasssiiiccm ficatiionnnn [Accessed 17-May- 2023].
Felix Hill, Roi Reichart, and Anna Korhonen. 2015. SimLex-999: Evaluating semantic models with (gen- uine) similarity estimation. Computational Linguis- tics, 4 (14):6655665
Ning Dai, Jianze Liang, Xipeng Qiu, and Xuan-Jing Huang. 2019. Style transformer: Unpaired text style transfer without disentangled latent representation. In Proceedings of the 57th Annual Meeting of the As- sociation for Computational Linguistics, pages 5997- 6007.
David I. Holmes. 1994. Authorship attribution. Com- puters and the Humanities, 28(2):87-106.
Ona de Gibert, Naiara Perez, Aitor Carcia-Pablos, and Montse Cuadros. 2018. Hate Speech Dataset from a White Supremacy Forum. In Proceedings of the 2nd Workshop on Abusive Language Online (ALW2), pages 11-20, Brussels. Belgium. Association for Computational Linguistics.
Or Honovich, Thomas Scialom, Omer Levy, and Timo Schick. 2022. Unnatural instructions: Tuning lan- guage models with (almost) no human labor. arXiv preprint ar arXiv:2212.09689)
Jiaxin Huang, Shixiang Shane Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han. 2022. Large language models can self-improve. arXiv preprint arXiv:2210.116110
Dorottya Demszky, Dana Movshovitz-Attas Jeongwao Ko, Alan Cowen, Gaurav Nemade, and Sujith Ravi. 2020. GoEmotions: A dataset of fine tha-grained emo- tions. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4040-4054 Online. Association for Computational Linguisties.
Somayeh Jafaritazehjani, Gwénolé Lecorvé. Damien Lolive, and John Kelleher. 2020. Style versus con- tent: A distinction without a (Icarnable) difference? In International Conference on Computational Lin- guistics.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understand- ing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Com- putational Linguistics: Human Language Technolo- gies, Volume 1 (Long and Short Papers), pages 4171- 4186.
Vineet John, Lili Mou, Hareesh Bahuleyan, and Olga Vechtomova. 2019. Disentangled representation learning for non- parallel text style transfer. In Pro- ceedings of the 57th Annual Meeting of the Associa- tion for Computational Linguistics, pages 424 434..
Jared Kaplan, Sam McCandlish, T. Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeff Wu, and Dario Amodei. 2020. Scaling laws for neural language models. ArXiv. abs/2001.082
Zhenxin Fu, Xiaoye Tan, Nanyun Peng, Dongyan Zhao, and Rui Yan. 2018. Style transfer in text: Explo- ration and evaluation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32.

---

Aleem Khan, Elizabeth Fleming, Noah Schofield, Mar- cus Bishop, and Nicholas Andrews. 2021. A deep metric learning approach to account linking. In Pro- ceedings of the 2021 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5275-5287.
Bo Pang and Lillian Lee. 2005. Seeing stars: Exploit- ing class relationships for sentiment categorization with respect to rating scales. In Proceedings of the 43rd Annual Meeting on Association for Computa- tional Linguistics, ACL '05, page 115-124, USA. Association for Computational Linguisties.
Ajay Patel, Nicholas Andrews, and Chris Callison- Burch. 2022. Low-resource authorship style trans- fer with in-context learning. arXiv preprint arXiv:2212.08986.
Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishman. 2020. Su- pervised contrastive learning. Advances in neural information processing systems, 13:18661-1
Ellie Pavlick and Joel Tetreault. 2016. An empiri- cal analysis of formality in online communication. Transactions of the Association for Computational Linguistics, 4:61-74.
Moshe Koppel, Jonathan Schler, and Shlomo Argamon. 2009. Computational methods in authorship attribu- tion. Journal of the American Sociery for Information Science and Technology, 60(1):9-26.
Shrimai Prabhumoye. Yulia Tsvetkov, Ruslan Salakhut- dinov, and Alan W Black. 2018. Style transfer through back translation. Annual Meeting of the Association for Computational Linguistics ( (Volume I: Long Papers), pages 866-876, Melbourne, Australia. Association for Computational Linguistics.
Dianqi Li, Yizhe Zhang, Zhe Gan, Yu Cheng, Chris Brockett, William B Dolan, and Ming-Ting Sun. 2019. Domain adaptive text style transfer. In Pro- ceedings of the 2019 Conference on Empirical Meth- ods in Natural Language Processing and the 9th In- ternational Joint Conference Natural Language Processing (EMNLP-IJCNLP), pages 3304-331 3313.
Proceedings of the 56th
Reid Pryzant, Richard Diehl Martinez, Nathan Dass, Sadao Kurohashi, Dan Jurafsky, and Diyi Yang. 2020. Automatically neutralizing subjective bias in text. Proceedings of the AAAI Conference on Artificial Intelligence, 34(01) (4(01):480-48.
Frederick Liu, Siamak Shakeri, Hongkun Yu, and Jing Li. 2021. Enct5: Fine-tuning 15 encoder for non-autoregressive tasks. arXiv preprint arXiv:2110.08420
Yinhan Liu, Myle Ou,, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis. Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining ap- proach. arXiv preprint arXiv: 1907. 1692.
Ella Rabinovich, Raj Nath Patel, Shachar Mirkin, Lucia Specia, and Shuly Wintner. 2017. Personalized ma- chine translation: Preserving original author traits. In Proceedings of the 15th Conference the the European Chapter of the Association for Computational Lin- guistics: Volume 1, Long Papers, pages 1074 1084.
Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 142-150, Portland, Oregon, USA. Association for Computational Lin- guistics.
Colin Raffel, Noam Shazeer, Adam Roberts. Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the lim- its of transfer learning with unified text-to-text transformer. Journal of Machine Learning Research, 21:1-6.
Sudha Rao and Joel Tetreault. 2018. Dear sir or madam, may introduce the GYAFC dataset: Corpus, bench- marks and metrics for formality style transfer. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, Volume (Long Papers), pages 129-140, New Or- Icans, Louisiana. Association for Computational Lin- guistics.
Frederick Mosteller and David L. Wallace. 1963. In- ference in an authorship problem. Journal of the American Statistical Association, 58(302):75-309.
Ibrahim Naji. 2012. TSATC: Twitter Sentiment Analy- sis Training Corpus. In thinknook.
Xing Niu, Mariannaa Martindale, and Marine Carpuat. 2017. A study of style in machine translation: Con- trolling the formality of machine translation output. In Proceedings of the 2017 Conference on Empiri- cal Methods in Natural Language Processing, pages 2814-2819.
Nils Reimers and Iryna Gurevych. 2019. Sentence-bert, Sentence embeddings using siamese nert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-JJCLP),, pages 3982-3992.
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Car- roll Wainwright, Pamela Mishkin, Chong Zhang. Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow in- structions with human feedback. arXiv preprint arXiv:2203.02155.
Parker Riley, Noah Constant, Mandy Guo, Girish Ku- mar, David C Uthus, and Zarana Parekh. 2021. Textsettr: Few-shot text style extraction and tunable

---

targeted restyling. In Proceedings of the 59th Annual Meeting of the Association for Computational Lin- guistics and the 11th International Joint Conference on Natural Language Processing (Volume I: Long Pupers), pages 3786-3800.
Alex Warstadt, Alicia Parrish, Haokun Liu, Anhad Mo- hananey, Wei Peng, Sheng-Fu Wang, and Samuel R. Bowman. 2020. BLiMP: The benchmark of linguis- tic minimal pairs for English. Transactions of the Association for Computational Linguistics, 8:377- 392.
Rafael A Rivera- Olivia Elizabeth Miano, Juanita Ordonez, Barry Y Chen, Aleem Khan, Marcus Bishop, and Nicholas Andrews. 2021. Learning uni- versal authorship representations. In Proceedings of the 2021 Conference Empirical Methods in Natu- ral Language Processing, pages 913-919.
Alex Warstadt, Amanpreet Singh, and Samuel R. Bow- man. 2019. Neural network acceptability judgments. Transactions of the Association for Computational Linguistics, 7:625-641.
Anna Wegmann and Dong Nguyen. 2021. Does it cap- ture stel? modular, similarity based linguistic style evaluation framework. In Proceedings of the 2021 Conference Empirical Methods in Natural Lan- guage Processing, pages 7109-7130.
Paolo Rosso, Francisco Rangel, Martin Pouthast Elfs- tathios Stamatatos, Michael Tschuggnall, and Benno Stein. 2016. Overview of pan' 16: new challenges for authorship analysis: cross-genre profiling, clus- tering, diarization,, and obfuscation. In Experimental IR Meets Multilinguality, Multimodality, and Inter- action: 7th International Conference of the CLEF Association, CLEF 2016, Évora, Portugal, September 5-8, 2016, Proceedings 7, pages 332-350. Springer.
Anna Wegmann, Marijn Schraagen, Dong Nguyen, et al. 2022. Same author or just same topic? towards content-independent style representations. In Pro- ceedings of the 7th Workshop on Representation Learning for NLP, page 249. Association for Compu- tational Linguistics.
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. ArXiv, abs/1910.01108.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier- ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, and Jamie Brew. 2019. Huggingface's transformers: State-of oh-the-art natural language processing. CoRR. 0tbs/191000777
Elvis Saravia, Hsien-Chi Toby Liu, Yen-Hao Huang, Junlin Wu, and Yi-Shin Chen. 2018. CARER: Con- textualized affect representations for emotion recog- nition. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3687-3697, Brussels, Belgium. Association for Computational Linguistics.
Wei Xu. 2017. From shakespeare to Twitter: What are language styles all about? In Proceedings of the Workshop on Stylistic Variation, pages 1-9, Copen- hagen, Denmark. Association for Computational Lin- guistics.
Florian Schroff, Dmitry Kalenichenko, and James Philbin. 2015. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 815-823.
Xiaoyuan Yi, Zhenghao Liu, Wenhao Li, and Maosong Sun. 2021. Text style transfer via learning style in- stance supported latent space. In Proceedings of the Twenty-Ninth International Conference on Interna- tional Joint Conferences on Artificial Intelligence, pages 3801-3807.
Tianxiao Shen, Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2017. Style transfer from non-parallel text by cross-alignment. Advances in neural information processing systems, 30.
Xiang Zhang. Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text clas- sification. In Advances in Neural Information Pro- cessing Systems, volume 28, Curran Associates, Inc
Efstathios Stamatatos. 2009. A survey of modern au- thorship attribution methods. Journal of the Ameri- can Society for Information Science and Technology, 60(3):538-55
Jian Zhu and David Jurgens. 2021. Idiosyncratic but not arbitrary: Learning idiolects in online registers reveals distinctive yet consistent individual styles. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 279- 297.
Yla R Tausczik and James W Pennebaker. 2010. The psychological meaning of words: Liwe and comput- erized text analysis methods. Journal of language and social psychology, 29(1):24554
Peter Tiersma and Lawrence M. Solan. 2002. The lin- guist on the witness stand: Forensic linguistics in american courts. Language, 78(2):221-239.
Kangchen Zhu, Zhiliang Tian, Ruifeng Luo, and Xi- aoguang Mao. 2022. Styleflow: Disentangle la- tent representations via normalizing flow for un- supervised text style transfer. arXiv preprint 27X.v.2212.09670.
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Al- isa Liu, Noah A Smith, Daniel Khashabi, and Han- nanch Hajishirzi. 2022. Self-instruct: Aligning lan- guage model with self generated instructions. arXiv preprint 221 10560.

---

A Prompt Templates
A.1 Open-ended Prompt Templates
Grammar Style: write long paragraph describing the unique grammar style of the following passage without referring specifics about the topic.
Passage:
((passage)
Description:
Vocabulary Style: Write a leng paragraph describing the unique vocabulary style the following passage without referring to specifics about the topic.
Passage:
((passage))
Description:
Punctuation Style: Write long paragraph deseribing the unique punctuation style of the Tol lowing passage without reterring to spect fics about the topic.
Passage: ((passage))
Description:
Grammar Errors: Write a long paragrapa describing the grammar errors (if any) of the following passage without referring to specifics about the tepic.
Passage:
[[passage))
Description:
Spelling Errors: write a long paragraph describing the spelling errors (if any) of the following passage without referring to specifics about the tepic.
Passage:
((passage)
Description:
Forensic Linguist: Write long paragraph describing the unique stylemetric features of tollowing passage without reterring spect about the topic from the perspective forensic linguist psychoanalyzing the writer.
Passage:
([passage)
Description:

---

A.2 Targeted Prompt Templates
We use the following template:
Write description whether the author of the following passage has any ([target_toature]]
Passage: ((passage)
Description:
for targeted prompts, substituting Ctaaget_feeturee with each of the following targeted fea-
tures:
figurative language
negative tone
words related to polities
sarcasm
neutral tone
words related to culture
sentence fragment
words related auditory percep- tion
words
run on sentences
foreign words
an active voice
words related to visual perception
scholarly words
passive voice
words related space perception
slang words
agreement errors
words related motion percep- tion
social media slang words
male pronouns
filler words
female pronouns
words related attention
words focusing on the past
prosocial behaviors
words related allure
words focusing the present
antisocial behaviors
words related to curiosity
words focusing on the future
being polite
words related risk
words related time
showing interpersonal conflict moralizing
words related to reward
misspelled words
words expressing needs
repeated words
communication words
words expressing wants
words expressing quantity
indicators of power
words expressing acquisition
words indicating family
talk of achievement
words expressing lack
words indicating friends
words expressing fulfillment
indication of certitude
words indicating men
words expressing fatigue
being tentative
words indicating women
insight
words expressing illness
words indicating pets
all or none thinking
words expressing wellness
words indicating social status
words related to memory
words related to mental health
words indicating poverty
positive emotion
words related food or cating
words indicating wealth
words related death
negative emotion
punctuation symbols
anxicty
words related to self-harm
hyphenated words
anger
sexual content
oxford comma
sadness
words related to leisure
parentheticals
swear words
words related home
numbers
positive tone
words related to work
clongated words
words related money
words related religion
To give GPT-3 more context, we also substitute ( the target feature, also generated by GPT-3, The full set of targeted prompts can be found in the released source package for this paper.
with a definition

---

A.3 Standardization Prompt Templates
The descriptions of style generated from the prompts in Appendix A.I and Appendix A.2 are substituted into the following standardization prompt:
Here's 8 description of an author's writing style for passage: (description)
Rewrite this description long 1ist of short sentences describing the author's writing style where each sentence is in the format of "The author X. or "The author uses x.".
Output:
which transforms the verbose descriptions into short, declarative, uniform sentences beginning with "The author..."" which are the final style attributes used in building the STYLEGENOME dataset that SFAM is trained on.
B Effect of STYLEGENOME Dataset Size
When training SFAM, we experiment with artificially limiting the size of the synthetic dataset, by limiting the number of authors in the dataset, to determine the effect of dataset size on the validation performance. Figure 2, we find that as the synthetic dataset grows, validation performance improves and SFAM generalizes to better predict agreement scores for unseen style attribute and text pairs.
0.9
Validation Fl
0.7
0.6
10'
10a
10
Number of Authors
Figure Best F1 achieved by SFAM on held-out validation set of examples at various dataset sizes.
C Annotation Prompts Ablation
Prompts Used to Generate STYLEGENOME
Validation F1
Open-ended Prompts
0.865
Targeted Prompts
0.898
Open-ended Prompts & Targeted Prompts
0.920
Table 7: Best Fl achieved by SFAM held-out validation set of examples with different sets of Stage prompts used to annotate Reddit posts and generate the synthetic training data used during distillation.
D STYLEGENMEE Annotation Cost
Our inference cost with the OpenAl APT was priced at $0.02/ / 1K tokens, with a cost of ~$8 annotate 10 Reddit posts by a single author with all our prompts. Our full dataset of 1,000 authors cost $880000 to annotate.

---

E Training Details
E.1
SFAM
We use the EncT5 architecture (Liu et al., 2021) with binary classifier in Hugging Face (Wolf et al., 2019). We randomly sample training batches of size 1,440 and use the AdamW optimizer with learning rate of 0.001. We employ early stopping with threshold of 0.01 the validation set Fl metric and patience of 50 batches.
E.2 LISA
We the EncT5 architecture (Liu et al., 2021) with 768 regression labels in Hugging Face (Wolf al., 2019) and use MSELoss. We randomly sample training batches of size 1,440 and use the A optimizer with a learning rate of 0.001. We employ carly stopping with threshold le-6 on the validation set MSE metric and a patience of 20 epochs.
E.3 LISA Embedding Layers
We experiment with two types of embedding layers w and W. We also experiment with two training datasets, the Wegmann dataset (Wegmann et al., 2022) and the LUAR dataset (Rivera-Sote et al., 2021). For LUAR, use the train split with 5% held out as validation and we sample random authors as negative examples. For Wegmann, we the Conversation dataset train split and the dev split as validation. We use a margin of 1.0, batch size of 32, an AdamW optimizer with a learning rate of 0,001, and employ early stopping with threshold of 0.001 on the validation set loss.

---

F
Full SFAM Evaluation Results
Spcarman
Type
Dataset
Style Attribute
Correlation (p)
Formality
Grammarty's Yahoo Answers Formality Corpus
Formality in Online Communication
The author uses informal language.
0.200
0.599
Yelp Reviews
0.788
IMDB Large Movie Review Dataset
0.665
Sentiment
Amuzon Customer
Datuset
The author uses negative tone.
Rotten Tomaroes Movie Review Data
0,463
App Reviews
0.350
Twitter Sentiment Analysis Training Corpus
0.299
DAIR. Al Emotion (I Love Anger)
0.542
Emotion (Jay vs. Sad)
0.531
GoEmotions (Love vs. Anger)
0.769
GoEmotons (Joy Sadness)
0.639
Emotion
GoEmotions (Disgust Desire)
0.630
GoEmotions (Disappoiatmentt vs. Admiration)
0.571
GoEmoties (Pride vs. Embarassment)
0.419
GoEmotioos (Nervausness Optimism)
0.447
(Disappovaa Approval)
0.432
GoEmotions (Admiration vs. Neutral)
0.578
GoFmintions (Amusement Neutral)
0.508
GoEmotions (Anger Neutral)
0.372
(Annoyance Neutral)
0.355
GoEmotioos (Approval vs, Neutral)
0, 0,257
GoEmotions (Caring Neutral)
0.303
GoEmotions Coaftusoo
Noutral)
0.343
GoEmotions CCriisityy vs, Neutral)
0.464
GoEmotions (Desire vs. Neutral)
0.315
GoEmotion Disappoiatment
Neutral)
The author expressing [[mmotion]]
0.317
(Disapproval Neutral)
0.284
GoEmotions (Disgust vs. Neutral)
0.307
GoEmotions (émbarrassment vs. Neutral)
0.173
GoEmmttios [Excitement Neutral)
0.295
Neutral)
0.309
GoEmotions (Gratitude vs. Neutral)
0.626
GoFmitiios (Grief vs. Neutral)
0.093
GoEmotions (Joy Neutra))
0.390
GoEmotions (Love vs. Neutral)
0.497
GoEmotions (Nervousness vs, Neutral)
0.171
GoEmotions (Optimism Neutral)
0.407
GoEmotions (Pride Neutral)
0.097
GoEmotions (Realization vs. Neutral)
0.121
GoEmotions (Relief vs. Neutral)
0.123
GoEmotions (Remorse Neutral)
0.276
(Sadness vs. Neutral)
0.413
GoFrotioon (Surprise vs. Neutral)
0.317
Political Slant
The author a Democrat.
0.005
Author Profiling
Twitter User Gender Classification
The author is female.
0.166
The author
African American Vernac-
Vernacular English
English.
0.238
Shakespeare (Barly Modern English)
The author uses Early Modem English.
0. 1,108
Wikipedia Bias
The author has biased point of view.
0.014
Harmful
HateSpech18
The author's 's writing contains hate speech.
0.229
Speech
Offensive Social Media
The author offensive language.
0.401
Simplification
Text
Simple Wikipedia
ASSET
The author simple language.
0.050
0.043
Linguistic
CoLA
The uses grammarr.
0.078
Acceptability
BLiMP
0.020
Average
0.342
Table Correlation agreement scores produced by SFAM against human judgments on texts over wide variety linguistic and authorship dimensions. The natural language style attributes used input SFAM when producing the agreement scores for each dataset are also provided.

---

G
Interpretable Authorship Verification
Texts
Top 3 Common/Distinet Style Attributes
Anchor: Devices that use two pronged instead of three pronged plugs are required to meet certain safe design re- quirements. Among other things, if a device has switch, the switched line MUST BE hot, not neutral. The polarized plugs make sure that the right prong/wire is hot. This is why devices that have no switches (primarily wall warts) need not have polarized plugs.
Same Author: Your diaphragm would be trying to contract against the air pressure in your lungs. That's 's why deep sea diving requires regulators, to match the pressure of the air supply to the pressure surrounding your rib cage. You can hreathe against a maximum of ahout 1/2 PSI, which is not enough pressure to udequately oxygenate your blood.
(0.89, 1.00)
The author is using a scientific approach.
(0.96, 0.98)
The author is using combination of
technical terms and everyday language.
(0.91, 0.84)
The author is using formal and
professional language.
Different Author: That's great! I'm glad seems to be finding its' niche. Now if they could just make Star Wars version of this game, I'd happily swallow that fat learning curve and overcome my frustrations with the combat system.
(0.06, 0.99)
The author is using words related to the
game they are discussing.
(0.00, 0.88)
The author is using an emoji.
(0.02, 0.87)
The author uses an emoticon at the end.
Not sure what the income tax is in Germany, but in the Netherlands the income can be up 50% for the higher income classes.
Anchor:
Same Author: The salarics in the US alway hlow my mind. A software developer in Amsterdam gets like y40.0000yeaa, maybe E5500000year if your good, and mayhe 660.0000ycaa
(0.84, 0.90)
The author is using words indicating poverty.
(0.87, 1.00)
The author is using words indicating wealth.
if you're some kind of manager. Anything position over C100.000/year is basically running the entire company.
(0.80, 0.93)
The author is using words related to money.
(0.00, 1.00)
The author is comfortable with technology.
Different Author: How would you even test this software? The setup would he just insane.
(0.00, 0.85)
The author is discussing a product. *
(0.13, 0.92)
The author is using formal and
professional language.
Anchor: If only there was something he could have done avnid this backlash. Like mayhe not acting like complete d**khead.
Same Author: I take issue with a faster landing being marked as less skilled. By that logic the slowest, smoothest possible landing would be the most skilled and that plain wrong, Maybe war machine intentionally does faster and harder landings.
(1.00, 0.96)
The author is emphasizing the contrast
between the two ideas.
(0.78, 0.89)
The author is able to draw conclusions.
(0.97, 0.86)
The author is using an all-or-none thinking style.
(0.83, 0.05)
The author is describing sexual content.
Different Author: She was the Ronald Reagan of the UK in the same time period.
(0.38, 0.94) - The author is using words related to politics.
(0.74, 0.38) The author is using parentheticals.
Table 9: Example interpretable explanations on the Contrastive Authorship Verification task. The top style attributes in common between the Anchor text and a text by the Same Author are shown. The top distinct style attributes between the Anchor text and a text by a Different Author are also shown. The scores of each style attribute against the texts is shown in (t, s/-). We manually inspect the style attributes and annotate them as reasonable, plausible, or incorrect explanations. Attributes annotated with blur the line between style and content. Error analysis can be found in Section 4.1.
We perform this task with LISA (LUAR W) and demonstrate interpretability on a few task instances. To rank the top common or distinct style attributes between two style vectors यî and oz we perform a simple calculation. We first calculate the contribution of each dimension d to the Euclidean distance as a measure of the general importance of each dimension. The importance score is defined as:
D
(UiR vix)²
+ d
[(d)- = - - -ill2
d

---

To retrieve the top common style attributes, we rank the dimensions, and the corresponding style attributes. in descending order by the following score function:
I(d) * Fid 02d
ScOREcoomon (d)
I(k)
k=0
To retrieve the top distinct style attributes, rank the dimensions, and the corresponding style attributes, in descending order by the following score function:
I(d)
SCOR Fedstinct (d)
max via)) * (1.0- miin v2a))
D
Σ I(k)
k=0

---

H Resources
We provide links and citations to resources used in this paper which provide license information, docu- mentation, and their intended use. Our usage follows the intended usage of all resources.
We utilize the following models:
GsT (text -eest-ddvvini---e 30011nnni--0)))))) (Brown et al.. 2020; Ouyang et al., 2022)
EncT55 (t5-base) (Devlin et al., 2019; Liu al., 2019)
DistilRoBERTa Gurevych, 2019)
base-v2) (Sanh et al., 2019; al., 2019: Devlin et al., 2019; Reimers
Learning Universal Authorship Representations (LUAR) Embedding model (Rivera-Soto al., 2021)
Style embedding model from Wegmann et al. (2022)
We utilize the following datasets:
Reddit Million User Dataset (Khan et al., 2021; Andrews and Bishop, 2019)
STEL dataset (Wegmann Nguyen, 2021)
Contrastive Authorship Verification dataset (Wegmann et al., 2022)
Formality in Online Communication (Pavlick and Tetreault, 2016)
Grammarly's Yahoo Answers Formality Corpus (Rao and Tetreault, 2018)
Yelp Reviews Dataset (Zhang al., 2015)
IMDB Large Movie Review Dataset (Maas al., 2011)
Amazon Customer Reviews Dataset (Amazon.com, 2018) https://s3 atw.ao/www.ccmmmmmooooorrrririea
pds/ hols/ran /ee/dee.ee.... html
Rotten Tomatoes Movie Review Data (Pang and Lee, 2005)
App Reviews Dataset (Grano al., 2017)
Twitter Sentiment Analysis Training Corpus (Naji, 2012)
DAIR.AI Emotion Dataset (Saravia et al., 2018)
GoEmotions Dataset (Demszky et al., 2020)
Political Slant Dataset (Prabhumoye et al., 2018)
Twitter User Gender Classification Datasct (CrowdFlower, 2017) hotps: httppp:://wwwkaaae cot.//e..cmm/maeeettttee wwwflowerr/wwtter--eeee wwweereedddddereeeee www.derr-oaassiicaaiiio..
African-American Verncuuaar English Dataset (Groenwold et al., 2020)
Shakespeare Dataset (Xu, 2017)
Wikipedia Rias Datasct (Pryzant al., 2020)
HateSpeech18 (de Gibert al., 2018)
Offensive Social Media Dataset (Atwell et al., 2022)
Simple Wikipedia Dataset (Coster and Kauchak, 201
ASSET (Alva-Manchego al., 2020)
CoLAA (Warstadt al., 2019)
BLiMP (Wanstadt et al., 2019)

---

We utilize the following software:
Transformers (Wolf al., 2019)
Centence-Transformme (Reimers and Gurevych, 2019)
emoji - httppp///pppp..oooppppctt/aaaaet
sentence-splitter
We estimate the total compute budget and detail computing infrastructure used run the computational experiments found in this paper below:
Ix NVIDIA RTX A 30GB RAM/ CPU 230 hours


Reviews:
[{"review_id": "b3t7Kb8ivR", "reviewer": "Reviewer_9v8q", "summary": "", "questions": "- How similar are the styles of texts written by the same user? Though you show some sample reddit posts, are their styles consist across different posts?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their comments. Below we respond to some of the \"Reasons to Reject\", however, we find the critiques in this review are misaligned with the actual content of the paper / factually incorrect and not in alignment with other reviewers. We provide the reasoning below and hope the review is re-considered in this light.\n\n**Reason 1:** The review states that our model correlates well with styles that are more semantics-relevant vs. linguistics-relevant, however, this dichotomy by the review doesn't make sense. Semantics is a *part of* linguistics; linguistics includes semantics [1]. If our model is capturing semantics, it is, by definition, capturing linguistic properties. Moreover, the review asserts \"LISA is very likely to be content-aware\" with no evidence and is directly contrary to our experimental results. We discuss and cite a number of works that find that content and style are inherently entangled and cannot be cleanly separated in many cases (Line 389, [2]). Therefore, *all* style representations will capture *some* content, even the prior style embeddings work by (Wegmann 2022) we compare to. How much style vs. content a model captures will inevitably fall on a continuous spectrum. We never claim otherwise and state it upfront (Line 389). In our discussion, we find that this percentage is reasonably low (3% of dimensions) (Line 389).\n\n**Reason 2:** We are not trying to avoid comparisons with content-aware representations and no other review has brought this as an issue. This paper is explicitly about training style representations that are content-independent. There is an entire discussion about this in the paper (Line 349). Content-aware representations like SBERT and LUAR are not what we are training in this paper; we are not trying to create a general-purpose sentence embedding model like SBERT. SBERT will produce a high cosine similarity for two sentences with different styles but similar content by the nature of its training, the exact opposite of the objective of this paper. Representations trained to specifically maximize capturing style and minimize capturing content, like LIWC and the style embeddings from (Wegmann 2022) are the appropriate baselines to compare against.\n\n**Unclear:** This review contains ungrammatical sentences like \"This could also evidence the interpretability.\", making it impossible for us to respond to these portions, could you clarify what you mean?\n\n[1] https://linguistics.ucla.edu/undergraduate/what-is-linguistics/\n\n[2] Style versus Content: A distinction without a (learnable) difference?: https://aclanthology.org/2020.coling-main.197/", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- They introduce StyleGenome, a large-scale dataset containing 10000 reddit posts paired with all possible style attributes annotated by GPT-3.\n- Their proposed LISA embedding is able to outperform existing style representations on some styles under the STEL framework.", "reasons_to_reject": "- Despite correlating well with human on styles that are more semantics-relevant (e.g., sentiment, emotion), their SFAM scorers which serve as fundamental components of the LISA embedding do not correlate well with human on more linguistics-relevant styles (e.g., simplification, linguistic acceptability). This probably makes the LISA more content-focused, rather than style-focused.\n- The authors try to avoid comparisons with content-aware representations, claiming those representations have direct access to the text and focus more on content (Line 349). However, as mentioned above, LISA is very likely to be content-aware. One thing the authors can do is checking what dimensions are contributing to the similarity measure. This could also evidence the interpretability. It is currently unclear whether the formality (or other style) dimension in LISA is really assisting finding texts that are similarly formal (or other style).", "questions_to_authors": "- How similar are the styles of texts written by the same user? Though you show some sample reddit posts, are their styles consist across different posts?", "paper_topic_and_main_contributions": "This paper aims to design an interpretable text representation (named LISA) that describes the style. To sum up, each dimension in their proposed representation vector corresponds to a style, where the value is obtained by running a style scorer (named SFAM). To select styles embedded by the representation and train the scorers, they prompt GPT-3 to describe and consolidate the styles in the MUD dataset that consists of Reddit posts from different users (StyleGenome). In their experiments, the authors find that their SFAM scorers are able to correlate well with human on some styles and the LISA embedding can perform comparably well with existing style embeddings on finding texts with similar styles (STEL evaluation framework). The contributions of this paper are:\n- They design a framework for obtaining style annotations from LLMs, producing the StyleGenome dataset.\n- They propose LISA, a text representation that measures intensity of different styles, thus interpretable.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "qUyP5jppTM", "reviewer": "Reviewer_UhN1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the comments.\n\nThe review is correct that we find high correlation with tasks like sentiment, and we would like to highlight that we also find medium correlation with tasks like emotion detection, harmful language, and author profiling, all of which can contribute to writing style to some degree. In some of our examples, we demonstrate capturing some fairly interesting niche elements of writing style like \"*The author is being polite.*\" or \"*The author is using a personal anecdote to illustrate their point.*\", but there aren't many large datasets available for evaluating these kinds of fine-grained stylistic features so they are under-represented in our SFAM evaluation in Table 4. Through manual human annotation, however, we found that our overall error rate (18%) (Line 408,415,422) is fairly low. We believe this model does capture various linguistic properties and would be useful to the community for broad automated stylistic analysis, for which there are not many tools today.\n\nFor reproducibility of the \"Correlation to Human Judgements\" experiments, we have submitted a code package with this submission, however, files too large to upload (datasets & model checkpoints), will be released with the final version of the paper on GitHub / HuggingFace Hub. We have, however, anonymously uploaded the code & data for this experiment here: [https://anonymous-drive.s3.amazonaws.com/sfam_eval.zip](https://anonymous-drive.s3.amazonaws.com/sfam_eval.zip) that should help alleviate any reproducibility concerns around these experiments.\n\nOverall, as this reviewer stated, we believe that our trained model is interesting to the community for wide-ranging stylometric analysis of text corpora and it would be one of the first models available to do so. This track is about \"Stylistic Analysis\" and we believe it's one of the first models that is capable of doing broader stylometric analysis over many dimensions of style.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The examples of style attribute provided as examples seem indeed interesting for stylometric analysis of text corpora. \nThe method correlates well with human judgement for sentiment/emotion.\nThe authors propose to share de dataset and the model that output the style embedding. This would largely benefit the community\nThe experiments are convincing and appropriate/comprehensive.\nThe method is relatively new (even if quite incremental)\n", "reasons_to_reject": "The output of SFAM does not correlate with human judgement on pure linguistic (which is what author are trying to achieve here, not building another sentiment analysis model).\nSome details are missing to reproduce, e.g. for the experiment on Correlation to Human Judgement. \n", "questions_to_authors": "", "paper_topic_and_main_contributions": "In this article, authors propose a three stages approach to build a model that outputs a stylometric representation of a piece of text in a single forward pass. To do that, 1) they build a training dataset by generating stylistic description of texts with a decoder LM (GPT-3) 2) they use these to train a model to match stylistic attribute with the text (in a contrastive manner) 3) by downsampling the number of attributes with different heuristics, they train a regression model on the second model output probability. As attribute are Natural language description of style, the obtained dimensions are fully interpretable. The approach performs well on the STEL benchmark compared to prior work and correlates partially with human judgement ", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "N9laVtL7xl", "reviewer": "Reviewer_cLTM", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their comments. \n\nWe respond to some of the main and minor concerns below and believe they will be adequately addressed in the camera-ready. We believe they are fully addressed here as well, but if they are not fully addressed, we are happy to respond further:\n\n1. **StyleGenome:** We don't directly analyze StyleGenome as it is very large (millions of annotations by GPT-3). As you mentioned, however, we provide some evidence the dataset provides signal by evaluating a downstream model trained using the dataset. By design, this is a synthetic dataset, so we don't claim it is 100% correct and clearly state the limitations of an unsupervised approach (Line 430-431) and provide error analysis, however, we release it following other papers that have recently released synthetic datasets for reproducibility (in case others want to train models similar to our SFAM/LISA models using this dataset). It was also fairly expensive to produce (~$8,000 - Line 950), so we believe it could be a useful reusable resource to the community that is lacking these kinds of resources around text style annotations.\n\n2. **Pipeline Decisions:** Many of these decisions such as prompt design were manually made at the time of producing the dataset. Decisions like filtering were simply made to get to an embedding of D=768 dimensions (an arbitrary choice as we state, but is the traditional choice of the # of dimensions with BERT models). Evaluations on SFAM / LISA were performed only at the end after training. We can make a small revision in our camera-ready to make these minor manual choices more clear.\n\n3. **Near-Duplicate Dimensions:** That's correct, there may be some near-duplicates in the dimensions. However, the main reason we filter out near-duplicates and less discriminative features is to reduce our feature space from ~ 1.3M style attributes to a more manageable 768 dimensions, since a 1.3M dimensional vector would be too unwieldy. Further, human de-duplication is difficult as it requires approx. 768 choose 2 (~300K) pairwise inspections, so we rely solely on the automated near-duplicate detection using SBERT. In the camera-ready draft, however, we can mention the fact that a small number of near-duplicates may remain even after SBERT-filtering.\n\nWe also briefly respond to your minor concerns:\n\n1. **TF-IDF**: We do actually use a TF-IDF library to this, this is evident in our code package we submitted, and we treat \"style attributes = terms\" and \"authors = documents\" using it to select attributes that are most discriminative between authors similar to how TF-IDF will upweight terms most important to a document. We allude to this in the paper when we discuss selecting discriminative style attributes by using how often they appear for each author (or \"document frequency\"), but we can make this more clear in the camera-ready along with our code package.\n\n2. **Presentation of Features:** Thanks for the suggestion, that's helpful, we'll definitely take this into consideration when rephrasing for camera-ready.\n\n3. **Million User Dataset citation:** You're correct, our citation is only half-correct. Thank you for pointing this out. Recent guidance from the authors of that dataset after we submitted this draft states the full citation should be two citations: (Andrews and Bishop 2019; [Khan et al. 2021](https://arxiv.org/pdf/2105.07263.pdf)). We will update this citation in the camera-ready as well.\n\n4.  **Open-source LLMs:** We briefly experimented with open source LLMs for this task early on in our research, however, at the time this research began, many of the current instruction-tuned models (LLaMa-2-chat) were not available. It's possible they are also performant for this task, however, we release our synthetic dataset (StyleGenome) and our own distilled, efficient, smaller open source models based on T5 (SFAM & LISA) as part of this work, so there should be no further dependence to closed source models.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* Creative use of LLMs to produce a new dataset with style attribute annotations.\n* The distillation framework appears to successful transfer knowledge from GPT-3 into a significantly smaller / more efficient model.\n* Evaluation comparing to both human judgements as well as other recent style embedding approaches, including error analysis\n* Release of dataset and models", "reasons_to_reject": "Main concerns:\n* I would have liked to see more investigation of the quality of the StyleGenome dataset. For example, although Appendix B provides some evidence that there's signal there, the trend could be more complicated when looking at generalization for individual attributes, and there is the risk that essentially noise/hallucinations from the LLM are being fit.\n\n* It's unclear what feedback was used to guide the various manual decisions that went in to the pipeline (e.g., prompt design, normalization, filtering). What this all done on the basis of some validation data or manual inspection?\n\n* It's unclear how distinct the various stylistic attributes are since they're based on natural language descriptions, and SBERT similarity won't identify all cases of of near duplicate attributes. Perhaps some kind of dimensionality reduction could help.\n\nMinor concerns:\n\n* For selecting the attributes to keep, is there a more standard / less heuristic approach, such as TFIDF, that could be used?\n* I found it pretty difficult to wrap my head around the mapping from an open-set of natural language attributes to a fixed and closed set of features. The presentation could likely be improved to make this easier.\n* The paper cited for the \"Million User Dataset\" does not appear to describe this dataset.\n* I would have liked to see other versions of StyleGenome compared for different LLMs, especially open-source ones.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper describes a data augmentation framework to learn interpretable style embeddings. To create synthetic training data, GPT-3 is prompted to produce natural language descriptions of the writing style of a given document, using a hand-crafted set of 87 prompts based on a survey of stylometry literature. An additional 6 open-ended prompts aim to elicit a general description of the writing style. The resulting natural language descriptions are post-processed into separate clauses each starting with \"The author [...]\" (with *yet another* LLM prompt), each of which is treated as a separate interpretable stylistic attribute. The resulting synthetic dataset is called \"StyleGenome.\"\n\nTo make use of StyleGenome, \"LISA Style Vectors\" are created based on an arbitrary choice of D = 768 dimensions. The first 87 dimensions correspond to the features of targeted prompts. The remaining 681 are downselected from the ~1.3M style attributes using filtering heuristics: appear for at least 10 authors but no more than 600 authors. Also, once an attribute is selected, duplicates are avoided using SBERT cosine similarity. Also, attributes are rejected if they are too long, contain specifics like \"mentions\", and negations. Finally, an EncT5 model is trained with a binary classifier head to produce predictions efficiently on held-out data, where natural language descriptions allow for generalization across various natural language stylistic descriptors.\n\nThe resulting features are evaluated using an evaluation framework from Wegmann and Nguyen (2021) which aims to characterize style  discrimination ability for categories such as formal and complex. The proposed LISA embeddings are found to be competitive with recent alternatives while having superior interpretability.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper studies the task of learning interpretable style embeddings. To achieve this, the paper presents a synthetic data generation protocol from GPT-3 (using hand-crafted prompts) which is then used to train style representations. The style embeddings are shown to be competitive with recent approaches, while having a better interpretability. The use of LLM in the context of this goal is interesting and creative. I do share very similar concerns raised by cLTM, as the quality of the StyleGenome could have gone through a more rigorous evaluation, but the resource (to my opinion as the main contribution of this work) as-is has merits and potential for use by the community.
Decision: Accept-Findings