Patent Publication Number: US-2023137209-A1

Title: Counterfactual Text Stylization

Description:
BACKGROUND 
     In textual communications, words, syntax, punctuation, and the like significantly affect how a reader interprets an author&#39;s message. For instance, by using different punctuation without changing words or syntax, an author can convey starkly different emotions (e.g., “sure!” vs. “sure . . . ”). Style aspects of text such as sentiment, formality, tone, excitement, and so forth are important considerations for an author when conveying a message to a particular audience, as a style for communicating with close friends is often inappropriate for professional communications. For entities that require substantial amounts of text to be authored in varying styles for different audiences, it becomes increasingly difficult to ensure that text is appropriately styled for a particular audience, and a success is entirely dependent on an individual author&#39;s ability to understand and replicate the appropriate style for the particular audience. 
     Conventional approaches rely on experienced copywriters to manually author messages that convey the same message content in different styles for different target audiences. However, this manual styling is intractable for diverse audiences. Some conventional approaches attempt to provide automated generation of stylized text. However, these conventional automated approaches provide only a coarse control over style applied to text and do not allow for fine-grained control of style changes. Consequently, conventional approaches are unable to scale to generate stylized text for diverse audiences and remain susceptible to human error. 
     SUMMARY 
     A text style transfer system is described that generates a plurality of different stylized versions of input text by rewriting the input text according to a target style. Examples of a target style used for generating the stylized versions of the input text include a positive sentiment, a negative sentiment, formal, informal, excited, indifferent, and so forth. To do so, the text style transfer system receives input text and employs a variational autoencoder to derive separate content and style representations for the input text, where the content representation specifies semantic information conveyed by the input text and the style representation specifies one or more style attributes expressed by the input text. To ensure that the style and content representations remain disentangled from one another, the text style transfer system guides output of the variational autoencoder using variational autoencoder loss and a combination of multitask and adversarial losses for each of style and content latent space representations. 
     The style representation generated by the variational autoencoder is then processed by a multi-layer perceptron classifier trained using counterfactual generation loss to identify different transfer strengths for applying the target style to the input text. By virtue of using counterfactual reasoning, each of the different transfer strengths represents a minimum change to the input text that achieves a different expression of the target style. The transfer strengths are then used to generate style representation variants, which in turn are each concatenated with the content representation of the input text to generate the plurality of different stylized versions of the input text. The text style transfer system is configured to output controls that enable specification of specific transfer strengths to be used for applying various target style attributes when rewriting the input text. 
     This Summary introduces a selection of concepts in a simplified form that are further described below in the Detailed Description. As such, this Summary is not intended to identify essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The detailed description is described with reference to the accompanying figures. In some implementations, entities represented in the figures are indicative of one or more entities and thus reference is made interchangeably to single or plural forms of the entities in the description. 
         FIG.  1    is an illustration of a digital medium environment in an example implementation that is operable to employ a text style transfer system to generate stylized versions of input text using counterfactual reasoning. 
         FIG.  2    depicts a digital medium environment showing operation of the text style transfer system of  FIG.  1    in greater detail. 
         FIGS.  3 A and  3 B  depict a digital medium environment that includes an example architecture for a transformer-based variational autoencoder implemented by the text style transfer system of  FIG.  1   . 
         FIG.  4    depicts a digital medium environment that includes an example user interface of the text style transfer system of  FIG.  1   . 
         FIG.  5    is a flow diagram depicting a procedure in an example implementation of generating stylized versions of input text using the techniques described herein. 
         FIG.  6    illustrates an example system including various components of an example device to implement the techniques described with reference to  FIGS.  1 - 5   . 
     
    
    
     DETAILED DESCRIPTION 
     Overview 
     Textually conveying information in an appropriate fashion becomes increasingly difficult when conveying the information to diverse audiences, as a style (e.g., tone, formality, sentiment, etc.) expressed by the text may be appropriate for one audience and inappropriate for another audience. When scaled, the sheer number of audience considerations makes manually drafting different variations of textual information intractable. In an effort to automate the creation of text variants that target specific audience considerations (e.g., formal tone for textual workplace communications), conventional approaches provide automated tools for generating text variants. 
     However, these conventional approaches provide tools that support only a single-shot transfer of limited text constructs (e.g., individual sentences) by changing individual words to express a different overall style. These conventional approaches are further limited by often requiring labeled datasets, such as annotations that call out specific style attributes of input text (e.g., information specifying certain words that evoke excitement, certain words that express formality, and so forth) or parallel corpuses of text that convey similar information using different styles (e.g., a first sentence conveying information formally and a second sentence conveying the information informally). 
     These labeled datasets are then used to train a language model under supervised learning, with the objective of the language model identifying aspects of a writing style. However, these conventional approaches are limited in scope to styles for which parallel, labeled datasets are available, and are unable to scale to accommodate variations in style not explicitly expressed by the labeled datasets. As an alternative to these supervised learning approaches, some conventional language modeling approaches define linguistic rules for a designated style and attempt to generate text under constraints specified by the linguistic rules. However, these linguistic rule-based approaches are intractable given the large number of rules required to capture a text style, and are limited in their performance by the level of detail included in the linguistic rules, which are often manually generated (e.g., via user inputs to define the rules). Thus, conventional approaches to text rewriting are unable to provide control over a degree in which a target style attribute is applied to input text when rewriting the input text. 
     Accordingly, a system and techniques for generating different stylized versions of input text are described herein. A text style transfer system receives input text and an indication of a target style to be used for rewriting the input text. Examples of a target style used for generating the stylized versions of the input text include sentiment (e.g., positive to negative sentiments), formality (e.g., formal to informal expressions), excitement (e.g., excited to indifferent expressions), gender tone of voice (e.g., feminine to masculine tone), and so forth. To preserve semantic information conveyed by the input text, the text style transfer system employs a variational autoencoder that generates separate style and content representations for the input text in a latent space. 
     To ensure that the style and content representations are disentangled from one another, such that the style representation does not include content information and the content representation does not include style information, the variational autoencoder is guided using a combination of losses. This combination of losses includes a variational autoencoder loss, a multitask style loss, an adversarial style loss, a multitask content loss, and an adversarial content loss. The resulting content representation thus expresses in latent space semantic information conveyed by the input text while the style representation describes style attributes expressed by the input text in the same latent space. In some implementations, the variational autoencoder is configured using a transformer-based architecture, which ensures strong content preservation and disentanglement from style, consequently enabling strong control over strengths at which a target style is applied to the input text. 
     The text style transfer system then processes the style representation generated by the variational autoencoder with a multi-layer perceptron classifier trained using counterfactual generation loss to identify different transfer strengths useable for applying the target style to the input text. By virtue of using counterfactual reasoning, each of the different transfer strengths represents a minimum change to the input text that achieves a different expression of the target style. In this manner, the transfer strengths identified by the text style transfer system are useable to provide fine-grained control of generating text variants for specific target style attributes. The transfer strengths are used to apply the target style to the style representation, which results in the generation of different style representation variants for the input text. The style representation variants are then each concatenated with the content representation of the input text to generate a plurality of different stylized versions of the input text. 
     In this manner, given an indication of a target style (e.g., formality), the text style transfer system is configured to identify a plurality of transfer strengths that each specify a magnitude of the target style relative to a direction (e.g., a magnitude relative to a formal expression or informal expression). From the plurality of formality transfer strengths, the text style transfer system generates a plurality of stylized versions of input text, where each of the stylized versions expresses a different degree of formality while preserving semantic information conveyed by the input text. The text style transfer system is configured to output controls that enable specification of transfer strengths to be used for applying various target style attributes when rewriting the input text. 
     Thus, the system and techniques described herein provide fine-grained control over generation of text variants for one or more target style attributes. Further discussion of these and other examples is included in the following sections and shown in corresponding figures. 
     In the following discussion, an example environment is described that is configured to employ the techniques described herein. Example procedures are also described that are configured for performance in the example environment as well as other environments. Consequently, performance of the example procedures is not limited to the example environment and the example environment is not limited to performance of the example procedures. 
     Example Environment 
       FIG.  1    is an illustration of a digital medium environment  100  in an example implementation that is operable to employ techniques described herein. As used herein, the term “digital medium environment” refers to the various computing devices and resources utilized to implement the techniques described herein. The digital medium environment  100  includes a computing device  102 , which is configurable in a variety of manners. 
     The computing device  102 , for instance, is configurable as a desktop computer, a laptop computer, a mobile device (e.g., assuming a handheld or wearable configuration such as a tablet, mobile phone, smartwatch, etc.), and so forth. Thus, the computing device  102  ranges from full resource devices with substantial memory and processor resources (e.g., personal computers, game consoles, etc.) to low-resource devices with limited memory and/or processing resources (e.g., mobile devices). Additionally, although a single computing device  102  is shown, the computing device  102  is representative of a plurality of different devices, such as multiple servers utilized to perform operations “over the cloud.” 
     The computing device  102  is illustrated as including a text style transfer system  104 . The text style transfer system  104  is representative of functionality of the computing device  102  to receive an input text  106  and rewrite the input text  106  to generate stylized text  108  that includes a plurality of variants of the input text  106 , modified according to one or more target style attributes. For example, in the illustrated example of  FIG.  1   , the input text  106  represents the sentence “This hotel was the worst I have ever stayed in and felt very unsafe!” The stylized text  108  includes the input text  106  rewritten by applying varying strengths of a “positive sentiment” target attribute to achieve stylized versions  110 ,  112 ,  114 , and  116  of the input text  106 . 
     Contrasted with the input text  106 , the stylized version  110  represents a result generated by applying a first strength of the positive sentiment target attribute when modifying the input text  106  and recites “This hotel was the worst hotel I have ever stayed in and was unsafe.” The stylized version  112  represents a result generated by applying a second strength of the positive sentiment target attribute, which is greater than the first strength used to achieve the stylized version  110 , and recites “This hotel was the worst hotel I have ever stayed in.” The stylized version  114  represents a result generated by applying a third strength of the positive sentiment attribute, which in turn is greater than the second strength used to achieve the stylized version  112 , and recites “This hotel was not the worst hotel I have ever stayed in.” Finally, the stylized version  116  represents a result generated by applying a fourth strength of the positive sentiment attribute, which is greater than the third strength used to generate the stylized version  114 , and recites “This hotel was great and clean!” 
     In implementations, differences between each of the first, second, third, and fourth strengths by which the positive sentiment attribute is applied to the input text  106  to achieve the stylized versions  110 ,  112 ,  114 , and  116  are approximately uniform. In this manner, the text style transfer system  104  is configured to provide precise control in applying target style attributes to input text  106 , thus enabling controlled style transfer rather than a one-shot attempt at binary style changes (e.g., a simple negative sentiment to positive sentiment change). Although illustrated as including four variations of the input text  106 , the text style transfer system  104  is configured to generate the stylized text  108  to include any number of variations of the input text  106  using the techniques described herein. 
     To generate the stylized text  108 , the text style transfer system  104  employs a disentanglement module  118 , a counterfactual reasoning module  120 , and a stylization module  122 . The disentanglement module  118 , the counterfactual reasoning module  120 , and the stylization module  122  are each implemented at least partially in hardware of the computing device  102  (e.g., through use of a processing system and computer-readable storage media), as described in further detail below with respect to  FIG.  6   . 
     The disentanglement module  118  is configured to process the input text  106  and derive a content representation and a style representation for the input text  106 . As described herein, the content representation of the input text  106  includes information describing the semantic meaning of the input text  106 . In contrast, the style representation of the input text  106  includes information describing stylistic attributes of the input text  106 , such as formality, sentiment, tone, excitement level, political inclination, author gender, and so forth. To derive the content and style representations for the input text  106 , the disentanglement module  118  is configured to implement a variational autoencoder model trained to identify style representations for various stylistic attributes, such as the positive sentiment attribute used to generate the stylized versions  110 ,  112 ,  114 , and  116  represented in  FIG.  1   . 
     In some implementations, the variational autoencoder implemented by the disentanglement module  118  is configured as a Recurrent Neural Network (RNN)-based variational autoencoder. Alternatively, in some implementations the variational autoencoder is configured using a transformer-based architecture. To ensure that the latent space in which both the content and style of the input text  106  is represented remains disentangled, the disentanglement module  118  leverages both multitask and adversarial losses for generating each of the style and content representations. 
     The counterfactual reasoning module  120  is configured to determine style transfer strengths that are useable to derive variants of the style representation for the input text  106 . The style transfer strengths are determined in a manner that ensures the resulting variants remain close to the style representation of the input text  106  in the latent space containing the style and content representations. Additionally, the style transfer strengths are computed so that different style transfer strengths create variants of the style representation with minimum differences relative to one another while ensuring that the variants do not convey the same target style attribute. 
     For instance, consider an example scenario where the counterfactual reasoning module  120  computes style transfer strengths for an “informal” target style attribute, such that not applying a style transfer (e.g., a transfer strength of zero) maintains a current formality of the input text  106  and a maximum style transfer strength generates a most informal variant of the input text  106 . In this example scenario, the counterfactual reasoning module  120  is configured to derive the greatest number of style transfer strengths between zero and the maximum transfer strength. As a limit, the counterfactual reasoning module  120  ensures that no two transfer strengths create identical variants of the input text  106  or variants that convey a same degree of formality. 
     To compute the style transfer strengths for a target style attribute, the counterfactual reasoning module  120  leverages a multi-layer perceptron (MLP) classifier trained on disentangled style representations generated by the disentanglement module  118  for a given target style. The MLP classifier is trained using a counterfactual generation loss, which ensures that the style transfer strengths output by the counterfactual reasoning module  120  generate variants of the input text  106  that each convey a different degree of a target style attribute while minimizing distances between the style transfer strengths. 
     The stylization module  122  is configured to generate the stylized text  108  by generating variants of the style representation of the input text  106  using the style transfer strengths generated by the counterfactual reasoning module  120 . The stylization module  122  then combines variants of the style representation with the content representation generated by the disentanglement module  118  to generate stylized versions of the input text  106  (e.g., the stylized versions  110 ,  112 ,  114 , and  116 ). In this manner, the text style transfer system  104  ensures that a semantic meaning of the input text  106  is preserved while generating the stylized text  108 . In some implementations, the stylization module  122  outputs each stylized version of the stylized text  108  together with an indication of the target style attribute used to generated the stylized version and a corresponding style transfer strength with which the target style attribute was applied. 
     The stylized text  108 , the input text  106 , and other information processed or generated by the text style transfer system  104  are configured to be stored in storage of the computing device  102 , as described in further detail below with respect to  FIG.  6   . Alternatively or additionally, the text style transfer system  104  is configured to provide the input text  106 , the stylized text  108 , and additional information described in further detail below with respect to  FIGS.  2 - 5    to a remote storage location for subsequent retrieval and/or access by the computing device  102  or different computing devices. For instance, the text style transfer system  104  communicates information to remote storage  124 , or directly to a different computing device, via network  126 . 
     In general, functionality, features, and concepts described in relation to the examples above and below are employable in the context of the example procedures described in this section. Further, functionality, features, and concepts described in relation to different figures and examples in this document are interchangeable among one another and are not limited to implementation in the context of a particular figure or procedure. Moreover, blocks associated with different representative procedures and corresponding figures herein are configured to be applied together and/or combined in different ways. Thus, individual functionality, features, and concepts described in relation to different example environments, devices, components, figures, and procedures herein are useable in any suitable combinations and are not limited to the combinations represented by the enumerated examples in this description. 
     Having considered an example digital medium environment, consider now a discussion of an example system useable to generate stylized text in the writing style of a target author in accordance with aspects of the disclosure herein. 
       FIG.  2    depicts a digital medium environment  200  in an example implementation showing operation of the text style transfer system  104  of  FIG.  1    in greater detail. 
       FIGS.  3 A and  3 B  depict a digital medium environment  300  that includes an example transformer architecture for a transformer-based variational autoencoder implemented by the text style transfer system  104  of  FIG.  1   . 
       FIG.  4    depicts a digital medium environment  400  in an example implementation that includes a user interface for the text style transfer system  104  of  FIG.  1   . 
     As illustrated in  FIG.  2   , the text style transfer system  104  receives the input text  106 . The input text  106  is representative of digital text content that includes at least one sentence. Although the input text  106  is representative of digital text content authored in any language, it is described herein in the context of being written in the English language. The input text  106  is provided to the disentanglement module  118  together with an indication of a target style  202  to be used for rewriting the input text  106 . In some implementations, the indication of the target style  202  is received via input at a user interface of the text style transfer system  104 , as described in further detail below with respect to  FIG.  4   . 
     The target style  202  designates one or more style attributes (e.g., formality, sentiment, tone, excitement level, political inclination, author gender, and so forth) and a direction for each of the one or more attributes (e.g., positive or negative). In this manner, the target style  202  defines an objective for rewriting the input text  106  when generating the stylized text  108  that constrains a latent space distance between stylized versions of the stylized text  108  and the target style  202 . 
     The disentanglement module  118  implements a variational autoencoder  204  configured to represent the input text  106  in a latent space. Specifically, the disentanglement module  118  implements a variational autoencoder  204  trained to output a content representation  206  and a style representation  208  for the input text  106 . The content representation  206  is representative of data that encapsulates the semantic meaning of the input text  106  and the style representation  208  is representative of data that encapsulates style attributes of the input text  106  (e.g., formality, sentiment, tone, excitement level, political inclination, author gender, and so forth) and a magnitude (e.g., strength) of each style attribute with respect to a direction (e.g., positive or negative) for the style attribute. 
     Notably, the variational autoencoder  204  is configured to encode the input text  106  into a latent vector space in which the content representation  206  and the style representation  208  are mapped, independent of a corpus of text that contrasts a style expressed by the input text  106  with a different style, (e.g., the target style  202 ). Similarly, the variational autoencoder  204  is configured to output the content representation  206  and the style representation  208  independent of an annotation identifying one or more style attributes expressed by the input text  106 . 
     The variational autoencoder  204  is configured to encode the input text  106  into a latent vector space in which the content representation  206  and the style representation  208  are mapped. For instance, representing the input text  106  as x=(x 1 , x 2 , . . . x n ) where n represents text elements (e.g., words and punctuation marks) included in the input text  106 , the variational autoencoder  204  maps the input text  106  into a latent distribution H=q E (h|x). To do so, an encoder portion the variational autoencoder  204  reads the input text  106  text element-by-text element (e.g., word-by-word) and performs a linear transformation to obtain a hidden vector representation h. A decoder portion of the variational autoencoder  204  generates a sentence text element-by-text element, which when fully trained is the input text  106  itself. 
     The text style transfer system  104  is configured to train the variational autoencoder  204  using variational autoencoder loss  210 . For instance, suppose at a time step t the decoder portion of the variational autoencoder  204  predicts the word x t  for the input text  106  with probability p(xt|h, x 1  . . . x t-1 ). The variational autoencoder loss  210  (J VAE ) is represented by Equation 1. 
         J   VAE (θ E ,θ D )= J   REC +λ kl     [q   E ( h|x )∥ p ( h )]  (Eq. 1)
 
     In Equation 1, θ E  represents the encoder parameters, θ D  represents the decoder parameters, and J REC  represents the reconstruction loss implemented by the variational autoencoder  204 , which is dependent on the specific neural blocks used in the variational autoencoder  204 , as described in further detail below. λ k1  represents the hyperparameter balancing the Kullback-Leibler ( ) term that penalizes divergence between the input text  106  and the output text predicted by the decoder portion of the variational autoencoder  204 . p(h) is the prior, typically the standard normal  (0, 1), q E (h|x) is the posterior in the form of  (μ, diag σ 2 ), where μ and σ are predicted by the encoder portion of the variational autoencoder  204 . 
     In implementations where the variational autoencoder  204  is configured as an RNN-based variational autoencoder, the encoder portion of the variational autoencoder  204  is a bi-directional gated recurrent unit (Bi-GRU). The Bi-GRU encoder learns the hidden representation h by reading the input text  106 , modeled as x=(x 1 , x 2 , . . . x n ), sequentially. The decoder portion of the RNN-based variational autoencoder then decodes the output of the encoder portion sequentially over time, predicting probabilities of each token output by the encoder, conditioned on previous tokens and the latent representation. The reconstruction loss J REC  for such an RNN-based variational autoencoder is a negative-log-likelihood loss represented by Equation 2. 
     
       
         
           
             
               
                 
                   
                     J 
                     REC 
                   
                   = 
                   
                     
                       𝔼 
                       
                         h 
                         ~ 
                         
                           
                             q 
                             E 
                           
                           ( 
                           
                             h 
                             ❘ 
                             x 
                           
                           ) 
                         
                       
                     
                     [ 
                     
                       - 
                       
                         
                           ∑ 
                           
                             t 
                             = 
                             1 
                           
                           n 
                         
                         
                           log 
                           ⁢ 
                           
                             p 
                             ⁡ 
                             ( 
                             
                               
                                 
                                   x 
                                   t 
                                 
                                 ❘ 
                                 h 
                               
                               , 
                               
                                 
                                   x 
                                   1 
                                 
                                 ⁢ 
                                     
                                 … 
                               
                                   
                               , 
                               
                                 x 
                                 
                                   t 
                                   - 
                                   1 
                                 
                               
                             
                             ) 
                           
                         
                       
                     
                     ] 
                   
                 
               
               
                 
                   ( 
                   
                     Eq 
                     . 
                         
                     2 
                   
                   ) 
                 
               
             
           
         
       
     
     Alternatively, in some implementations the variational autoencoder  204  is configured as a transformer-based variational autoencoder. An example architecture of a transformer-based variational autoencoder is illustrated in  FIGS.  3 A and  3 B . 
     As illustrated in the digital medium environment  300  of  FIG.  3 A , when configured with a transformer-based architecture, the variational autoencoder  204  derives a positional encoding  302  for the input text  106 . The positional encoding  302  includes token embeddings  304  and position embeddings  306 . The token embeddings  304  describe a text element included in the input text  106  (e.g., words, numbers, punctuation, etc.) and are each correlated with one of the position embeddings  306  that describes the text element&#39;s position in the input text  106 . 
     The positional encoding  302  for the input text  106  is provided to a transformer encoder  308  of the variational autoencoder  204 . Specifically, the positional encoding  302  is communicated to an attention module  310  of the transformer encoder  308  in the form of a matrix, where an input sequence of symbol representations (e.g., token embeddings  304 ) are mapped to a sequence of continuous representations (e.g., position embeddings  306 ). Upon receipt of the positional encoding  302 , the attention module  310  of the transformer encoder  308  applies an attention function to the positional encoding  302 . 
     As described herein, the attention function applied by the attention module  310  represents a mapping of a query and a set of key-value pairs to an output, where the query, keys, values, and output are vectors. The output is computed as a weighted sum of the values, where the weight assigned to each value is computed by a compatibility function of the query with the corresponding key. In accordance with one or more implementations, the attention function applied by the attention module  310  is masked multi-headed self-attention  312 , which reduces the computational cost associated with determining relationships between words in the input text  106  having greater distances between their associated positional encodings  302 . 
     The masked multi-headed self-attention applied by attention module  310  enables the transformer encoder  308  to model information from different representation subspaces at different positions. In this manner, the masked multi-headed self-attention  312  may be represented as MultiHead(Q, K, V)=Concat(head 1 , . . . , head h )W O , where head i =Attention(QW i   Q , KW i   K , VW i   V ). In this representation, Q represents the queries, K represents the keys, and V represents the values of the attention function implemented by the attention module  310 , where the queries and keys are of dimension d k  and the values are of dimension d ν . 
     Projections output by the attention module  310  in implementing the masked multi-headed self-attention  312  are parameter matrices W i   Q ∈   d     model     ×d     k   , W i   K ∈   d     model     ×d     k   , W i   V ∈   d     model     ×d     ν   , and W O ∈   hd     ν     ×d     model   . In these parameter matrices, h is an integer representing a number of parallel attention layers, or heads, employed by attention module  310 . In accordance with one or more implementations, attention module  310  employs eight parallel attention layers. The output of the attention function applied to the positional encoding  302  by the attention module  310  is then normalized by the layer normalization module  314  before being passed to the feed forward network  316 . 
     The feed forward network  316  is representative of functionality of the transformer encoder  308  to apply linear transformations to received data, with a rectified linear unit (ReLU) activation applied in between the linear transformations. Functionality of the feed forward network  316  is thus represented as: FFN(x)=max(0, xW 1 +b 1 ) W 2 +b 2 . Linear transformations of the feed forward network  316  are applied in a pointwise manner, such that the linear transformations are applied to each position noted in the positional encoding  302  in a separate and identical manner Output values from the feed forward network  316  are again normalized by a layer normalization module  318 , which performs functionality similar to that of the layer normalization module  314  to generate a latent space text representation  320  for the input text  106 . The latent space text representation  320  is a hidden word representation (z 1 , z 2 , . . . , z n ) learned by the transformer encoder  308  and pooled into one or more sentences z, which is encoded into the probabilistic latent space q E (h|x). A sample from this latent space text representation  320  is provided as an input to an encoder/decoder attention module in the decoder portion of the variational autoencoder  204 , as illustrated in  FIG.  3 B . 
     As depicted in  FIG.  3 B , the positional encoding  302  for the input text  106  is passed to a transformer decoder  322  of the variational autoencoder  204 . The transformer decoder  322  is similar to the transformer encoder  308 , with an additional encoder-decoder attention portion. In this manner, the attention module  324  of the transformer decoder  322  performs functionality similar to that of the attention module  310  using masked multi-headed self-attention  312 . Layer normalization modules  326 ,  330 , and  334  thus perform functionality similar to that of layer normalization module  314  and feed forward network  332  performs functionality similar to that of feed forward network  316 . 
     Between the two sublayers in the transformer decoder  322  (e.g., the attention module  324  and the layer normalization module  326 ; and the feed forward network  332  and the layer normalization module  334 ), the transformer decoder  322  includes a layer including the encoder/decoder attention module  328  and the layer normalization module  330 . Queries used by the decoder attention module  328  include outputs from the feed forward network  316  and keys and values are obtained from the latent space text representation  320 . Consequently, the transformer decoder  322  is configured to reconstruct the input text  106  with condition on the hidden vector representation h, that is separable into two spaces s and c, where s is the style representation  208  and c is the content representation  206 . This reconstruction of the input text  106  is represented as the output text  336  generated by the transformer decoder  322 . 
     In some implementations, a transformer-based variational autoencoder  204  implemented by the disentanglement module  118  is pre-trained, such that the output text  336  is the same as the input text  106 . Alternatively, in some implementations the text style transfer system  104  is configured to train a transformer-based instance of the variational autoencoder  204  by iteratively processing the input text  106  to generate output text  336  and adjusting internal weights of the transformer encoder  308  and/or the transformer decoder  322  until the output text  336  matches the input text  106 . 
     Although the digital medium environment  300  is illustrated as including only a single transformer encoder  308  and a single transformer decoder  322 , in implementations a transformer-based variational autoencoder  204  is configured to include multiple instances of the transformer encoder  308  layered with one another and multiple instances of the transformer decoder  322  layered with one another. In implementations where the variational autoencoder  204  is configured as a transformer-based variational autoencoder, the reconstruction loss J REC  is represented by Equation 3. 
     
       
         
           
             
               
                 
                   
                     J 
                     REC 
                   
                   = 
                   
                     
                       𝔼 
                       
                         h 
                         ~ 
                         
                           
                             q 
                             E 
                           
                           ( 
                           
                             h 
                             ❘ 
                             x 
                           
                           ) 
                         
                       
                     
                     [ 
                     
                       - 
                       
                         
                           ∑ 
                           
                             i 
                             = 
                             1 
                           
                           
                             
                               ❘ 
                               &#34;\[LeftBracketingBar]&#34; 
                             
                             x 
                             
                               ❘ 
                               &#34;\[RightBracketingBar]&#34; 
                             
                           
                         
                         
                           ( 
                           
                             
                               
                                 ( 
                                 
                                   1 
                                   - 
                                   ϵ 
                                 
                                 ) 
                               
                               ⁢ 
                               
                                 
                                   ∑ 
                                   
                                     i 
                                     = 
                                     1 
                                   
                                   v 
                                 
                                 
                                   
                                     p 
                                     i 
                                   
                                   ⁢ 
                                   
                                     log 
                                     ⁡ 
                                     ( 
                                     
                                       p 
                                       i 
                                     
                                     ) 
                                   
                                 
                               
                             
                             + 
                             
                               
                                 ϵ 
                                 v 
                               
                               ⁢ 
                               
                                 
                                   ∑ 
                                   
                                     i 
                                     = 
                                     1 
                                   
                                   v 
                                 
                                 
                                   log 
                                   ⁡ 
                                   ( 
                                   
                                     p 
                                     i 
                                   
                                   ) 
                                 
                               
                             
                           
                           ) 
                         
                       
                     
                     ] 
                   
                 
               
               
                 
                   ( 
                   
                     Eq 
                     . 
                         
                     3 
                   
                   ) 
                 
               
             
           
         
       
     
     In Equation 3, v represents the vocabulary size, ∈ represents the label smoothing parameter,  p   i  represents the ground truth probability, and p i  represents the predicted probability over the vocabular at every time step for word-wise decoding. 
     The variational autoencoder loss  210 , as set forth in Equation 1, enables disentangled representation learning for the variational autoencoder  204  and trains the variational autoencoder  204  to separate the hidden vector representation h into two spaces s and c, where s is the style representation  208  and c is the content representation  206 . In this manner, the hidden vector representation h is represented as h=[s; c], where [;] denotes concatenation. However, while the reconstruction loss J REC  is the primary loss generation in Equation 1, it does not itself take into consideration the style of the input text  106  or the controlled generation of the content representation  206  or the style representation  208 . 
     To address these considerations, the text style transfer system  104  leverages additional multitask and adversarial losses on the latent space h to disentangle the embeddings generated by the variational autoencoder  204  into the content representation  206  and the style representation  208 . To ensure proper disentanglement, the disentanglement module  118  leverages both multitask and adversarial losses for each of the style and content domains of the latent space in which the input text  106  is represented. These multitask and adversarial losses are represented in  FIG.  2    as multitask style loss  212 , adversarial style loss  214 , multitask content loss  216 , and adversarial content loss  218 . 
     With respect to style-oriented loss, the multitask style loss  212  ensures that the style space is discriminative for style and the adversarial loss for style ensures that the content space is not discriminative of the style. The multitask style loss  212  is denoted J mul(s)  and used to train a style classifier on the style domain of the latent space in which the input text  106  is represented, jointly with the autoencoder loss for the variational autoencoder  204 . The multitask style loss  212  is defined as J mul(s) (θ E ; θ mul(s) )=−Σ lεlabels t s (l) log y s (l), where θ mul(s)  represents parameters for the style classifier, y s  represents the style probability distribution predicted by the style classifier, and t s (.) is the ground truth style distribution. 
     The adversarial style loss  214  is denoted J dis(s)  and used to train an adversarial classifier that deliberately discriminates a true style label for the input text  106  using the content representation  206  expressed as a vector in the latent space including the content representation  206  and the style representation  208 . The adversarial style loss  214  is defined as J dis(s) (θ dis(s) )=−Σ l∈labels t s (l) log y s ′(l). In the definition for the adversarial style loss  214 , θ dis(s)  represents parameters for style adversary and y s ′ is the style probability distribution predicted by the adversarial classifier on the content space. The encoder implemented by the variational autoencoder  204  (e.g., the transformer encoder  308 ) is trained to learn a content vector space c, from which its adversary cannot predict style information. In this manner, the objective is to maximize the cross entropy H(p)=Σ l∈labels  p i  log(p i ), with the adversary J adv(s) (θ E )=H(y s ′|c; θ dis(s) ). 
     The multitask content loss  216  is configured to ensure that all content information remains in the content space without seeping into the style space. Content information included in the content space is defined using part-of-speech tags (e.g., nouns) identified in the input text  106  and used to designate content. By defining content in this manner, the disentanglement module  118  allows for a generic content loss for all style dimensions, in contrast to conventional text modeling approaches that define content as a bag-of-words in a sentence, where stop-words and specific style-related lexicon (e.g., sentiment vocabulary) are excluded from being designated as content. 
     In this manner, the multitask content loss  216  is analogical to the multitask style loss  212  and defined as J mul(c) (θ E ; θ mul(c) )=−Σ w∈content  t c (w) log y c  (w). In the definition for the multitask content loss  216 , y c  represents the content probability distribution predicted by a content classifier and t c (.) represents the ground truth content distribution. 
     The adversarial content loss  218  ensures that the style space does not contain content information. In implementing the adversarial content loss  218 , the disentanglement module  118  trains a content adversary classifier on the style space to predict content features in the input text  106 . The disentanglement module  118  then trains the encoder of the variational autoencoder  204  (e.g., the transformer encoder  308 ) to learn a style vector space s, from which the content adversary cannot predict content information. The adversarial content loss  218  is defined as J dis(c) (θ dis(c) )=Σ w∈content  t c (w) log y c ′(w), with the adversary being J adv(c) (θ E )=H(y c ′|s; θ dis(s) ). 
     By training the variational autoencoder  204  with the variational autoencoder loss  210 , the multitask style loss  212 , the adversarial style loss  214 , the multitask content loss  216 , and the adversarial content loss  218 , the disentanglement module  118  ensures that the latent space used to represent the input text  106  is disentangled. Collectively, the final loss used by the disentanglement module  118  to train the variational autoencoder  204  is defined as J total =J VAE +λ mul(s) J mul(s) −λ adv(s) J adv(s) +λ mul(c) J mul(c) −λ adv(c) J adv(c) . By ensuring disentanglement, the disentanglement module  118  generates the content representation  206  without including style attribute information for the input text  106  and generates the style representation  208  without including content information for the input text  106 . 
     Although described and illustrated as generating only a single style representation  208 , the disentanglement module  118  is configured to generate a plurality of style representations  208  for the input text  106 , such that a different style representation  208  is generated for each of a plurality of style attributes conveyed by the input text  106 . In this manner, the disentanglement module  118  is configured to generate a first style representation  208  describing a formality expressed by the input text  106 , a second style representation  208  describing an excitement level expressed by the input text  106 , and so forth. 
     The disentanglement module  118  then provides the content representation  206  and the style representation  208  for the input text  106  to the counterfactual reasoning module  120 . The counterfactual reasoning module  120  is configured to identify an incremental degree by which the style representation  208  is adjustable to generate different stylized versions of the input text  106 , such that each version represents the target style  202  in a different manner. To do so, the counterfactual reasoning module  120  implements a multi-layer perceptron (MLP) classifier  220  trained on the disentangled style latent representations learned by the variational autoencoder  204 . 
     The MLP classifier  220  predicts, for each style representation  208  generated by the disentanglement module  118 , a variant of the style representation  208  with respect to the target style  202 . The objective of the MLP classifier  220  is to generate variants of the style representation  208  as close as possible to the style representation  208  while still achieving a different representation of the target style  202  (e.g., if the target style  202  is a positive sentiment the MLP classifier  220  attempts to generate a variant of the style representation  208  that is close to the style representation  208  and expresses a slightly less negative sentiment). 
     To enforce this objective, the counterfactual reasoning module  120  implements counterfactual generation loss, J cfactual . The counterfactual generation loss is defined as J cfactual =L(s′|s)=λ(f t (s′)−p t ) 2 +L 1 (s′,s). In the counterfactual loss definition, t represents the desired target style  202 , s′ represents the variant of the style representation  208  generated by the MLP classifier  220 , and p t  represents the probability with which the target style  202  is applied to generate s′. In this manner, p t =1 is representative of a perfect transfer of the target style  202  to the style representation  208 , such that the resulting s′ conveys precisely one or more style attributes specified by the target style  202 . 
     In the counterfactual loss definition, f t  is the model prediction on class t and L 1  is the distance between s′ and the style representation  208 , s. The first term in the counterfactual loss guides the MLP classifier  220  towards generating an s′ that adapts to the target style  202 , and λ represents a weighting term. The MLP classifier  220  uses the L 1  distance to ensure that a minimum number of features of the target style  202  are changed when generating s′. The counterfactual reasoning module  120  generates a target style embedding  222  that includes the resulting style representation variants  224 . Each variant of the style representation  208  included in the style representation variants  224  is defined by a style transfer strength that represents one of the probabilities p t  used to achieve s′. 
     The style representation variants  224  included in the target style embedding  222  are obtained by optimizing arg min s′ max λ L(s′|s), subject to |f t  (s′−p t )|≤∈, where ∈ is a tolerance parameter. For instance, in an example implementation where the trained MLP classifier  220  identifies at least three different probabilities p t  that are useable to generate variants of the style representation  208  with respect to the target style  202 , each of the probabilities p t  is represented as one of the transfer strengths  226 ,  228 , or  230 . Although illustrated in  FIG.  2    as including three variations of the style representation  208 , represented by transfer strengths  226 ,  228 , and  230 , the counterfactual reasoning module  120  is configured to generate any number of style representation variants  224 , constrained by the text elements included in the input text  106 . 
     In some implementations, different transfer strengths used to generate the style representation variants  224  (e.g., transfer strengths  226 ,  228 , and  230 ) are spaced equidistant from one another, such sequential transfer strengths represent proportionally incremental applications of the target style  202  to the style representation  208 . Alternatively or additionally, in implementations the style transfer strengths used to generate style representation variants  224  are not uniformly spaced from one another, such that when mapped onto an axis representing a spectrum of style attributes, a distance between a first and second transfer strength is not the same as a distance between a third transfer strength and the second transfer strength, and so forth. 
     The style representation variants  224  included in the target style embedding  222  are communicated to the stylization module  122  together with the content representation  206 . The stylization module  122  is configured to generate stylized versions of the input text  106  that each express a different degree of the target style  202  by combining the style representation variants  224  with the content representation  206 . To do so, the stylization module  122  uses an encoder-decoder framework, such as the example transformer-based encoder-decoder framework illustrated in the digital medium environment  300  of  FIGS.  3 A and  3 B  to concatenate the content representation  206  with one of the style representation variants  224 . 
     Each one of the style representation variants  224  is thus combinable with the content representation  206  to generate a stylized version of the input text  106  for output as part of the stylized text  108 . For instance, in the illustrated example of  FIG.  2   , the stylization module  122  concatenates the content representation  206  with one of the style representation variants  224  corresponding to transfer strength  226  to generate stylized version  232  of the input text  106 . Similarly, the content representation  206  is concatenated with one of the style representation variants  224  corresponding to transfer strength  228  to generate stylized version  234  and with a different one of the style representation variants  224  corresponding to transfer strength  230  to generate stylized version  236 . 
     Although the stylized text  108  is depicted in  FIG.  2    as including only three stylized versions of the input text  106  (e.g., stylized versions  232 ,  234 , and  236 ), the stylization module  122  is configured to generate any number of stylized versions of the input text  106 , such as one stylized version for each of the style representation variants  224  included in the target style embedding  222 . In some implementations, the stylization module  122  is configured to output (e.g., display) each stylized version of the stylized text  108  via a computing device implementing the text style transfer system  104 . For instance, the stylization module  122  is configured to display the stylized versions  110 ,  112 ,  114 , and  116  depicted in  FIG.  1    simultaneously, thereby providing an overview of how different strength applications of the target style  202  affect the input text  106 . 
     In some implementations, the stylization module  122  is configured to provide one or more controls that are configured to receive user input specifying an output strength  238  to be used for applying the target style  202  to the input text  106 . Specifying the output strength  238  to be used for applying the target style  202  to the style representation  208  of the input text  106  controls what stylized version of the stylized text  108  is output by the stylization module  122 . 
       FIG.  4    depicts an example of a user interface  402  for the text style transfer system  104 . In the illustrated example, the user interface  402  includes an input text region  404  configured to display input text (e.g., the input text  106 ) to be rewritten by applying attributes of a target style (e.g., the target style  202 ). In some implementations, the input text region  404  is configured to receive data from an input device of a computing device implementing the text style transfer system  104 , (e.g., a keyboard, a microphone, a touchscreen, and so forth) defining the input text  106 . 
     The user interface  402  further includes a style attributes region  406  configured with controls that enable designation of the output strength  238  to be used in generating stylized text  108 . Input to one or more controls included in the style attributes region  406  affects a stylized version of the input text  106  presented in a stylized text region  408  of the user interface  402 , such as one or more of the stylized versions  232 ,  234 , or  236 . 
     In the illustrated example of  FIG.  4   , the style attributes region  406  includes controls for adjusting three example style attributes for rewriting input text displayed in the input text region  404  as stylized text output in the stylized text region  408 . Specifically, the style attributes region  406  includes example controls for specifying a formality, an excitement, and a sentiment to be expressed by stylized text  108 . In the style attributes region  406 , controls for specifying style attributes are configured as slider controls, where a position of a slider relative to ends of a control bar specify a magnitude of the style attribute relative to a direction for the style attribute. 
     For instance, the illustrated example depicts a formality control, where a slider  410  is positioned relative to a first end  412  of a formality control bar and a second end  414  of the formality control bar. The first end  412  of the control bar indicates a negative direction of the formality style attribute (e.g., informal text) and the second end  414  of the control bar indicates a positive direction of the formality style attribute (e.g., formal text). In this manner, adjusting a position of the slider  410  towards the second end  414  increases a degree of formality expressed by the stylized version of the input text displayed in the stylized text region  408 . Conversely, adjusting a position of the  410  towards the first end  412  increases a degree of informality expressed by the stylized version of text displayed in the stylized text region  408 . 
     The excitement control is similarly configured with a slider  416  positioned relative to a first end  418  and a second end  420  of a control bar. The first end  418  indicates a negative direction of the excitement style attribute (e.g., text expressing a lack of excitement or indifference) and the second end  420  indicates a positive direction for the excitement style attribute (e.g., text emoting excitement or enthusiasm). Adjusting a position of the slider  416  towards the first end  418  thus decreases a level of excitement conveyed by the stylized text output in the stylized text region  408 , while adjusting a position of the slider  416  towards the second end  420  increases the level of excitement conveyed by the stylized text. 
     The sentiment control is configured with a slider  422  positioned relative to a first end  424  and a second end  426  of a control bar. The first end  424  indicates a negative direction of the sentiment style attribute (e.g., a negative sentiment expressed by text) and the second end  426  indicates a positive direction for the sentiment attribute (e.g., a positive sentiment expressed by text). Adjusting a position of the slider  422  towards the first end  424  thus causes output of a stylized version of input text expressing an increasingly negative sentiment, while adjusting a position of the slider  422  towards the second end  426  causes output of a stylized version of input text expressing an increasingly positive sentiment. 
     The formality, excitement, and sentiment controls depicted in the illustrated example of  FIG.  4    are merely representative of example controls supported by the text style transfer system  104  that enable specification of the output strength  238  to use in generating stylized text  108 . Although depicted as including three style attribute slider controls, the user interface  402  is configurable to include any number and format of style attribute controls. For instance, in implementations style attribute controls presented in the style attributes region  406  are configured as scroll wheels, buttons (e.g., up/down arrows), numeric entry fields, combinations thereof, and so forth. 
     The style attribute controls included in the style attributes region  406  thus enable a user of the text style transfer system  104  to specify, for one or more style attributes, a magnitude relative to a direction of the style attribute to use in selecting one or more of the style representation variants  224  for output in the user interface  402 . By deriving different transfer strengths for individual style attributes using counterfactual generation loss, style attribute controls provided by the text style transfer system  104  enable fine-tune adjustment of various style attributes when generating stylized text. 
     In some implementations, the style attributes region  406  is configured to provide an indication (e.g., a visual display, an audible report, combinations thereof, and so forth) of the specific style attribute transfer strength selected via each of the style attribute controls. For instance, in an example implementation the tick marks on the formality control bar depicted in  FIG.  4    each correspond to one of the transfer strengths  226 ,  228 , and  230  derived by the counterfactual reasoning module  120  for a formality target style  202 . Input positioning the slider  410  relative to one of the tick marks causes the style attributes region  406  to provide an indication of the corresponding transfer strength selected for the formality attribute (e.g., transfer strength  228 ) used to generate the stylized text output in stylized text region  408 . In this manner, the user interface  402  provides a clear indication to a user of the text style transfer system  104  regarding the specific strength at which a target style  202  is applied to input text  106  and how moderating influence of a particular style attribute impacts the resulting stylized text  108 . 
     Although the stylized text region  408  in  FIG.  4    depicts only a single instance of a stylized version of input text  106 , the stylized text region  408  is configurable to display a plurality of stylized versions, such as a simultaneous display of the stylized versions  232 ,  234 , and  236 . In implementations where the stylized text region  408  outputs a simultaneous display of multiple stylized versions of input text, the user interface  402  is configured to visually distinguish one of the stylized versions from other stylized versions based on the style attributes region  406 . For instance, the text style transfer system  104  is configured to visually emphasize (e.g., highlight, bold, italicize, border, etc.) one of a plurality of stylized versions output in the stylized text region  408  that would be generated using a current designation of style attributes to be applied to input text as specified by controls in the style attributes region  406 . 
     Using the stylized text  108  depicted in  FIG.  1    as an example, input to controls presented in style attributes region  406  causes the text style transfer system  104  to visually distinguish one of the stylized versions  110 ,  112 ,  114 , or  116  from other ones of the stylized versions that would not result from a current style attribute control configuration. The current position of the slider  416  as depicted in  FIG.  4   , for instance, might cause the text style transfer system  104  to highlight or otherwise visually distinguish the stylized version  116  from the stylized versions  110 ,  112 , and  114 . Continuing this example, input at the excitement style attribute in style attributes region  406  moving the slider  416  towards the first end  418  of the excitement control bar causes the text style transfer system  104  to highlight or otherwise emphasize different ones of the stylized versions  110 ,  112 , and  114  (e.g., iteratively highlighting stylized version  114 , then stylized version  112 , until ultimately highlighting stylized version  110  upon positioning of the slider  416  and the first end  418 ). 
     In this manner, the user interface  402  provides a clear indication of how applying style attributes at varying strengths to input text depicted in the input text region  404  affects a resulting stylized version of text output in the stylized text region  408 . Advantageously, the techniques described herein provide fine-tuned control of applying one or more style attributes to input text, generate a plurality of versions of stylized text for each strength application of the one or more style attributes, and clearly demonstrate how moderating influence of a particular style attribute impacts the resulting stylized text, which is not enabled by conventional text styling approaches. 
     Having considered example systems and techniques, consider now example procedures to illustrate aspects of the techniques described herein. 
     EXAMPLE PROCEDURES 
     The following discussion describes techniques that are configured to be implemented utilizing the previously described systems and devices. Aspects of each of the procedures are configured for implementation in hardware, firmware, software, or a combination thereof. The procedures are shown as a set of blocks that specify operations performed by one or more devices and are not necessarily limited to the orders shown for performing the operations by the respective blocks. In portions of the following discussion, reference is made to  FIGS.  1 - 4   . 
       FIG.  5    is a flow diagram depicting a procedure  500  in an example implementation of generating stylized versions of input text using the techniques described herein. To begin, input text is received (block  502 ). The text style transfer system  104  for instance, receives input text  106 . In some implementations, the input text  106  is received via an input device (e.g., keyboard, microphone, etc.) of a computing device implementing the text style transfer system  104 . For instance, in some implementations the input text  106  is received via input to an input text region  404  of a user interface  402  for the text style transfer system  104 . 
     An attribute of the input text to change when rewriting the input text in a target style is identified (block  504 ). The disentanglement module  118 , for instance, receives an indication of a target style  202  to use for generating stylized text  108  that includes a plurality of stylized versions of the input text  106 , each generated by applying a different strength of the target style  202  to the input text  106 . In some implementations, the target style  202  is specified via input to one or more controls presented in a user interface  402  for the text style transfer system  104 , such as the example style attribute controls depicted in the style attributes region  406  of  FIG.  4   . 
     The plurality of stylized versions of the input text are then generated (block  506 ). As part of generating the plurality of stylized versions of the input text  106  (e.g., the stylized versions  110 ,  112 ,  114 , and  116 ), a content representation and a style representation of the input text are disentangled (block  508 ). To do so, the disentanglement module  118  implements a variational autoencoder  204  using variational autoencoder loss  210 , multitask style loss  212 , adversarial style loss  214 , multitask content loss  216 , and adversarial content loss  218  to generate the content representation  206  and the style representation  208  for the input text  106 . Using this combination of losses, the disentanglement module  118  ensures that style information for the input text  106  is not expressed in the content representation  206  and that content information for the input text  106  is not expressed in the style representation  208 . 
     In some implementations, the style representation  208  is generated based the target style  202  specified for use in generating the stylized text  108 . For instance, in an example implementation where the target style  202  indicates that the input text  106  is to be rewritten as stylized text  108  using varying degrees of formality, the style representation  208  includes information describing a formality expressed by the input text  106 . In such implementations, the disentanglement module  118  leverages a variational autoencoder  204  pre-trained to derive style representations for the target style  202 . Alternatively or additionally, the disentanglement module  118  is configured to train the variational autoencoder  204  to output style representations for the target style  202  using the techniques described herein. 
     A plurality of transfer strengths that are useable to modify the attribute of the input text for the target style are then identified (block  510 ). The counterfactual reasoning module  120 , for instance, implements a trained MLP classifier  220  using counterfactual generation loss to identify different probability values that are useable to achieve different text variants for the target style  202  having varied degrees of transferring the target style  202  to the input text  106 . These plurality values are expressed as transfer strengths for the target style  202 , such as the example transfer strengths  226 ,  228 , and  230  illustrated in  FIG.  2   . 
     A plurality of style representation variants are generated using the plurality of transfer strengths (block  512 ). The counterfactual reasoning module  120 , for instance, generates a target style embedding  222  that includes a plurality of style representation variants  224 , one for each of the plurality of transfer strengths derived by the trained MLP classifier  220  (e.g., one style representation variant for each of the transfer strengths  226 ,  228 , and  230 ). Each of the style representation variants  224  is thus representative of an instance of the style representation  208  generated via application of a different degree of the target style  202 . 
     To generate the plurality of stylized versions of the input text, the content representation is reconstructed using the plurality of style representation variants (block  514 ). The stylization module  122 , for instance, receives the content representation  206  and the style representation variants  224  included in the target style embedding  222 . The stylization module  122  then concatenates one of the style representation variants  224  with the content representation  206  to generate a stylized version of the input text  106 . The stylization module  122  repeats this process of concatenating one of the style representation variants  224  with the content representation  206  for each style representation variant represented by one of the transfer strengths in the target style embedding  222 , thereby generating a plurality of stylized versions of the input text  106  (e.g., stylized versions  232 ,  234 , and  236 ). 
     The plurality of stylized versions of the input text are then output (block  516 ). The stylization module  122 , for instance, outputs the plurality of stylized versions of the input text input text  106  included in the stylized text  108  for display via a computing device implementing the text style transfer system  104 . In some implementations, the stylized text  108  is displayed via a user interface for the text style transfer system  104 , such as via the stylized text region  408  of the example user interface  402  depicted in  FIG.  4   . 
     Having described example procedures in accordance with one or more implementations, consider now an example system and device to implement the various techniques described herein. 
     Example System and Device 
       FIG.  6    illustrates an example system  600  that includes an example computing device  602 , which is representative of one or more computing systems and/or devices that implement the various techniques described herein. This is illustrated through inclusion of the text style transfer system  104 . The computing device  602  is configured, for example, as a service provider server, as a device associated with a client (e.g., a client device), as an on-chip system, and/or as any other suitable computing device or computing system. 
     The example computing device  602  as illustrated includes a processing system  604 , one or more computer-readable media  606 , and one or more I/O interface  608  that are communicatively coupled, one to another. Although not shown, the computing device  602  is further configured to include a system bus or other data and command transfer system that couples the various components, one to another. A system bus includes any one or combination of different bus structures, such as a memory bus or memory controller, a peripheral bus, a universal serial bus, and/or a processor or local bus that utilizes any of a variety of bus architectures. A variety of other examples are also contemplated, such as control and data lines. 
     The processing system  604  is representative of functionality to perform one or more operations using hardware. Accordingly, the processing system  604  is illustrated as including hardware element  610  that are configurable as processors, functional blocks, and so forth. For instance, hardware element  610  is implemented in hardware as an application specific integrated circuit or other logic device formed using one or more semiconductors. The hardware elements  610  are not limited by the materials from which they are formed, or the processing mechanisms employed therein. For example, processors are alternatively or additionally comprised of semiconductor(s) and/or transistors (e.g., electronic integrated circuits (ICs)). In such a context, processor-executable instructions are electronically executable instructions. 
     The computer-readable storage media  606  is illustrated as including memory/storage  612 . The memory/storage  612  represents memory/storage capacity associated with one or more computer-readable media. The memory/storage  612  is representative of volatile media (such as random-access memory (RAM)) and/or nonvolatile media (such as read only memory (ROM), Flash memory, optical disks, magnetic disks, and so forth). The memory/storage  612  is configured to include fixed media (e.g., RAM, ROM, a fixed hard drive, and so on) as well as removable media (e.g., Flash memory, a removable hard drive, an optical disc, and so forth). In certain implementations, the computer-readable media  606  is configured in a variety of other ways as further described below. 
     Input/output interface(s)  608  are representative of functionality to allow a user to enter commands and information to computing device  602  and allow information to be presented to the user and/or other components or devices using various input/output devices. Examples of input devices include a keyboard, a cursor control device (e.g., a mouse), a microphone, a scanner, touch functionality (e.g., capacitive, or other sensors that are configured to detect physical touch), a camera (e.g., a device configured to employ visible or non-visible wavelengths such as infrared frequencies to recognize movement as gestures that do not involve touch), and so forth. Examples of output devices include a display device (e.g., a monitor or projector), speakers, a printer, a network card, tactile-response device, and so forth. Thus, the computing device  602  is representative of a variety of hardware configurations as further described below to support user interaction. 
     Various techniques are described herein in the general context of software, hardware elements, or program modules. Generally, such modules include routines, programs, objects, elements, components, data structures, and so forth that perform particular tasks or implement particular data types. The terms “module,” “functionality,” and “component” as used herein generally represent software, firmware, hardware, or a combination thereof. The features of the techniques described herein are platform-independent, meaning that the techniques are configured for implementation on a variety of commercial computing platforms having a variety of processors. 
     An implementation of the described modules and techniques are stored on or transmitted across some form of computer-readable media. The computer-readable media include a variety of media that is accessible by the computing device  602 . By way of example, and not limitation, computer-readable media includes “computer-readable storage media” and “computer-readable signal media.” 
     “Computer-readable storage media” refers to media and/or devices that enable persistent and/or non-transitory storage of information in contrast to mere signal transmission, carrier waves, or signals per se. Thus, computer-readable storage media refers to non-signal bearing media. The computer-readable storage media includes hardware such as volatile and non-volatile, removable and non-removable media and/or storage devices implemented in a method or technology suitable for storage of information such as computer readable instructions, data structures, program modules, logic elements/circuits, or other data. Examples of computer-readable storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, hard disks, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or other storage device, tangible media, or article of manufacture suitable to store the desired information for access by a computer. 
     “Computer-readable signal media” refers to a signal-bearing medium that is configured to transmit instructions to the hardware of the computing device  602 , such as via a network. Signal media typically embody computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as carrier waves, data signals, or other transport mechanism. Signal media also include any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media. 
     As previously described, hardware elements  610  and computer-readable media  606  are representative of modules, programmable device logic and/or fixed device logic implemented in a hardware form that is employed in some embodiments to implement at least some aspects of the techniques described herein, such as to perform one or more instructions. Hardware, in certain implementations, includes components of an integrated circuit or on-chip system, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a complex programmable logic device (CPLD), and other implementations in silicon or other hardware. In this context, hardware operates as a processing device that performs program tasks defined by instructions and/or logic embodied by the hardware as well as a hardware utilized to store instructions for execution, e.g., the computer-readable storage media described previously. 
     Combinations of the foregoing are employed to implement various techniques described herein. Accordingly, software, hardware, or executable modules are implemented as one or more instructions and/or logic embodied on some form of computer-readable storage media and/or by one or more hardware elements  610 . The computing device  602  is configured to implement instructions and/or functions corresponding to the software and/or hardware modules. Accordingly, implementation of a module that is executable by the computing device  602  as software is achieved at least partially in hardware, e.g., through use of computer-readable storage media and/or hardware elements  610  of the processing system  604 . The instructions and/or functions are executable/operable by one or more articles of manufacture (for example, one or more computing devices  602  and/or processing systems  604 ) to implement techniques, modules, and examples described herein. 
     The techniques described herein are supported by various configurations of the computing device  602  and are not limited to the specific examples of the techniques described herein. This functionality is further configured to be implemented all or in part through use of a distributed system, such as over a “cloud”  614  via a platform  616  as described below. 
     The cloud  614  includes and/or is representative of a platform  616  for resources  618 . The platform  616  abstracts underlying functionality of hardware (e.g., servers) and software resources of the cloud  614 . The resources  618  include applications and/or data that is utilized while computer processing is executed on servers that are remote from the computing device  602 . Resources  618  also include services provided over the Internet and/or through a subscriber network, such as a cellular or Wi-Fi network. 
     The platform  616  is configured to abstract resources and functions to connect the computing device  602  with other computing devices. The platform  616  is further configured to abstract scaling of resources to provide a corresponding level of scale to encountered demand for the resources  618  that are implemented via the platform  616 . Accordingly, in an interconnected device embodiment, implementation of functionality described herein is configured for distribution throughout the system  600 . For example, in some configurations the functionality is implemented in part on the computing device  602  as well as via the platform  616  that abstracts the functionality of the cloud  614 . 
     CONCLUSION 
     Although the invention has been described in language specific to structural features and/or methodological acts, the invention defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as example forms of implementing the claimed invention.