Behavior Gated Language Models	Most current language modeling techniques only exploit co-occurrence, semantic and syntactic information from the sequence of words. However, a range of information such as the state of the speaker and dynamics of the interaction might be useful. In this work we derive motivation from psycholinguistics and propose the addition of behavioral information into the context of language modeling. We propose the augmentation of language models with an additional module which analyzes the behavioral state of the current context. This behavioral information is used to gate the outputs of the language model before the final word prediction output. We show that the addition of behavioral context in language models achieves lower perplexities on behavior-rich datasets. We also confirm the validity of the proposed models on a variety of model architectures and improve on previous state-of-the-art models with generic domain Penn Treebank Corpus.

Methodology ::: Behavior Gated Language Model ::: Proposed Model	We propose to augment RNN language models with a behavior model that provides information relating to a speaker's psychological state. This behavioral information is combined with hidden layers of the RNNLM through a gating mechanism prior to output prediction of the next word. In contrast to typical language models, we propose to model $ P(\mathbf {y} \mid \mathbf {x}, \mathbf {z}) $ where $ \mathbf {z} \equiv f( P(\mathbf {B}\mid \mathbf {x}))$ for an RNN function $f(\cdot )$. The behavior model is implemented with a multi-layered RNN over the input sequence of words. The first recurrent layer of the behavior model is initialized with pre-trained weights from the model described in Section SECREF3 and fixed during language modeling training. An overview of the proposed behavior gated language model is shown in Figure FIGREF6. The RNN units shaded in green (lower section) denote the pre-trained weights from the behavior classification model which are fixed during the entirety of training.

Conclusion & Future Work	In this study, we introduce the state of the speaker/author into language modeling in the form of behavior signals. We track 5 behaviors namely acceptance, blame, negativity, positivity and sadness using a 5 class multi-label behavior classification model. The behavior states are used as gating mechanism for a typical RNN based language model. We show through our experiments that the proposed technique improves language modeling perplexity specifically in the case of behavior-rich scenarios. Finally, we show improvements on the previous state-of-the-art benchmark model with Penn Tree Bank Corpus to underline the affect of behavior states in language modeling. In future, we plan to incorporate the behavior-gated language model into the task of automatic speech recognition (ASR). In such scenario, we could derive both the past and the future behavior states from the ASR which could then be used to gate the language model using two pass re-scoring strategies. We expect the behavior states to be less prone to errors made by ASR over a sufficiently long context and hence believe the future behavior states to provide further improvements.

Methodology ::: Behavior Gated Language Model ::: Motivation	Behavior understanding encapsulates a long-term trajectory of a person's psychological state. Through the course of communication, these states may manifest as short-term instances of emotion or sentiment. Previous work has studied the links between these psychological states and their effect on vocabulary and choice of words  as well as use of language . Motivated from these studies, we hypothesize that due to the duality of behavior and language we can improve language models by capturing variability in language use caused by different psychological states through the inclusion of behavioral information.

Results ::: Penn Tree Bank Corpus ::: Previous state-of-the-art architectures	Finally we apply behavior gating on a previous state-of-the-art architecture, one that is most often used as a benchmark over various recent works. Specifically, we employ the AWD-LSTM proposed by  with QRNN  instead of LSTM. We observe positive results with AWD-LSTM augmented with behavior-gating providing a relative improvement of (1.42% on valid) 0.66% in perplexity (Table TABREF17).

Experimental Setup ::: Behavior Model	The behavior model was implemented using an RNN with LSTM units and trained with the Couples Therapy Corpus. Out of the 33 behavioral codes included in the corpus we applied the behaviors Acceptance, Blame, Negativity, Positivity, and Sadness to train our models. This is motivated from previous works which showed good separability in these behaviors as well as being easy to interpret. The behavior model is pre-trained to identify the presence of each behavior from a sequence of words using a multi-label classification scheme. The pre-trained portion of the behavior model was implemented using a single layer RNN with LSTM units with dimension size 50.

The abstract behavior outputs $ b_t $ of the pre-trained model are fed into a time-synced RNN, denoted in blue (upper section), which is subsequently used for gating the RNNLM predictions. The un-shaded RNN units correspond to typical RNNLM and operate in parallel to the former.

Results ::: Behavior Related Corpora ::: Couple's Therapy Corpus	We utilize the Couple's Therapy Corpus as an in-domain experimental corpus since our behavior classification model is also trained on the same. The RNNLM architecture is similar to , but with hyperparameters optimized for the couple's corpus. The results are tabulated in Table TABREF16 in terms of perplexity. We find that the behavior gated language models yield lower perplexity compared to vanilla LSTM language model. A relative improvement of 2.43% is obtained with behavior gating on the couple's data.

Results ::: Behavior Related Corpora ::: Cancer Couples Interaction Dataset	To evaluate the validity of the proposed method on an out-of-domain but behavior related task, we utilize the Cancer Couples Interaction Dataset. Here both the language and the behavior models are trained on the Couple's Therapy Corpus. The Cancer dataset is used only for development (hyper-parameter tuning) and testing. We observe that the behavior gating helps achieve lower perplexity values with a relative improvement of 6.81%. The performance improvements on an out-of-domain task emphasizes the effectiveness of behavior gated language models.

Results ::: Penn Tree Bank Corpus	Although the proposed model is motivated and targeted towards behavior related datasets, the hypothesis should theoretically extend towards any human generated corpora. To assess this, we also train models on a non-behavior-rich database, the Penn Tree Bank Corpus. We experiment with both the medium and large architectures proposed by . The perplexity results on PTB are presented in Table TABREF17. All language models showed an improvement in perplexity through the addition of behavior gates. It can also be observed that LSTM-Medium with behavior gating gives similar performance to baseline LSTM-Large even though the latter has more than three times the number of parameters.

The use of topic information to provide semantic context to language models has also been studied extensively , , , . Topic models are useful for extracting high level semantic structure via latent topics which can aid in better modeling of longer documents. Recently, however, empirical studies involving investigation of different network architectures, hyper-parameter tuning, and optimization techniques have yielded better performance than the addition of contextual information , . In contrast to the majority of work that focus on improving the neural network aspects of RNNLM, we introduce psycholinguistic signals along with linguistic units to improve the fundamental language model. In this work, we utilize behavioral information embedded in the language to aid the language model. We hypothesize that different psychological behavior states incite differences in the use of language , , and thus modeling these tendencies can provide useful information in statistical language modeling. And although not directly related, behavioral information may also correlate with conversation-type and topic. Thus, we propose the use of psycholinguistic behavior signals as a gating mechanism to augment typical language models. Effectively inferring behaviors from sources like spoken text, written articles can lead to personification of the language models in the speaker-writer arena.

Experimental Setup ::: Data ::: Behavior Related Corpora	For evaluating the proposed model on behavior related data, we employ the Couples Therapy Corpus (CoupTher)  and Cancer Couples Interaction Dataset (Cancer) . These are the targeted conditions under which a behavior-gated language model can offer improved performance. Couples Therapy Corpus: This corpus comprises of dyadic conversations between real couples seeking marital counseling. The dataset consists of audio, video recordings along with their transcriptions. Each speaker is rated by multiple annotators over 33 behaviors. The dataset comprises of approximately 0.83 million words with 10,000 unique entries of which 0.5 million is used for training (0.24m for dev and 88k for test). Cancer Couples Interaction Dataset: This dataset was gathered as part of a observational study of couples coping with advanced cancer. Advanced cancer patients and their spouse caregivers were recruited from clinics and asked to interact with each other in two structured discussions: neutral discussion and cancer related. Interactions were audio-recorded using small digital recorders worn by each participant. Manually transcribed audio has approximately 230,000 word tokens with a vocabulary size of 8173.

Introduction	Recurrent neural network language models (RNNLM) can theoretically model the word history over an arbitrarily long length of time and thus have been shown to perform better than traditional n-gram models . Recent prior work has continuously improved the performance of RNNLMs through hyper-parameter tuning, training optimization methods, and development of new network architectures , , , . On the other hand, many work have proposed the use of domain knowledge and additional information such as topics or parts-of-speech to improve language models. While syntactic tendencies can be inferred from a few preceding words, semantic coherence may require longer context and high level understanding of natural language, both of which are difficult to learn through purely statistical methods. This problem can be overcome by exploiting external information to capture long-range semantic dependencies. One common way of achieving this is by incorporating part-of-speech (POS) tags into the RNNLM as an additional feature to predict the next word , . Other useful linguistic features include conversation-type, which was shown to improve language modeling when combined with POS tags . Further improvements were achieved through the addition of socio-situational setting information and other linguistic features such as lemmas and topic .

Methodology ::: Behavior Model	The analysis and processing of human behavior informatics is crucial in many psychotherapy settings such as observational studies and patient therapy . Prior work has proposed the application of neural networks in modeling human behavior in a variety of clinical settings , , . In this work we adopt a behavior model that predicts the likelihood of occurrence of various behaviors based on input text. Our model is based on the RNN architecture in Figure FIGREF2, but instead of the next word we predict the joint probability of behavior occurrences $ P(\mathbf {B} \mid \mathbf {x}) $ where $ \mathbf {B} \hspace{2.77771pt}{=}\hspace{2.77771pt}\lbrace b_{i}\rbrace $ and $ b_{i} $ is the occurrence of behavior $i$. In this work we apply the behaviors: Acceptance, Blame, Negativity, Positivity, and Sadness. This is elaborated more on in Section SECREF3.

Methodology	In this section, we first describe a typical RNN based language model which serves as a baseline for this study. Second, we introduce the proposed behavior prediction model for extracting behavioral information. Finally, the proposed architecture of the language model which incorporates the behavioral information through a gating mechanism is presented.

Experimental Setup ::: Hyperparameters	We augmented previous RNN language model architectures by  and  with our proposed behavior gates. We used the same architecture as in each work to maintain similar number of parameters and performed a grid search of hyperparameters such as learning rate, dropout, and batch size. The number of layers and size of the final layers of the behavior model was also optimized. We report the results of models based on the best validation result.

Results	We split the results into two parts. We first validate the proposed technique on behavior related language modeling tasks and then apply it on more generic domain Penn Tree bank dataset.

Experimental Setup ::: Data ::: Penn Tree Bank Corpus	In order to evaluate our proposed model on more generic language modeling tasks, we employ Penn Tree bank (PTB) , as preprocessed by . Since Penn Tree bank mainly comprises of articles from Wall Street Journal it is not expected to contain substantial expressions of behavior.

Methodology ::: Language Model	The basic RNNLM consists of a vanilla unidirectional LSTM which predicts the next word given the current and its word history at each time step. In other words, given a sequence of words $ \mathbf {x} \hspace{2.77771pt}{=}\hspace{2.77771pt}x_1, x_2, \ldots x_n$ as input, the network predicts a probability distribution of the next word $ y $ as $ P(y \mid \mathbf {x}) $. Figure FIGREF2 illustrates the basic architecture of the RNNLM. Since our contribution is towards introducing behavior as a psycholinguistic feature for aiding the language modeling process, we stick with a reliable and simple LSTM-based RNN model and follow the recommendations from  for our baseline model.

Question: How is module that analyzes behavioral state trained?
Answer: pre-trained to identify the presence of behavior from a sequence of word using the Couples Therapy Corpus