The Manifold of Human Emotions
Authors: Seungyeon Kim, Fuxin Li, Guy Lebanon, Irfan Essa
ICLR 2013

Abstract: Sentiment analysis predicts the presence of positive or negative emotions in a text document. In this paper, we consider higher dimensional extensions of the sentiment concept, which represent a richer set of human emotions. Our approach goes beyond previous work in that our model contains a continuous manifold rather than a finite set of human emotions. We investigate the resulting model, compare it to psychological observations, and explore its predictive capabilities.

arXiv:1301.3214v1  [cs.CL]  15 Jan 2013
The Manifold of Human Emotions
Seungyeon Kim, Fuxin Li, Guy Lebanon, Irfan Essa
College of Computing
Georgia Institute of Technology
{seungyeon.kim@, fli@cc., lebanon@cc., irfan@cc.}gatech.edu
Abstract
Sentiment analysis predicts the presence of positive or negative emotions in a text
document. In this paper, we consider higher dimensional ext ensions of the senti-
ment concept, which represent a richer set of human emotions. Our approach goes
beyond previous work in that our model contains a continuous manifold rather
than a ﬁnite set of human emotions. We investigate the resulting model, compare
it to psychological observations, and explore its predictive capabilities.
1 Introduction
Sentiment analysis predicts the presence of a positive or ne gative emotion y in a text document x.
Despite its successes in industry, sentiment analysis is li mited as it ﬂattens the structure of human
emotions into a single dimension. An alternative that has attracted a few researchers in recent years
is to construct a ﬁnite collection of emotions and ﬁt a predic tive model for each emotion (see for
example [1]). There are several signiﬁcant difﬁculties wit h the above approach. First, it is hard to
capture a complex statistical relationship between a large number of binary variables (representing
emotions) and a high dimensional vector (representing the d ocument). It is also hard to imagine
a reliable procedure for compiling a ﬁnite list of all possib le human emotions. Finally, it is not
clear how to use documents expressing a certain emotion.Usi ng labeled documents only in ﬁtting
models predicting their denoted labels ignores the relationship among emotions, and is problematic
for emotions with only a few annotated.
We propose an alternative approach that models a stochastic relationship between the document X,
an emotion label Y (such as sleepy or happy), and a position on the mood manifold Z. We
assume that all the emotional aspects in the documents are ca ptured by the manifold, implying that
the emotion label Y can be inferred directly from the projectionZ of the document on the manifold,
without needing to consult the document again.
2 The Statistical Model
We make the following four modeling assumptions concerning the document X, the discrete emo-
tion label Y ∈ { 1, 2, . . . , C }, and the position on the continuous mood manifold Z ∈ Rl.
1. We have the graphical structure: X → Z → Y , implying that the emotion label Y ∈
{1, . . . , C } is independent of the document X given Z.
2. The distribution of Z ∈ Rl given a speciﬁc emotion label Y = y is Gaussian:
{Z|Y = y} ∼ N (µy , Σ y)
3. The distribution of Z given the document X (typically in a bag of words or n-gram repre-
sentation) is a linear regression model: {Z|X = x} ∼ N (θ⊤x, Σ x).
4. The distances between the vectors in {E(Z|Y = y) : y ∈ C} are similar to the correspond-
ing distances in {E(X|Y = y) : y ∈ C}.
1
chipper
tired
blah depressed
happy
accomplished
amused
cheerful
aggravatedcontent
busy anxious
annoyed
sad
cold
awake
contemplative
sleepy
excited
okay
calm
bouncy
curious
artistic
sick
bored
exhausted
creative
hungry
confused
blank
hopeful
Figure 1: (left) The two-dimensional structure of emotions from [2]. We can interpret top-left to bottom-right
axis as expressing sentiment polarity and the top-right to b ottom-left axis as expressing engagement. (right)
Mood centroids E(Z|Y = y) on the two most prominent dimensions in emotion space ﬁtted f rom blog posts.
See text for details.
We make the following observations. First, the ﬁrst assumption implies that the emotion label Y is
simply a discretization of the continuous Z. It is consistent with well known research in psychology
(see Section 2) and with random projection theory, which sta te that it is often possible to approxi-
mate high dimensional data by projecting it on a low dimensio nal continuous space. Second, while
X, Y are high dimensional and discrete, Z is low dimensional and continuous. This, together with
the conditional independence in assumption 1 above, implies a higher degree of accuracy than mod-
eling directly X → Y . Intuitively, the number of parameters is on the order of dim (X) + dim(Y )
as opposed to dim(X)dim(Y ). Third, the Gaussian models in assumptions 2 and 3 are simple , and
lead to efﬁcient computational procedures. We also found th em to work well in our experiments.
The model may be easily adapted, however, to more complex models such as mixture of Gaussians
or non-linear regression models (for example, we experimen ted with quadratic regression models).
Fourth, assumption 4 suggests that we can estimate E(Z|Y = y) for all y ∈ C via multidimen-
sional scaling. MDS ﬁnds low dimensional coordinates for a s et of points that approximates the
spatial relationship between the points in the original high dimensional space. Lastly, the models in
assumptions 2 and 3 are statistical and can be estimated from data using maximum likelihood.
Motivated by the fourth modeling assumption, we determine t he parameters µy = E(Z|Y =
y), y ∈ C by running multidimensional scaling (MDS) or Kernel PCA on t he empirical versions
of {E(X|Y = y) : y ∈ C}.
We estimate the parameter θ, deﬁning the regression X → Z, by maximizing the likelihood which
requires integrating over Z ∈ Rl, a computationally difﬁcult task when l is not very low. We make
use of approximating the Gaussian pdf with Dirac’s delta function on p(z),
ˆθ ≈ arg max
θ
∑
i
log p(y(i))pθ (z(i)∗
|x(i))
∑
y p(z(i)∗
|y)p(y) = arg max
θ
∑
i
log pθ (z(i)∗
|x(i)) (1)
where z(i)∗
= arg max z p(z|y(i)) = E(Z|y(i)), which is equivalent to a least squares regression.
One interpretation of our model X → Z → Y is that Z forms a sufﬁcient statistic of X for Y . We
can thus consider adapting a wide variety of predictive mode ls (for example, logistic regression or
SVM) on Z ↦→Y . These discriminative classiﬁers are trained on {( ˆZ(i), Y (i)), i = 1 , . . . , n }.
3 Experiments
We used crawled Livejournal1 data as the main dataset. About 20% of the blog posts feature t hese
optional annotations in the form of emoticons. The annotati ons may be chosen from a pre-deﬁned
list of possible emotions, or a novel emotion speciﬁed by the author. We crawled 15,910,060 doc-
1http://www.livejournal.com
2
Table 1: Macro F1 score and accuracy over the test set in multiclass emotion classiﬁcation over top 32 moods
(left) and sentiment polarity task (right): {cheerful, happy, amused} vs {sad, annoyed, depressed, confused}.
See text for details.
Original Space Mood Manifold
F1 Acc. F1 Acc.
LDA full n/a n/a 0.1247 0.1635
diag. 0.1229 0.1441 0.1160 0.1600
spher. 0.0838 0.1075 0.0896 0.1303
QDA full n/a n/a 0.1206 0.1478
diag. 0.0878 0.0931 0.1118 0.1463
spher. 0.0777 0.0989 0.0873 0.1253
Log.Reg. 0.1231 0.1360 0.1477 0.1667
Original Space Mood Manifold
F1 Acc. F1 Acc.
LDA full n/a n/a 0.7340 0.7812
diag. 0.7183 0.7436 0.7365 0.7663
spher. 0.6358 0.6553 0.7482 0.7699
QDA full n/a n/a 0.6500 0.7446
diag. 0.6390 0.6398 0.6704 0.7510
spher. 0.6091 0.6143 0.7472 0.7734
Log.Reg. 0.7350 0.7624 0.7509 0.7857
uments and selected 1,346,937 documents featuring the most popular 32 emotion labels (in respect
to the number of documents annotated in).
In Figure 1, we compare our model to Watson and Tellegen’s wel l known psychological model.
We make the following observations. First, the horizontal a xis expresses a sentiment polarity-like
emotion. The left part features emotions such as accomplished, happy and excited, while
the right part features emotions such as sad and depressed. This is in agreement with Watson
and Tellegen’s observations. Second, the vertical axis exp resses the level of mental engagement or
energy level. The top part features emotions such as exhausted or tired, while the bottom
part features emotions such as curious or excited. This agrees partially with the engagement
dimension in the psychological model. Third, the neutral mo ods blank, stay in the middle of the
picture. This agreement between our mood manifold and the ps ychological ﬁndings is remarkable
in light of the fact that the two models used completely diffe rent experimental methodology (blog
data vs. surveys).
We performed emotion classiﬁcation experiment (Table 1, left) on the Livejournal data. We consid-
ered the goal of predicting the most popular 32 moods. We also considered a binary classiﬁcation
tasks obtained by partitioning the set of moods into two clusters.
Table 1 compare classiﬁcation results using the original bag of words feature space and the manifold
model, using different types of classiﬁcation methods: LDA, QDA with different covariance matrix
models, and logistic regression. Bold faces are improvemen ts over the baseline with statistical
signiﬁcance of t-test of random trials. Most of experimental results show th at the mood manifold
model results in statistically signiﬁcant improvements than using original bag of words feature.
4 Summary
In this paper, we introduced a continuous representation fo r human emotions Z and constructed
a statistical model connecting it to documents X and to a discrete set of emotions Y . Our ﬁtted
model bears close similarities to models developed in the psychological literature, based on human
survey data. Several attempts were recently made at inferri ng insights from social media or news
data through sentiment prediction. It is likely that the cur rent multivariate view of emotions will
help make progress on these important and challenging tasks.
References
[1] G. Mishne. Experiments with mood classiﬁcation in blog p osts. In 1st Workshop on Stylistic Analysis Of
Text For Information Access, 2005.
[2] D. Watson and A. Tellegen. Toward a consensual structure of mood. Psychological bulletin, 98(2):219–
235, September 1985.
3


Peer review:
[Review]:
This paper proposes a new method for sentiment analysis of text
documents based on two phases: first, learning a continuous vector
representation of the document (a projection on the mood manifold) and
second, learning to map from this representation to the sentiment
classes. The assumption behind this model is that such an intermediate
smooth representation might help the classification, especially in the case where the number of sentiment classes is rather large (32) as it is studied here.

The idea of modeling the relationship between emotions labels (Y) and
documents (X, encoded using bag-of-words) via an intermediate
representation (Z) is appealing and seems to be a good direction to pursue.

The main idea of the present model is to build a kind of two-layers
network(X->Z->Y), where each layers has its own architecture and learning
process and is trained in a (weakly) supervised way. Unfortunately, it is not
exactly clear how this training works. On one hand, the layer X->Z is trained via maximum likelihood, setting the supervision on Z via least-square regression for Z->Y (X and Y are known). But on the other hand, it is written that the layer Z->Y is obtained via MDS or kernel PCA. This is a bit puzzling.

I also think that the dimension of the manifold (l) should be given.

There is a lack of references (maybe due to the page limit) Still,
(Glorot et al., ICML11), (Chen et al., ICML12) or (Socher et al.,
EMNLP11) should be discussed since all these papers presents neural
network archiecture for sentiment analysis and basically learn an
intermediate representation of documents.

Pros:
- interesting setting with weak supervision
- new data

Cons:
- many unclear points
- lack references