Venue: NAACL 2018

# Fine-grained Temporal Orientation and its Relationship with

Psyachi Kamila\({}^{1}\), Mohammed Hasanuzzaman\({}^{2}\), Asif Ekbal\({}^{1}\),

Pushpak Bhattacharyya\({}^{1}\) and Andy Way\({}^{2}\)

###### Abstract

Temporal orientation refers to an individual's tendency to connect to the psychological concepts of _past_, _present_ or _future_, and it affects personality, motivation, emotion, decision making and stress coping processes. The study of the social media users' psychodemographic attributes from the perspective of human temporal orientation can be of utmost interest and importance to the business and administrative decision makers as it can provide an extra precious information for them to make informed decisions. In this paper, we propose a very first study to demonstrate the association between the sentiment view of the temporal orientation of the users and their different psycho-demographic attributes by analyzing their tweets. We first create a temporal orientation classifier in a minimally supervised way which classifies each tweet of the users in one of the three temporal categories, namely _past_, _present_, and _future_. A deep Bi-directional Long Short Term Memory (BLSTM) is used for the tweet classification task. Our tweet classifier achieves an accuracy of 78.27% when tested on a manually created test set. We then determine the users' overall temporal orientation based on their tweets on the social media. The sentiment is added to the tweets at the fine-grained level where each temporal tweet is given a sentiment with either of the positive, negative or neutral. Our experiment reveals that depending upon the sentiment view of temporal orientation, a user's attributes vary. We finally measure the correlation between the users' sentiment view of temporal orientation and their different psycho-demographic factors using regression.

## 1 Introduction

The rapid growth of social media data in recent years has encouraged different studies which only existed at the psychological level (theory or pure logic). Various attributes of users can be analyzed from the texts they write on the social media platform. The studies include age, gender prediction Marquardt et al. (2014); Sap et al. (2014), psychological well being Dodds et al. (2011); Choudhury et al. (2013), and a host of other behavioural, psychological and medical phenomena Kosinski et al. (2013). However, a few works exist which analyze these factors using socio-economic characteristics of the Twitter users.

Time is generally defined by a dimension where the events are ordered from the past through the _present_ into the _future_ which includes duration and intervals. Major studies on time have been done for event detection Ihler et al. (2006); Batal et al. (2012); Sakaki et al. (2013) which are mainly of the subjective consent. In contrast, temporal orientation of a user is defined by his/her tendency to emphasize _past_, _present_ or _future_Zimbardo and Boyd (2015) which gives more objective consent of time. The growth of social media content has enabled us to study this objective consent more precisely.

Past studies have established a consistent link between the temporal orientation and several user characteristics such as age, gender, education, and psychological traits Webley and Nyhus (2006); Adams and Nettle (2009); Schwartz et al. (2013); Zimbardo and Boyd (2015). However, the sentiment dimension (positive, negative or neutral) of the temporal orientation is merely studied at the empirical level on a large-scale. For example, people who are optimistic are future-oriented and positive at the same time. So, only defining the temporal orientation cannot specify the optimistic people correctly. We need the sentiment dimension as well to find the exact correlation.

In this paper, we first develop a temporal-orientation classifier to classify tweets into _past_, _present_, and _future_ and then group over the users

to create user-level assessments. We use a Bidirectional Long Short Term Memory (Bi-LSTM) network for tweet temporal classification where tweet vectors are fed to generate the classification model. We propose a hash tag-based _minimally supervised_ method with the two-pass filtering to create the _past_, _present_ and _future_-oriented tweets for the training of the Bi-LSTM network. We manually examined trending hashtags in Twitter for a specific period of time and selected hashtags which represent _past_, _present/ongoing_, or _future_ events. The English tweets containing one of the selected hashtags are crawled using Twitter streaming API.1 The tweet temporal orientation classifier is validated on a manually annotated test set. Finally, we use this classifier to automatically classify a large dataset consisting of \(\approx\)10 million tweets from 5,191 users mapped to their user-level features.

Footnote 1: [https://dev.twitter.com/streaming/overview](https://dev.twitter.com/streaming/overview). Last accessed on 10-01-2018.

Besides these three temporal categories (_past_, _present_ or _future_), we have considered the positive, negative and neutral sentiments of the tweets for the fine-grained classification. The user-level tweets with a particular temporal orientation is further subdivided into either positive, negative or neutral sentiment. Finally, we evaluated whether the sentiment view of temporal orientation (i.e. past-positive, past-negative, past-neutral, present-positive, present-negative, present-neutral, future-positive, future-negative, and future-neutral) of the users is related to their several psycho-demographic attributes. In this research, we have considered five psycho-demographic attributes, namely age, education, relationship, intelligence, and optimism.

Our contributions are summarised as below:

* We introduce the sentiment dimensions in the human temporal orientation to infer the social media users' psycho-demographic attributes on a large-scale.
* We propose a minimally supervised approach to the temporal orientation classification task that leverages large quantities of unlabeled data and requires no hand-annotated training corpora. The empirical evidence shows that the method performs reasonably well.
* We create a gold-standard temporal orientation tweet corpus.
* We define a way to find a novel association between the sentiment view of temporal orientation and the different psychom demographic factors of the tweet users.

## 2 Related Background

The temporal study has recently received an increased attention in several application domains of Natural Language Processing (NLP) and Information Retrieval (IR). The introduction of the TempEval task Verhagen et al. (2009) and the subsequent challenges i.e. TempEval-2 and -3 Verhagen et al. (2010); UzZaman et al. (2013) in the Semantic Evaluation workshop series have clearly established the importance of time in dealing with the different NLP tasks. Alonso et al. (2011) reviewed the current research trends and presented a number of interesting applications along with the open problems.

The shared task like the NTCIR-11 Temporaliak Joho et al. (2014) further pushed this idea and proposed to distinguish whether a given query is related to _past_, _recency_, _future_ or _atemporal_. It is the first such challenge, which is organized to provide a common platform for designing and analyzing the time-aware information access systems. In parallel, new trends have emerged in the context of the human temporal orientation Schwartz et al. (2013); Sap et al. (2014); Park et al. (2015); Schwartz et al. (2015); Park et al. (2017). The underlying idea is to understand how the past, present, and future emphasis in the text may affect people's finances, health, and happiness. For that purpose, the temporal classifiers are built to detect the overall temporal dimension of a given sentence. For instance, the following sentence "_can't wait to get a pint tonight_" would be tagged as _future_.

In summary, most of the temporal text processing applications have been mainly relying on the rule-based time taggers, for e.g. HeidelTime Strotgen and Gertz (2015) or SUTime Chang and Manning (2012) to identify and normalize time mentions in the texts. Although interesting results have been reported UzZaman et al. (2013), but the coverage is limited to the finite number of rules they implement.

The time perspective and its importance in various social science and psychological studies is well established in literature. It plays a fundamental role in our interpersonal relation influenced by cognitive process Zimbardo and Boyd (2015).

 

This is also useful in forming goals, expectations and imaginations. Time perspective is a fundamental process, which intern, influenced by many user attributes such as age, religion, education etc. In their research, Zimbardo and Boyd (2015) have shown that the negative view of the past is related to depression, anxiety, unhappiness, and low self-esteem but the positive view of the past is related to self-esteem and happiness. The hedonistic view of the present is related to novelty seeking and sensation seeking whereas the fatalistic view of the present is related to aggression, anxiety and depression. The future is related to conscientiousness but negatively correlated with depression and anxiety.

Another research suggests that the satisfaction with life of the older adults depends on their positive views of past Kazakina (1999). In their research, Drake et al. (2008) described that the past-positive is positively correlated to happiness. The link between the past-negative and many psychological distress like depression and anxiety has been well established in literature Cully et al. (2001). A focus on the future is very effective for functioning positively. The future orientation also helps in better health in later life Kahana et al. (2005). In a research, George (2009) evaluated that subjective well-being, happiness, psychological well-being, positive effects and morale refer to the positive orientation towards life.

Past research has established that the time perspective is an important factor to determine the human emotional intelligence Stolarski et al. (2011). In our work, we measure the relationship with different level of intelligence and the sentiment view of the temporal orientation by more objective consent of the time perspective, i.e. temporal orientation from the tweets on the social media. In a social science research, Guthrie et al. (2009) have shown that the future time perspective is associated with the current socioeconomic status, and the past-fatalistic time perspective is associated with the both current and childhood socioeconomic status.

Although these kinds of research exist extensively in the psychological study, it is not well explored with the empirical study using more objective consent of the time perspective, i.e the temporal orientation. As per our best knowledge, only a very few studies exist that focus on the temporal orientation where only the coarse-grained classes have been considered Schwartz et al. (2015); Park et al. (2017). In these researches, many user attributes were correlated with the temporal orientation which include conscientiousness, age, gender, openness, extraversion, agreeableness, neuroticism, satisfaction with life, depression, IQ, number of friends etc. In our work, we incorporated fine-grained temporal orientation and found the correlation with the users' age, education, relationship, intelligence, and optimism. The fine-grained study of the temporal orientation only existed at the theoretical level validated with very limited user dataset. Besides validating these findings in the empirical way for the large number of users, we also discuss some previously unexplored relationships.

## 3 Methodology

We first create a deep temporal-orientation classifier to capture the temporal orientation (_past_, _present_ and _future_) of the users' tweets. Thereafter we further classify the users' tweets at the fine-grained level by associating sentiment, i.e. positive, negative or neutral for each temporal category. We compare our temporal-orientation classifier with an existing state-of-the-art method.

### Temporal Orientation Classification

The temporal orientation of tweets is defined by classifying each tweet T in one of the temporal categories t, where \(t\in\{\textit{past}\text{, present}\text{, or }\textit{future}\}\). Given the following tweet _"Let me change lanes and turn left legally"_, the temporal orientation classifier should predict it as an instance of _future_ orientation. At first we create a temporal oriented tweet dataset in a minimally supervised way by exploiting the hashtag information. Deep Bi-LSTM network is then trained on this dataset. We use LSTM networks Hochreiter and Schmidhuber (1997) as these are well known for capturing the long-term dependencies within the text.

Many times we fail to capture the temporal orientation of a text using just the tense information or the existing temporal keywords. In particular, the tweet _"Today I have a plan for a meeting at night."_ is future-oriented. Here, the temporal keyword _'Today'_ has a time sense of _present_ whereas the tense of the verb is also _present_. The deep learning networks have been very useful to correctly capture the temporal dimension of these kinds of tweets. Although the basic Artifi 

cial Neural Networks (ANNs) Schalkoff [1997] and Convolutional Neural Network (CNN) LeCun et al. [1995] capture the temporal orientation of many tweets correctly, they fail to properly identify where the validating temporal information in the tweet has a long dependency between them. For example, the tweet _"Working in the same unit today with different staff was much better."_ has temporal orientation as _past_. Here, the word which has a temporal sense (_i.e. working, today, was_) are placed at a distance from each other. This motivates us to use the LSTM network.

Bidirectional Long Short Term Memory Networks (Bi-LSTM):LSTMs are a special kind of recurrent neural network (RNN) capable of learning long-term dependencies in the text by effectively handling the vanishing or exploding gradient problem. The Bidirectional LSTMs Schuster and Paliwal [1997] train two LSTMs, instead of one, on the input sequence. The first on the input sequence and the second on a reversed copy of the input sequence. It is designed to capture information of the sequential dataset and maintain the contextual features from the past and the future. This can provide an additional context to the network and result in faster and even fuller learning on the problem without keeping the redundant context information.

The previous study on the temporal orientation classification based on machine learning includes a supervised classification based on the manually created training set Schwartz et al. [2015]. The multi-class classification was based on a one vs rest approach. But adapting the multiple binary classifiers is not always the best way to deal with a multi-class classification problem. It requires building of three independent classifiers for each temporal category, which consumes more time. Unlike this approach, we incorporate a deep learning-based multi-class classification method for the temporal orientation. The training corpus is generated in a minimally supervised way and fitted to the Bi-LSTM network.

Our experiment uses Bi-LSTM with 200 neurons at the input layer. The loss function we used is _categorical cross-entropy_ and the optimizer used is _Root Mean Square Propagation (rmsprop)_. We repeat the training for 100 _number of epochs_ with _batch size_ set to 128. We also employ dropout Srivastava et al. [2014] for regularization with a _dropout rate_ of 0.2 to prevent over-fitting. All of these attributes are finalized by parameter tuning with the performance obtained on 10-fold cross-validation using the grid search method. Tweet vectors are generated by existing Glove vectors Pennington et al. [2014] for tweets2 of 200 dimensions which are trained on 27 billion tweets. We also validate our model on the validation set which was 10% of the training set.

Footnote 2: [https://nlp.stanford.edu/projects/glove/](https://nlp.stanford.edu/projects/glove/)

### Sentiment View of Temporal Orientation

We use an existing sentiment classifier available with the NLTK toolkit Bird [2006] to classify the _user-level_ tweets into positive, negative or neutral.3 Sentiment is added at the fine-grained level of the temporal orientation. Given the tweets of a user, the sentiment view of temporal orientation of that user is defined by the following equation:

Footnote 3: As we are using a sentiment classifier from well known NLTK library, we are not validating it on a manually-tagged test set.

\[orientation_{s,t}(user)=\frac{|twects_{s/t}(user)|}{|tweeds_{t}(user)|} \tag{1}\]

where, (\(t\in\{\textit{ past, present, or future}\}\)), and (\(s\in\{\textit{ positive, negative, or neutral}\}\)), in equation (1). Here, we first classify each user's tweet into the past, present or future temporal category. Then for each temporal category, we find the percentage of each sentiment class (i.e positive, negative or neutral) to obtain the sentiment view of temporal orientation.

We measure the correlation between a user's sentiment view of temporal orientation with their

Figure 1: Temporal Orientation learning architecture.

 

age, education, intelligence, optimism, and relationship using a linear regression (LR) classifier [14].

## 4 Data Sets

For experiments we categorize the datasets into three kinds: training, test and user-level. Training set consists of 27k tweets, whereas the test set is manually annotated with 741 tweets.4 The user-level tweets consist of \(\approx\)10 million tweets from 5,191 users mapped to their user-level features.

Footnote 4: All the developed resources are available at [http://www.iitp.ac.in/~ai-nlp-ml/resources.html](http://www.iitp.ac.in/~ai-nlp-ml/resources.html)

### Training Set

Training tweets are collected using the Twitter streaming API.5 The tweets are collected for the duration of September 2017 and October 2017. We consider day-wise trending topics during this period.6 We only consider those hashtags which signify a temporal event. Finally, we chose worldwide trending events and collected the tweets based on the hashtags.

Footnote 5: [https://developer.twitter.com/en/docs](https://developer.twitter.com/en/docs)

The collection of the temporal tweets are based on the following three hypotheses: (a) if a trending topic is of a future event then mostly people would write the futuristic tweets; (b) if a trending topic is about a past incident, then the people would write more about the past but they also write about the present effects of that event; (c) the tweets of trending present event are most critical to handle as besides writing about the present incidents, people always join the links with the past incidents and also give opinion about the future effects.

The task was challenging as the tweets contain a lot of noises and people use various ways to refer to the past, the present and the future. To deal with the pitfalls described in the hypotheses, we filter the tweets using a two-pass filtering method. The method is based on two assumptions (a) every meaningful sentence should contain a verb. (b) mostly past-oriented tweets have tense of the verb as past.

The first assumption is well-established in literature, whereas the second assumption is based on our observation on the tweets and validation against a tense-based classifier. In the first pass of the filtering method, we filter out the tweets which do not contain a verb. The verb part-of-speech tag is determined using the CMU tweet-tagger [38]. In the second pass of the filtering method, we removed the tweets having tense as past from the tweets of the present and future events.

The CMU tweet-tagger does not provide verbs in different sub-categories. For this reason, we also retrieve the Part-of-Speech (PoS) tag information from the Standford PoS-dagger [10] for all the tweets to get the sub-categories of verb (i.e. VB, VBD, VBG, VBN, VBP, VBZ). We observed that although Standford PoS-dagger assigned the required verb subcategories, it also incorrectly tagged some nonverbs as verbs. This is the reason why we considered only those verbs for sub-categorization which were identified (as verbs) by the CMU tweet-tagger.

We varied the training set starting from 3K (equally distributed) to 30K and observed that the accuracy on the gold standard test set did not improve after 27K training instances. Few example tweets with the trending topics are depicted in Table 1.

### Test Set

We evaluate our temporal-orientation classifier on a manually created test set. To get proper assessment on the user-level test set, we randomly selected 800 tweets from the user-level test tweets. Three annotators (post-graduate level students) were asked to tag the tweets in one of the four available classes, namely _past_, _present_, _future_ and _other_. The annotation guidelines were as follows:

* Tag a tweet as _past_ if it talks about an event which has started as well as ended or the underlaying temporal connotation of the tweet refers to the _past_ time.
* Tag a tweet as _present_ if it talks about an event which started but not ended yet or the tweet has a _present_ temporal connotation,
* Tag a tweet as _future_ if it talks about an event which is yet to happen.
* Tag a tweet as _other_ in case they found it difficult to get the exact temporal tag for the tweets.

 

We measured the multi-rater kappa agreement (Fleiss, 1971) among the annotators and it was found to have a substantial kappa value of 0.82. The higher kappa value indicates that associating text with temporal dimensions, namely, _past, present, future,_ and _other_ is relatively straightforward task for humans by using world knowledge than words (Dias et al., 2014). Moreover, our inter-annotator agreement value is in line with the literature.7 Finally, we select the temporal class of a tweet based on the majority voting among the annotators. The distribution of annotated tweets is as follows:

Footnote 7: Inter-annotator agreement value for the same task in Schwartz et al. (2015) is 0.83.

* _Past_- 375 Tweets
* _Present_- 164 Tweets
* _Future_- 202 Tweets
* _Other_- 59 Tweets

We removed tweets tagged as _Other_ and used 741 tweets as the test set.8

Footnote 8: We only considered _past_, _present_ and _future_ classes for the reason justified in Schwartz et al. (2015).

### User-level Test Set

The user-level tweets consist of \(\approx\)10 million tweets from 5,191 users mapped to their user-level psycho-demographic features developed by Preotiuc-Pietro et al. (2015) are used for this current work. In particular, we use five psychodemographic attributes such as _age_, _education_, _intelligence_, _optimism_, and _relationship_ for our experiment. The users' psycho-demographic features are automatically deduced based on the users' published texts. Preotiuc-Pietro et al. (2015) used a predictive model to automatically infer user-level features. The method uses various user properties (annotated using crowdsourcing) including age, gender, income, education, relationship status, optimism and life satisfaction as well as all the tweets published by a user to infer user-level features.

## 5 Experimental Results

We first evaluate our temporal orientation classifier which measures the orientation of each tweet as either of _past_, _present_ or _future_. The classifier was trained on the training set and evaluated on the test set. We obtain the highest accuracy of 78.27% over 741 test samples. For comparative evaluation, we can consider a strong baseline system proposed by Schwartz et al. (2015). The baseline system was built following a supervised learning strategy over different features such as ngrams, time expression, PoS tags, tweet length, and temporal class-specific lexicons. The system achieved an accuracy of 71.8% when tested over 500 manually annotated data. The baseline was not reproducible as both the training and test set were manually tagged and the datasets were not available.9 The baseline model was constructed using the manually annotated data, creation of which involved considerable efforts and expenses. In contrast, we follow a minimally supervised method (does not incur any manual effort) to create our own datasets which are of acceptable quality. We show the results in Table 2.

[TABLE:

\begin{table}
\begin{tabular}{|c|c|c|} \hline
**Temporal Orientation** & **Hashtag** & **Example Sentence** \\ \hline \multirow{2}{*}{ Past} & \#Electoens2016 & did it have influence? of course it did. \\  & \#CC17 & just heard gazza made a guest appearance outside the story conference. \\ \hline \multirow{2}{*}{ Present} & \#HappyHallaween & when you leave for work early but altanta traffic has other ideas. \\ \cline{2-3}  & \#LHH & i am trying to figure out who this is \\ \cline{2-3}  & \#WorldTeachersDay & hats off to all the teachers who work hard to not only educate but protect kids everyday. \\ \hline \hline \multirow{2}{*}{ Future} & \#U17WC & 2017 fita u17 world cup starts in 3 days \\ \cline{2-3}  & \#2PointO & gonna treat us with 3d visual extravaganza! \\ \cline{2-3}  & \#FutureDecoded & want to get your hands on a new? enter our giveaway at for your chance to win. \\ \hline \end{tabular}
\end{table}
Table 1: Example tweets for different temporal orientation categories with trending topics.

\begin{table}
\begin{tabular}{|c|c|c|c|} \hline
**Orientation** & **Precision** & **Recall** & **F1-measure** \\ \hline Past & 81.75 & 92.0 & 86.57 \\ \hline Present & 79.04 & 50.61 & 61.71 \\ \hline Future & 71.02 & 75.24 & 73.07 \\ \hline \end{tabular}
\end{table}
Table 2: Precision, Recall and F1-measure of our proposed temporal orientation classification model on manually annotated test data.

 

Results in Table 2 show that the past class is the most correctly classified followed by the future and the present. We observe low recall for the _present_ class as many present tweets were mis-classified into either _past_ or _present_ class. The confusion matrix is shown in Figure 2.

The present class is mis-classified into future when the tweet is of the declarative type. For example, the tweet "_Its not a casserole as there's no binding matrix_" has _present_ orientation but our classifier classifies it as of _future_ orientation. Another reason could be the fact that the words in the sentence representing _present_ temporal orientation are not in the correct form (Its, theres). The _present_ classes are mis-classified into the past classes in those cases where mainly the existence of the tense of a verb is _past_ but actually the tweet has _present_ orientation. For example, the tweet "_For me gloves and mits made for Cross Country skiing work well for ventilated warmth_" is mis-classified into _past_ because of the existence of the word (made) which has tense as _past_. The tweets with _future_ orientation are mostly mis-classified into _past_ orientation. These kinds of mis-classification is due to either for the presence of past tense or the tweet is a compound sentence which has an independent clause of the past orientation. For example, the tweet "_Hopping to have fun among my friends but wishing I were with you instead_" has a future orientation but it is mis-classified into the past orientation.

We measure the potential limitations of the NLTK sentiment classifier on 100 randomly selected tweets from the test set. The manual observation shows that the classifier generally mis-classifies where the sentiment of the tweet is not well understood (example: "Big Trucks parked all over). In some cases, the tweets having conflict sentiment are mis-classified in either of positive or negative class. For example, the tweet "I am very sorry that is a working weekend for me but thanking you very much for the invitation has a conflict sentiment but the classifier classified it into an instance of negative sentiment.

We measure the predictive power of the sentiment view of temporal orientation by performing regression on different psycho-demographic factors. The correlation results between the users' sentiment view of temporal orientation and their psycho-demographic factors using linear regression are presented in Table 3 and Table 4. The performance is measured using a standard metric, namely _Pearson's correlation coefficient r_ between the inferred and the target values. All the results in Table 3 and Table 4 are statistically significant when tested against null hypothesis (\(p\_value<\)0.05).

## 6 Discussion and Findings

We measure the correlation between the Twitter users' psycho-demographic features and their sentiment view of temporal orientation. In this section, all the discussions and analyses are based on the correlation results over the user test set.

### Demographic Correlates

We select 'age', 'education' and 'relationship' as demographic features for this study. The correlation coefficients between the users' demographic attributes and their sentiment view of temporal orientation are shown in Table 3.

Results in Table 3 demonstrate that any user's past-orientation is significantly correlated (0.4677) with their age. In other words, it suggests that when people age they think more about the past than the present and future. To the best of our knowledge, psychology literature Nurmi (2005); Steinberg et al. (2009) has not established the correlation of the past orientation with age. Our finding is consistent with a recent computational study on the human temporal orientation Schwartz et al. (2015) which shows positive correlation between age and the past orientation. However, we also observed that the users' age has the highest positive correla

Figure 2: Confusion matrix for the temporal orientation classification.

 

future-positive. It indicates that people become positively future-orientated when they age, though not surprising, yet somewhat novel. The results indicate that only considering the temporal orientation without the sentiment dimensions can be sometimes misleading as we can observe that the negative future-orientation has a negative correlation (-0.4547) with age while the future-positive has a positive correlation with age.

Figure 3 explains how the trends of the sentiment view of temporal orientation varies from age 10 to 60. We observe that for all the temporal classes, the positive sentiment increases rapidly with the increase of age. Most interestingly, for all the temporal orientation people become negative up to the age of 28 and then their negative sentiment steadily reduces. We also observe that human's neutral sentiment rapidly decreases up to the age of 27 and then it decreases steadily.

The second demographic attribute we considered is 'education'. We measure the correlation between the temporal orientation and three different levels of education: _degree_, _graduate_degree_, and _high_school_. In the psychological literature Horstmanshof and Zimitat (2007); Richardson et al. (2012), it was reviewed that the students' temporal orientations is a new dimension to approaches in enhancing student engagement in the academics. It was found that the first year students of university were more _future-oriented_ rather than present or past oriented. From our results in Table 3, we found that the users who have education of _degree_ level are _present-oriented_. But interestingly they neither think positively nor negatively-they express more neutral sentiment. Users with education of _graduate_degree_ are found to be _future-oriented_. Here, the fine-grained classification suggests that they also express the _neutral_ sentiment. Interestingly, we found that the users with education of _high_school_ had _positive_ correlation with _past orientation_. However, when we considered the sentiment dimension, we found that it was actually correlated with _present orientation_ with _negative sentiment_.

Our third and final demographic feature 'relationship' is categorized into four types in our current study: _divorced_, _in_a_relationship_, _married_, and _single_. From the results in Table 3, we observe that the users who are _divorced_ found to be more future oriented and they seem to express the neu

\begin{table}
\begin{tabular}{|c|c c c c c c c c c c c c|} \hline
**User Attribute** & **Past** & **Post-Pos** & **Post-Neg** & **Past-Newn** & **Frestant** & **Frestal-Pos** & **Present-Nag** & **Freesent-Net** & **Future-Pos** & **Future-Neg** & **Future-Ven** \\ \hline Interlinescoiler number & 0.9654 & -0.0680 & 0.0742 & 0.0829 & 0.0370 & -0.0691 & **0.0885** & 0.0240 & 0.0865 & -0.0188 & 0.0929 & 0.091 \\ \hline Interlineperscope & **0.1777** & 0.1423 & 0.0996 & -0.1725 & 0.0327 & 0.1604 & 0.1105 & -0.1868 & -0.1694 & **1.897** & 0.1277 & 0.1904 \\ \hline Interlineperscope method.above & -0.2946 & -0.2466 & -0.2123 & 0.3198 & -0.0333 & -0.2928 & -0.2451 & 0.3590 & **0.2778** & -0.3122 & -0.2736 & **0.3625** \\ \hline Optimism.optimism & 0.0696 & 0.1197 & -0.0173 & -0.1212 & 0.0997 & 0.1486 & -0.0144 & -0.1245 & 0.0666 & **0.1604** & -0.0025 & -0.1417 \\ \hline Optimism.minim segment & 0.0536 & -0.0600 & 0.0659 & -0.0167 & 0.0116 & 0.0430 & **0.1815** & -0.0975 & -0.0420 & -0.0115 & 0.1708 & -0.0164 \\ \hline Optimism.neather & -0.0550 & -0.1027 & 0.0014 & 0.0944 & -0.0033 & -0.1132 & -0.0010 & 0.0999 & 0.0504 & -0.1224 & -0.0103 & 0.1119 \\ \hline \end{tabular}
\end{table}
Table 4: Correlation between users sentiment view of temporal orientation and their different flexophographic features using LR. Here, pos-positive, neg-negative, neu-neutral.

Figure 3: Standardized sentiment view of temporal orientation of the users over their age. Smoothing was done using loess smoothing estimates. Here, poss-positive, neg-negative, neu-neutral.

 

tral sentiment. The _in_a_relationship_ users seem to be more past-oriented. They also found to be negative minded. Married people are found to be present oriented while expressing the neutral sentiment. The users who are single, are generally futuristic but they are negative about it.

### Psychological Correlates

We chose two psychological factors, intelligence and optimism. The correlation coefficient results are shown in Table 4.

The intelligence level of the users was measured in three sub-categories: _intelligence:below_average_, _intelligence:average_ and _intelligence:much_above_. Some novel findings have been observed through our results. We found a modest yet significant positive correlation between intelligence below_average and negative view of the present orientation. It suggests that the users having intelligence below average are present oriented but they seem to have negative view of it. Surprisingly, we found that average intelligent users are past-oriented but considering the sentiment dimension they seem to be more future-positive. However, this should be validated with further investigation. We found that the users who have intelligence much_above are more future orientated. Interestingly, we found a negative correlation with the future-positive. However, we found a positive correlation (0.3614) with the future-neutral which suggests that the users with much above intelligence are futuristic and they express a neutral view.

We chose three categories of optimism: optimistic, pessimistic, and neither for our observation. The result shown in Table 4 suggests that the optimistic people are future oriented. They also seem to have positive sentiment. Although the link between the future orientation and optimism is well established in literature Lennings (2000); Busseri et al. (2013), there is no empirical study for a large number of users. We find a relatively higher positive correlation between the pessimist and the present-negative which suggests that the pessimistic people are negative minded and focus more on present. People who are neither optimistic nor pessimistic are found to be future oriented with the neutral sentiment which is also a novel finding.

## 7 Conclusion and Future Work

This paper presents a first large-scale study to associate the psycho-demographic profile of the Twitter users with their sentiment view of temporal orientation based on language they use in Twitter. We first detect the temporal orientation of the tweets using the Bi-LSTM based temporal orientation classifier. We generated the temporal categories of our training set in a minimally supervised way. We created a benchmark dataset for the evaluation of our temporal orientation classifier. The temporal orientation classifier achieved an accuracy of 78.27% when run on the manually tagged test data. We added the sentiment dimension at the fine-grained level of the temporal orientation. The associations between the users' sentiment view of temporal orientation and their different psycho-demographic attributes (age, education, intelligence, optimism, and relationship) are somewhat novel in the context of the computational social science studies. Whereas the study on the temporal orientation concentrated on a coarse-grained level, we focused on the fine-grained level of temporal orientation which opens more aspects of the social, economic, and psychological research which was not possible previously on a large scale.

Acknowledging the possible limitations of this study including the quality of the sentiment classifier and a low recall of the present temporal orientation, in future, we will consider more sophisticated sentiment classifier for better performance and explore more linguistic insight into consideration to improve the performance of the temporal orientation classifier. We also like to extend our work with the link to more behavioral study and analysis.

## 8 Acknowledgments

Asif Ekbal acknowledges Young Faculty Research Fellowship (YFRF), supported by Visvesvaraya PhD scheme for Electronics and IT, Ministry of Electronics and Information Technology (MeitY), Government of India, being implemented by Digital India Corporation (formerly Media Lab Asia). Mohammed Hasanuzzaman and Andy Way would like to acknowledge ADAPT Centre for Digital Content Technology, funded under the SFI Research Centres Programme (Grant 13/RC/2106) and is co-funded under the European Regional Development Fund 



## References

* Adams and Nettle (2009) Jean Adams and Daniel Nettle. 2009. Time Perspective, Personality and Smoking, Body Mass, and Physical Activity: An Empirical Study. _British journal of health psychology_ 14(1):83-105.
* Alonso et al. (2011) Omar Alonso, Jannik Strotgen, Ricardo A. Baeza-Yates, and Michael Gertz. 2011. Temporal Information Retrieval: Challenges and Opportunities. In _WWW2011 Workshop on Linked Data on the Web_. Hyderabad, India, pages 1-8.
* Batal et al. (2012) Iyad Batal, Dmitriy Fradkin, James H. Harrison Jr., Fabian Moerchen, and Milos Hauskrecht. 2012. Mining Recent Temporal Patterns for Event Detection in Multivariate Time Series Data. In _The 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_. Beijing, China, pages 280-288.
* Bird (2006) Steven Bird. 2006. NLTK: The Natural Language Toolkit. In _Proceedings of the COLING/ACL 2006 Interactive Presentation Sessions_. Sydney, Australia, pages 69-72.
* Buseri et al. (2013) Michael A Buseri, Annette Malinowski, and Becky L Choma. 2013. Are Optimists Oriented Uniqued Toward the Future? Investigating Dispositional Optimism from a Temporally-expanded Perspective. _Journal of Research in Personality_ 47(5):533-538.
* Chang and Manning (2012) Angel X. Chang and Christopher D. Manning. 2012. SUTime: A Library for Recognizing and Normalizing Time Expressions. In _Proceedings of the Eighth International Conference on Language Resources and Evaluation_. Istanbul, Turkey, pages 3735-3740.
* De Choudhury et al. (2013) Munmun De Choudhury, Scott Counts, and Eric Horvitz. 2013. Predicting Postpartum Changes in Emotion and Behavior via Social Media. In _2013 ACM SIGCHI Conference on Human Factors in Computing Systems_. Paris, France, pages 3267-3276.
* Cully et al. (2001) Jeffrey A Cully, Donna LaVoie, and Jeffrey D Gfeller. 2001. Reminiscence, Personality, and Psychological Functioning in Older Adults. _The Gerontologist_ 41(1):89-95.
* Dias et al. (2014) Gael Harry Dias, Mohammed Hasanuzzaman, Stephane Ferrari, and Yann Mathet. 2014. TempoWordNet for Sentence Time Tagging. In _Proceedings of the 23rd International Conference on World Wide Web_. New York, USA, pages 833-838.
* Dodds et al. (2011) Peter Sheridan Dodds, Kameron Decker Harris, Isabel M Kloumann, Catherine A Bliss, and Christopher M Danforth. 2011. Temporal Patterns of Happiness and Information in a Global Social Network: Hedonometrics and Twitter. _PloS one_ 6(12):1-26.
* Drake et al. (2008) Lisa Drake, Elaine Duncan, Fi Sutherland, Clare Abernethy, and Colette Henry. 2008. Time Perspective and Correlates of Wellbeing. _Time & Society_ 17(1):47-61.
* Fleiss (1971) Joseph L Fleiss. 1971. Measuring Nominal Scale Agreement among Many Raters. _Psychological Bulletin_ 76(5):378-382.
* George (2009) Linda K George. 2009. Still Happy after all these Years: Research Frontiers on Subjective Wellbe being in Later Life. _Journals of Gerontology Series B: Psychological Sciences and Social Sciences_ 65(3):331-339.
* Gimpel et al. (2011) Kevin Gimpel, Nathan Schneider, Brendan O'Connor, Dipanjan Das, Daniel Mills, Jacob Eisenstein, Michael Heilman, Dani Yogatama, Jeffrey Flanigan, and Noah A. Smith. 2011. Part-of-Speech Tagging for Twitter: Annotation, Features, and Experiments. In _The 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies_. Portland, Oregon, USA, pages 42-47.
* Guthrie et al. (2009) Lori C Guthrie, Stephen C Butler, and Michael M Ward. 2009. Time Perspective and Socioeconomic Status: A Link to Socioeconomic Disparities in Health? _Social science & medicine_ 68(12):2145-2151.
* Hochreiter and Schmidhuber (1997) Sepp Hochreiter and Jurgen Schmidhuber. 1997. Long Short-term Memory. _Neural computation_ 9(8):1735-1780.
* Horstmanshof and Zimitat (2007) Louise Horstmanshof and Craig Zimitat. 2007. Future Time Orientation Predicts Academic Engagement among First-year University Students. _British Journal of Educational Psychology_ 77(3):703-718.
* Ihler et al. (2006) Alexander T. Ihler, Jon Hutchins, and Padhraic Smyth. 2006. Adaptive Event Detection with Time-varying Poisson Processes. In _Proceedings of the Twelfth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_. Philadelphia, PA, USA, pages 207-216.
* Joho et al. (2014)Hideo Joho, Adam Jatowt, Roi Blanco, Hajime Naka, and Shuhei Yamamoto. 2014. Overview of NTCIR-11 Temporal Information Access (Temporalia) Task. In _Proceedings of the 11th NTCIR Conference on Evaluation of Information Access Technologies_. Tokyo, Japan, pages 429-437.
* Kahana et al. (2005) Eva Kahana, Boaz Kahana, and Jianping Zhang. 2005. Motivational Antecedents of Preventive Proactivity in Late Life: Linking Future Orientation and Exercise. _Motivation and emotion_ 29(4):438-459.
* Kazakina (1999) Elena Kazakina. 1999. _Time Perspective of Older Adults: Relationships to Attachment Style, Psychological Well-being, and Psychological Distress_. Ph.D. thesis, ProQuest Information & Learning.
* Kosinski et al. (2013) Michal Kosinski, David Stillwell, and Thore Graepel. 2013. Private Traits and Attributes are Predictable from Digital Records of Human Behavior. _Proceedings of the National Academy of Sciences_ 110(15):5802-5805

Yann LeCun, Yoshua Bengio, et al. 1995. Convolutional Networks for Images, Speech, and Time Series. _The handbook of brain theory and neural networks_ 3361(10):276-279.
* Lennings [2000] CJ Lennings. 2000. Optimism, Satisfaction and Time Perspective in the Elderly. _The International Journal of Aging and Human Development_ 51(3):167-181.
* Manning et al. [2014] Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Rose Finkel, Steven Bethard, and David McClosky. 2014. The Stanford CoreNLP Natural Language Processing Toolkit. In _Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics_. Baltimore, MD, USA, pages 55-60.
* Marquardt et al. [2014] James Marquardt, Golnoosh Farnadi, Gayathri Vasudevan, Marie-Francine Moens, Sergio Davalos, Ankur Teredesai, and Martine De Cock. 2014. Age and gender identification in social media. In _Working Notes for CLEF 2014 Conference_. Sheffield, UK, pages 1129-1136.
* Neter et al. [1996] John Neter, Michael H Kutner, Christopher J Nachtsheim, and William Wasserman. 1996. _Applied Linear Statistical Models_, volume 4. Irwin, Chicago.
* Nurmi [2005] Jiari-Erik Nurmi. 2005. Thinking about and Acting upon the Future: Development of Future Orientation across the Life Span. _Understanding behavior in the context of time: Theory, research, and application_ pages 31-57.
* Park et al. [2015] Gregory Park, H Andrew Schwartz, Johannes C Eichstaedt, Margaret L Kern, Michal Kosinski, David J Stillwell, Lyle H Ungar, and Martin EP Seligman. 2015. Automatic Personality Assessment through Social Media Language. _Journal of personality and social psychology_ 108(6):934-952.
* Park et al. [2017] Gregory Park, H Andrew Schwartz, Maarten Sap, Margaret L Kern, Evan Weingarten, Johannes C Eichstaedt, Jonah Berger, David J Stillwell, Michal Kosinski, Lyle H Ungar, et al. 2017. Living in the Past, Present, and Future: Measuring Temporal Orientation With Language. _Journal of personality_ 85(2):270-280.
* Pennington et al. [2014] Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global Vectors for Word Representation. In _Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP)_. pages 1532-1543.
* Preotiuc-Pietro et al. [2015] Daniel Preotiuc-Pietro, Svitlana Volkova, Vasileios Lampos, Yoram Bachrach, and Nikolaos Aletras. 2015. Studying User Income through Language, Behaviour and affect in Social Media. _PloS one_ 10(9):1-17.
* Richardson et al. [2012] Michelle Richardson, Charles Abraham, and Rod Bond. 2012. Psychological Correlates of University Students' Academic Performance: a Systematic Review and Meta-analysis. _Psychological bulletin_ 138(2):353-387.
* Sakaki et al. [2013] Takeshi Sakaki, Makoto Okazaki, and Yutaka Matsuo. 2013. Tweet Analysis for Real-time Event Detection and Earthquake Reporting System Development. _IEEE Transactions on Knowledge and Data Engineering_ 25(4):919-931.
* Sap et al. [2014] Maarten Sap, Gregory J. Park, Johannes C. Eichstaedt, Margaret L. Kern, David Stillwell, Michal Kosinski, Lyle H. Ungar, and H. Andrew Schwartz. 2014. Developing Age and Gender Predictive Lexica over Social Media. In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing_. Doha, Qatar, pages 1146-1151.
* Schalkoff [1997] Robert J Schalkoff. 1997. _Artificial Neural Networks_, volume 1. McGraw-Hill, New York.
* Schuster and Paliwal [1997] Mike Schuster and Kuldip K Paliwal. 1997. Bidirectional Recurrent Neural Networks. _IEEE Transactions on Signal Processing_ 45(11):2673-2681.
* Schwartz et al. [2015] H Andrew Schwartz, Johannes C Eichstaedt, Margaret L Kern, Lukasz Dziurzynski, Stephanie M Ramones, Megha Agrawal, Achal Shah, Michal Kosinski, David Stillwell, Martin EP Seligman, et al. 2013. Personality, Gender, and Age in the Language of Social Media: The Open-vocabulary Approach. _PloS one_ 8(9):1-16.
* Schwartz et al. [2015] H. Andrew Schwartz, Greg Park, Maarten Sap, Evan Weingarten, Johannes Eichstaedt, Margaret Korn, Jonah Berger, Martin Seligman, and Lyle Ungar. 2015. Extracting Human Temporal Orientation in Facebook Language. In _Proceedings of The 2015 Conference of the North American Chapter of the Association for Computational Linguistics-Human Language Technologies (NAACL)_. Denver, Colorado, USA, pages 409-419.
* Srivastava et al. [2014] Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a Simple Way to Prevent Neural Networks from Overfitting. _Journal of machine learning research_ 15(1):1929-1958.
* Steinberg et al. [2009] Laurence Steinberg, Sandra Graham, Lia O Brien, Jennifer Woolard, Elizabeth Cauffman, and Marie Banich. 2009. Age Differences in Future Orientation and Delay Discounting. _Child development_ 80(1):28-44.
* Stolarski et al. [2011] Maciej Stolarski, Joanna Bitner, and Philip G Zimbardo. 2011. Time Perspective, Emotional Intelligence and Discounting of Delayed Awards. _Time & Society_ 20(3):346-363.
* Stratgen and Gertz [2015] Jannik Stratgen and Michael Gertz. 2015. A Baseline Temporal Tagger for all Languages. In _Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing_. Lisbon, Portugal, pages 541-547.
* UzZaman et al. [2013] Naushad UzZaman, Hector Llorens, Leon Deczynski, James F. Allen, Marc Verhagen, and James Pustejovsky. 2013. SemEval-2013 Task 1: TempEval-3: Evaluating Time Expressions, Events, and 

Temporal Relations. In _Proceedings of the 7th International Workshop on Semantic Evaluation, SemEval@NAACL-HLT 2013_. Atlanta, Georgia, USA, pages 1-9.
* Verhagen et al. (2009) Marc Verhagen, Robert Gaizauskas, Frank Schilder, Mark Hepple, Jessica Moszkowicz, and James Pustejovsky. 2009. The Tempeval Challenge: Identifying Temporal Relations in Text. _Language Resources and Evaluation (LRE)_ 43(2):161-179.
* Verhagen et al. (2010) Marc Verhagen, Roser Sauri, Tommaso Caselli, and James Pustejovsky. 2010. SemEval-2010 Task 13: TempEval-2. In _Proceedings of the 5th International Workshop on Semantic Evaluation, SemEval@ACL 2010_. Uppsala, Sweden, pages 57-62.
* Webley and Nyhus (2006) Paul Webley and Ellen K Nyhus. 2006. Parents Influence on Children's Future Orientation and saving. _Journal of Economic Psychology_ 27(1):140-164.
* Zimbardo and Boyd (2015) Philip G Zimbardo and John N Boyd. 2015. Putting Time in Perspective: A Valid, Reliable Individual-differences Metric. _Journal of Personality and Social Psychology_ 77(6):1271-1288.