Venue: NAACL 2018

# Early Text Classification using Multi-Resolution Concept Representations

A. Pastor Lopez-Monroy\({}^{*}\), Fabio A. Gonzalez\({}^{\dagger}\), Manuel Montes-y-Gomez\({}^{\ddagger}\), Hugo Jair Escalante\({}^{\ddagger}\) and Thamar Solorio\({}^{\star}\)

###### Abstract

This paper proposes a novel document representation, called Multi-Resolution Representation (MulR), to improve the early detection of risks in social media sources. The goal is to effectively identify the potential risk using as little evidence as possible and with as much anticipation as possible. MulR allows us to generate multiple "views" of the text. These views capture different semantic meanings for words and documents at different levels of granularity, which is very useful in early scenarios to model the variable amounts of evidence. The experimental evaluation shows that MulR using low resolution is better suited for modeling short documents (very early stages), whereas large documents (medium/late stages) are better modeled with higher resolutions. We evaluate the proposed ideas in two different tasks where anticipation is critical: _sexual predator detection_ and _depression detection_. The experimental evaluation for these early tasks revealed that the proposed approach outperforms previous methodologies by a considerable margin.

100F256798430

are obtained by applying clustering techniques to word representations, where the resultant "centroids" comprise the meta-words. Documents are then represented by a Bag-of-Centroids (BoC), that is, a histogram accounting for the occurrence of coarse thematic/semantic primitives, i.e., the meta-words. This part of the work is inspired by the Bag-of-Visual-Words (BoVW), which is widely used in computer vision to represent images [234, 10].

The key aspect for early scenarios is that the number and size of meta-words, allow us to manipulate the level of granularity or the _resolution_ of the representation. This property is very useful to capture discriminative information along the growing amount of available evidence at each early stage. We thus propose a multi-resolution approach, in which primitives at different resolutions are combined to capture feature concepts at multiple levels of detail. The contributions of this paper are twofold: (i) a new Multi-Resolution (MulR) document representation, a generalization to represent documents by exploiting wordvectors at different levels of resolutions; (ii) an empirical validation of the usefulness of multiple resolution levels for early risk detection on social media documents. Our experimental results show that this approach is a promising alternative for early text classification scenarios, where there is a need to make predictions as soon as possible, with little evidence, while at the same time, being robust to incorporate more evidence as it becomes available. We recorded experimental results of an extensive evaluation of our proposed techniques over two benchmarks for early scenarios: sexual predator detection and depression detection. Results showed that in all cases our methodology outperforms state-of-the-art methodologies.

Interestingly, document representations based on partitioning the word-embedding space, like ours, are somewhat similar to topic modeling based representations. In the experimental section we also compare the performance of our method to different topic-based representations like Latent Semantic Analysis (LSA) [59] and Latent Dirichlet Allocation (LDA) [67]. Experimental results showed that our method outperformed the reference techniques. We elaborate on the benefits and limitations of our proposed techniques later in this paper.

## 2 Related Work

The Early Text Categorization problem is an emerging research topic with scant work [8, 10, 11]. Recently, the relevance of the problem has motivated specialized forums such as eRisk-CLEF17 [13]. One of the first attempts is based on processing documents in a sentence-level basis [8]. At every time \(t\), the method reads a sentence and attempts to determine the class of the document. The key aspect of the work is a Markov Decision Process (MDP), where each sentence is modeled in a TFIDF vector. More recently, [1] proposed a straightforward solution for early detection scenarios by using the naive Bayes classifier. The idea consists in training with full documents, but when partial information has to be classified, the maximum a posteriori probability was estimated over the available text. Using this simple yet effective approach, the authors obtained competitive performance with the method in [8]. Furthermore, results reported in [1] were the first evaluation on early sexual predator detection.

In [1] the authors propose methods to exploit Profile Based Representations (PBR's) for words [10]. PBRs are Distributional Term Representations of terms in the vocabulary. Similar to word embeddings these representations build a vector for each word, which aim to extract/learn concepts from simple occurrence statistics of terms in the target classes. PBRs capture discriminative information in a very low dimensional and non-sparse space suitable for early text classification problems. In other work, [1] successfully adapted a version of PBR's for the problem of early depression detection in the context of the eRisk-CLEF17 shared task. The evidence about PBRs suggests that this representation can naturally cope with missing information and obtain discriminative representations for incomplete documents. Nevertheless, just as the vast majority of word embeddings in the literature for standard text classification, there is no consensus about how to exploit these term vectors to represent entire phrases or documents (e.g., the most common strategy is to average the term vectors in documents).

The proposed method is based on creating meta 

words to represent documents. Clustering words into meaningful groups based on some measure of similarity to represent text is not a new concept. One of the classic approaches is term clustering in an _unsupervised_ manner that was first investigated by Lewis (1992). He called his method _reciprocal nearest neighbor clustering_. His method consists of joining words that are similar according to a measure of similarity. In other work, Brown et al. (1992) explored the idea of discovering similarities between words to obtain clusters at different levels. One key difference with our proposal is that in Brown et al. (1992), terms are deterministically/probabilistically associated with a discrete class, where terms that are in the same class are similar in some aspect. However in our proposed strategy, we exploit word vectors instead of a discrete random deterministic variable (e.g., soft/hard partitions of word sets). This makes possible to discover different clusters and meta-words if we change the word representation. Thus, the proposed strategy is highly adaptable to other domains, where the specialization would be achieved by changing the word representation for the problem. In other work, Li and Jain (1998) found that term grouping helps to reduce the feature dimensionality, and at the same time, overcomes the generalization problem of feature selection. The evidence has showed that the performance of the classifier is, at least maintained Li and Jain (1998); Slonim and Tishby (2001). Finally, other authors have also studied the problem of term clustering under a _supervised_ scheme. For example, Baker and McCallum (1998) used a _supervised_ scheme to cluster similar words. They carried out experiments using a Naive Bayes classifier and found results improvement by using a single word representation.

The methods proposed in this research work follow a line of thinking focused on the document representation rather than term representation. Hence, the proposed method takes advantage of specialized vector representation of words (e.g., PBR), but several extensions can be envisioned using other word embeddings in the literature. The benefits of our approach are that it is model independent, easy to implement, and computes lower dimensional and less-sparse representations than traditional BoW. More important, our method improves over state of the art methods, outperforming the methods in Errecalde et al. (2017); Escalante et al. (2017) that in turn, outperform that in Dulac-Arnold et al. (2011); Escalante et al. (2016).

## 3 Multi-Resolution Document Representation

We propose a multi resolution representation that allows to generate multiple "views" of the analyzed document. The intuition behind the proposal of a multi-resolution representation is that words will activate differently each view according to the amount of available text. We assume that having different resolution levels will allow to effectively represent the content of short and large texts as needed along different early stages. The proposed multi-resolution framework is depicted in Figure 1. The idea consists in associating words with similar meaning to the same meta-words in each resolution space. Documents are then represented by multiple Bag-of-Centroids (BoC), that is, multiple histograms accounting for the occurrence of coarse concepts. Hence, this representation can be seen as multiple BoW representations that incorporate multiple semantic resolutions. In Section 3.1 we describe the process to build a Bag-of-Centroids at a single resolution, then in Section 3.2 we formally present the Multi-Resolution variant.

### Single Resolution: Bag of Centroids

Let \({\mathcal{D}}=\left\{(d_{1},y_{1}),\ldots,(d_{h},y_{h})\right\}\) be a training set of \(h\)-pairs of documents \(d_{i}\) and class labels \(y_{i}\). Also let \({\mathcal{V}}=\left\{w_{1},\ldots,w_{r}\right\}\) denote the vocabulary of terms (in our case words). In order to create the Bag of Centroids (**_BoC_**) representation of each document, we first compute the vector representation \(v_{i}\) of each word \(w_{i}\) in the vocabulary of the collection. Note that our framework is agnostic to the underlying process for learning word representations and therefore any word vector representation can be used, for example word embeddings Mikolov et al. (2013) or distributional term representations Lavelli et al. (2004).

The proposed framework is based on the idea of clustering words using the semantic "distance" in the word embedding space. Thus, the first step of the algorithm consists of clustering the word embedding vectors \(v_{i}\) and finding the cluster centers to create the proposed meta-words. The representation of the vocabulary collection in the word embedding space \(W\) is the input for the clustering 

algorithm. For this purpose a variety of clustering approaches can be used. In our experimental evaluation, we explored different algorithms and found out that \(k\)-means offers a good trade-off between performance and speed. We applied \(k\)-means to the \(W\) representation to find the center of the clusters \(C=\{c_{1},c_{2},\ldots,c_{k}\}\), with \(k\) being the number of selected centroids. Then, based on these cluster centers and using \(l_{1}\)-norm, we found a one-to-one association of each word to the _closest_ cluster center in the word embedding space. In other words, for each word \(v_{i}\), we can find an associated cluster center or meta-word \(c_{u}\) with \(u\in\{1,2,\dots,k\}\). We denote this mapping by \(c_{u}=closest(v_{i},C)\), where \(closest\) returns the centroid in \(C\) with the minimum distance to \(v_{i}\). Finally, the **BoC\({}_{k}\)** representation for each document \(d_{j}\) corresponds to \(\textbf{BoC}_{k}(d_{j})=\{(c_{\ell},n_{\ell})\}_{\ell=1...k}\) where \(c_{\ell}\) corresponds to each of the \(k\) centroids and \(n_{\ell}=|\{v_{i}|\forall v_{i}\in d_{j},c_{\ell}=closest(v_{i},C)\}|\). In other words, \(\textbf{BoC}_{k}(d_{j})\) corresponds to a histogram of centroid frequencies, where each pair \((c_{\ell},n_{\ell})\) represents a centroid (meta-word) and its corresponding frequency in the document.

The BoC algorithm depends on one parameter: the number of clusters used to represent each document. This parameter is associated with the level of semantic coarseness used in the representation. In this regard, coarseness refers to the level of meta-word inclusivity: the more words associated with a single meta-word, the coarser the representation. Conversely, with fewer words, the representation becomes more granular. Note that this representation has well known parallels in the extreme cases. When each word becomes a centroid, the resulting representation is equivalent to the typical BoW representation, whereas a coarser representation, with only one meta word, will be equivalent to having the average meta-word of the entire collection.

### Multi-Resolution BoC

The above proposed framework is particularly suitable for incorporating multi-resolution processing, given that the main parameter is related to the granularity or coarseness of the representation. As we will show in our analysis, this property is useful for early scenarios, since few/coarse meta-words allow to better encode documents with little text, whereas many/granular meta-words are useful when more text become available. We propose to exploit this multi-resolution version of the **BoC** representation. In this extension of the basic algorithm, we use a partition of the word embedding space at multiple levels and concatenate them into a new representation. Combining the different granularities into a single representation results in a more robust document model that can help to capture different amounts of text as needed. Intuitively, the coarser levels sufficiently classify documents in early stages, while the more granular levels exploit the additional evidence from longer documents on late stages. We present quantitative and qualitative experiments that support this claim in two datasets: Sexual Predator Detection and Depression Detection.

We call this variation of the BoC representation Multi-Resolution-BoC (**MulR**). Formally: \(\textbf{MulR}(d_{j})=\{\textbf{BoC}_{k_{1}}(d_{j})\cup\textbf{BoC}_{k_{2}}(d_{ j})\cup\ldots\cup\textbf{BoC}_{k_{n}}(d_{j})\}\), where \(\{k_{1},k_{2},\ldots k_{n}\}\) correspond to a set of granular levels. Figure 1 shows the general framework, graphically depicting the process involved in transforming a document into a representation based on meta-words. The figure also includes the process of multi-resolution modification described above. In the figure, the

Figure 1: Algorithm to represent documents as meta-words using three hypothetical resolutions. The document is represented using the “meta-words” defined by the clustering of the vector representations of the vocabulary.

 

meta-words depicted with 'blue' represent the more granular clusters, and those depicted with 'green' represent the less granular clusters. The multi-resolution BoC variation improves the performance by combining the information present at various levels of granularity. Moreover, when documents are closely related, more fine grained features allow to capture finer details and therefore produces better text classification results. This multi-resolution approach combines the advantages of both approaches to create an overall more effective classification method.

## 4 Data collections

For experiments we considered the two data sets described in Table 2. The tasks are Sexual Predator Detection (SPD) and Depression Detection, where clearly early detection is crucial. For the former we used the only publicly available data set for sexual predator detection Inches and Crestani [10]. This data set was released in the context of the sexual predator identification task at PANCLEF'12 and comprises a large number of chat conversations that include real sexual predators. Thus, the task approached is that of identifying those conversations that potentially include a sexual predator, as in Villatoro-Tello et al. [38, 9]. For the depression detection task we use the dataset presented in Losada et al. [6]. In this dataset, each instance has the post history for a user, and depressed users were self-identified as having been diagnosed with depression.

## 5 Evaluation Framework

For our experiments, we lower case the text in documents and use words and punctuation marks as terms1. The representation obtained for each document is then processed by a Support Vector Machine (SVM) with a linear kernel.

Footnote 1: We used terms with frequency higher than 10 in the training datasets.

For the evaluation of the _earliness_ performance, we report the performance of the different methods when using increasing amounts of textual evidence (chunk by chunk evaluation). This evaluation allows to quantify prediction performance when using partial information in documents, and it is a strategy that has been used to evaluate early classification Escalante et al. [7]; Errecalde et al. [1]; Losada et al. [6]. For the evaluation of performance we used the \(f_{1}=\frac{2\times precision+recall}{precision+recall}\) measure. This decision was made in agreement with previous work that reports this metric for the positive class Errecalde et al. [1]. Please note that, contrary to other measures, such as accuracy, \(f_{1}\) measure accounts for the class imbalance problem when only the positive class is analyzed. This is desirable for the data sets we consider as they are highly unbalanced.

**Word-vector representations:** As previously mentioned, the proposed MulR representation generalizes word-vector representations and thus can extend any representation that models each term in the vocabulary using a vector. For this purpose a wide variety of word embeddings or distributional term representations could be used. Both of them exploit the distributional hypothesis to build word vectors, nonetheless they differ in the strategy to capture the relevant information. In this work we use the widely used word2vec, but also other representations that have been used in recent works for these collections. In Table 1 we describe each of the word vector representations considered for this work2.

Footnote 2: For distributional representations we used the framework at [https://github.com/lopez-monroy/FeatureSpaceTree](https://github.com/lopez-monroy/FeatureSpaceTree)

**Baselines:** The main baselines in this work are methods based on the idea of topic modeling for text classification. Topic-based representations group words into topics defined by a set of related words3. Given the strong relation to our method we compare our proposal against Latent Semantic Analysis (LSA) and Latent Dirichlet Allocation (LDA). Furthermore, we also compare with Bag-of-Words using Term Frequency Inverse Document Frequency, since it is a traditional baseline in text categorization tasks.

Footnote 3: We empirically set to 200 the size of the concept space.

## 6 Experimental Results

In this section we report the experimental results for the MulR representation and the selected approaches from the state-of-the-art. In all the experiments we trained the reference classifier (SVM) using full-length documents in the training dataset. In the testing phase, each approach uses all the available information in each of the ten chunks (each chunk increases the available text in 10%). More specifically, we generate document representations starting with the first chunk, and then incrementally adding one ch 

models will then make predictions incrementally as well. We report \(f_{1}\) performance when using different amounts of text from test documents. For the proposed MulR representation, we build 5 different resolutions: 10, 50, 100, 500, and 1000. The goal was to generate meta-words at different levels of granularity, and we plan to further explore the impact of these resolutions in our future research.

In the following experiments, we used the word representations in Table 1 to build our proposed MulR document representation. For comparison purposes we also generate an alternative document representation by averaging (Avg) term-vectors of words in each document, which is a popular strategy to build document representations. Finally, we also compare against several traditional baselines such as the Bag-of-Words, LSA, and specialized methods in each collection Escalante et al. (2017); Errecalde et al. (2017). We evaluate the usefulness of all these different representations in the two early classification tasks mentioned earlier.

### Sexual Predators Detection

In this section we evaluate the performance of the proposed MulR and other reference methodologies for the SPD early detection task (Figure 2). We also show results for MulR and different word representations in Table 3, where several findings can be outlined. First of all, results obtained in early stages (chunk 1 to 4) using the proposed MulR are clearly superior to those obtained averaging word vectors. This is an interesting outcome, since the MulR representation seems to be useful for early scenarios independently of the word vector representation. In the particular case of MulR(TVT), the representation obtains an outstanding performance when having little information (e.g., performance between \(\approx\)71% and \(\approx\)90% before reading 50% of the text). More important, performance improves as more evidence is available (i.e., see the steady improvement up to \(\approx\)97%). These results show that MulR is a robust representation, even in the presence of different amounts of textual evidence, with a clear advantage for early classification stages.

In Figure 2 we can also observe that MulR representation outperformed, by a large margin, the proposed baselines; BoW-TFIDF, LSA, LDA. Furthermore, MulR representation obtains better performance than the work in Escalante et al. (2017), which consists in averaging the PBRs (same that Avg-PBR) and is the state-of-the-art in early SPD. Note that different than Escalante et al. (2017), the proposed MulR significantly improves even after reading 40% of the information. The experimental results in Table 3 also show the

\begin{table}
\begin{tabular}{|p{86.7pt}|p{113.8pt}|} \hline
**Word Representation** & **Description** \\ \hline W2V Mikolov et al. (2013) & Word2Vec uses the Skip-ngram model to find word representations that are useful to predict the surrounding words of a sentence or a document. The method is efficient for learning high-quality vector representations of words from large amounts of text data. We empirically set to 200 the vector dimension. \\ \hline DOR Lavelli et al. (2004) & Document Occurrence Representation (DOR) captures the semantics of a word by observing occurrence distribution over documents in the corpus. DOR represents each word \(v_{i}\) as a vector \(\mathbf{t_{i}}=\langle t_{i,1},\ldots,t_{i,|\mathcal{D}|}\rangle\), where \(|\mathcal{D}|\) is the number of documents in the training collection, and \(t_{i,k}\) indicates the relevance of the document \(D_{k}\) to characterize \(v_{i}\). \\ \hline TCORLavelli et al. (2004) & In Term Co-occurrence Representation (TCOR) the semantics of a word is captured by observing its co-occurrences with other words across documents in the corpus. Thus, each word \(v_{i}\) is associated to a vector \(\mathbf{t_{i}}=\langle t_{i,1},\ldots,t_{i,|\mathcal{V}|}\rangle\), where \(|\mathcal{V}|\) indicates the vocabulary size, and \(t_{i,k}\) denotes the contribution of the word \(v_{k}\) to the semantic description of \(v_{i}\). \\ \hline PBR LopezMonroy et al. (2015) & Profile Based Representation exploits occurrence-statistics of words over a set of documents in target categories. PBR represents each word \(v_{i}\in V\) with a vector \(\mathbf{t_{i}}=\langle t_{i,1},\ldots,t_{i,q}\rangle\), where the \(t_{i,k}\) is the degree of association between word \(v_{i}\) and category \(C_{k}\). The target categories can be taken from the task or artificially created by means of clustering such as in Escalante et al. (2017). \\ \hline TVT Erresalde et al. (2017) & Temporal Variation Terms (TVT) is an adapted version of PBR for early scenarios. TVT builds new artificial target classes/labels in the training set simulating a text stream to generate enriched representations of \(\mathbf{t_{j}}\). The idea is to exploit the positive category to create a set of new artificial categories using text-fragments. \\ \hline  & \multicolumn{1}{c}{} \\ \end{tabular}
\end{table}
Table 1: Word vector representations for early experimentation.

\begin{table}
\begin{tabular}{|l|l|r|r|} \hline
**Task** & **Data set** & **Training** & **Test** \\ \hline Sexual predator det. & PAN’12 & 6588 & 15329 \\ Depression det. & eRisk’17 & 486 & 401 \\ \hline \end{tabular}
\end{table}
Table 2: Data sets considered for early experimentation. There are only two classes in each dataset.

 

following interesting findings:

1. The most useful word vector representation is TVT (Errecalde et al., 2017). This is not surprising, since TVT is a specialized distribution term representation for early prediction scenarios.
2. Word2Vec 4 representations obtained moderate performance in all experiments. We infer that much more data of these specific social media domains are needed in order to build suitable models. Footnote 4: Embeddings were trained in each dataset. We tested pretrained word embeddings for wikipedia/twitter, but the performance was worse.
3. MulR representation is an effective solution for all early chunks, but as more text is available, the other methodologies significantly increase their discriminative power, as seen in results for later chunks. In fact, some representations such as Avg(DOR) can outperform MulR(DOR) representation in late stages. However, even under these conditions MulR(TVT) and MulR(DOR) outperform all reference methodologies.

### Depression Detection

In Table 4 we show the experimental results for early depression detection. In Figure 3 we highlight the performance of the proposal and the reference methodologies. From these results we point out that our proposed multi-task model outperforms the previous state-of-the-art deep learning method with the same accuracy as the baseline. For example, the proposed multi-task model achieves better performance than the baseline.

\begin{table}
\begin{tabular}{|l|c|c|c|c|c|c|c|c|c|} \hline
**Method** & \(ch_{1}\) & \(ch_{2}\) & \(ch_{3}\) & \(ch_{4}\) & \(ch_{5}\) & \(ch_{6}\) & \(chr\) & \(ch_{8}\) & \(ch_{9}\) & \(ch_{10}\) \\ \hline BoW-TFIDF & 27.40 & 46.51 & 59.62 & 68.33 & 71.84 & 75.11 & 76.49 & 80.19 & 81.51 & 83.63 \\ LSA & 35.22 & 38.93 & 48.25 & 55.27 & 61.21 & 68.24 & 70.12 & 72.54 & 74.49 & 77.91 \\ LDA & 32.22 & 36.98 & 45.27 & 51.27 & 58.70 & 63.87 & 67.94 & 70.81 & 72.41 & 72.98 \\ \hline Avg(W2V) & 55.74 & 63.87 & 70.11 & 82.53 & 87.24 & 88.97 & 88.01 & 87.45 & 84.71 & 83.12 \\ MulR(W2V) & 58.97 & 65.78 & 71.97 & 83.09 & 85.49 & 87.21 & 88.46 & 89.00 & 89.15 & 89.49 \\ \hline Avg(DOR) & 66.71 & 76.54 & 87.01 & 91.14 & 92.23 & 93.91 & 95.19 & 95.87 & 96.47 & 96.59 \\ MulR(DOR) & 68.24 & 78.77 & 87.14 & **92.07** & **94.18** & 94.04 & 98.44 & 95.24 & 95.46 & 95.97 \\ \hline Avg(TCOR) & 60.17 & 67.97 & 74.41 & 78.51 & 81.24 & 82.71 & 83.97 & 82.51 & 82.90 & 82.27 \\ MulR(TCOR) & 61.51 & 69.12 & 75.43 & 78.89 & 80.26 & 79.97 & 81.01 & 81.59 & 82.14 & 83.01 \\ \hline Avg(PBR) & 67.10 & 76.97 & 81.69 & 85.00 & 86.03 & 87.21 & 88.14 & 89.16 & 90.25 & 91.21 \\ MulR(PRB) & 69.16 & 77.41 & 83.01 & 87.05 & 88.07 & 89.27 & 90.14 & 91.51 & 92.01 & 92.41 \\ \hline Avg(TVT) & 65.74 & 80.24 & 86.19 & 90.25 & 92.02 & 93.13 & 94.39 & 95.23 & 95.89 & 96.58 \\ MulR(TVT) & **71.15** & **84.00** & **88.56** & 91.66 & 94.11 & **94.92** & **95.31** & **96.50** & **97.16** & **97.43** \\ \hline Escalante et al. (2017) & 67.10 & 76.97 & 81.69 & 85.00 & 86.03 & 87.21 & 88.14 & 89.16 & 90.25 & 91.21 \\ \hline  & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} \\ \end{tabular}
\end{table}
Table 3: \(F_{1}\) results for the chunk by chunk evaluation of different approaches in Sexual Predator Detection. The proposed MulR is evaluated using different word vector representations in the literature.

Figure 2: \(F_{1}\) scores for the chunk by chunk evaluation of the reference methodologies in Sexual Predator Detection.

 

out several interesting findings. The first one is that for this collection, results obtained by the proposal are clearly superior to others in early stages. In general, we can observe the following:

1. The most useful representation in early stages was MulR(TVT), which have considerable improvements between \(\approx\)5% and \(\approx\)2% in chunks 1 to 4.
2. Word Embeddings and DOR showed a similar behavior than in SPD. But in late stages, the best representation was Avg(DOR).
3. Depression Detection problem is a much harder problem than SPD. The \(F_{1}\) measure is under \(\approx\)60% in most of the results. This could be due to the highly unbalanced dataset in two ways: i) the number of instances in each class, and ii) the amount of text contained in documents.

### The Relevance of Individual Resolutions

In this section we aim to study the role of the different resolutions in early scenarios.5 The purpose of the first analysis is to observe the performance of each individual resolution in MulR. In Table 5 we show the results of MulR(TVT) under each of the five resolutions (\(R_{1}=10,R_{2}=50,R_{3}=100,R_{4}=500,R_{5}=1000\)) and each chunk.

Figure 3: \(F_{1}\) scores for the chunk by chunk evaluation of the reference methodologies in Depression Detection.

 

For early SPD the evidence is clear; as the resolution increases the performance in early stages decrease.6 Also note that the higher the resolution, the more chunks needed to outperform the result of the previous resolution. For example, resolution \(R_{3}\) outperforms \(R_{2}\) in chunk 8. Also note that \(R_{4}\) and \(R_{5}\) needed more chunks to obtain comparable performance than \(R_{2}\). Our experimental results excluding one resolution at the time showed worse performance, therefore all of them are essential in the overall classification. Clearly, this evidence shows that the MulR representation is in fact very useful.

Footnote 6: The only exception to this is \(R_{1}\), which has the lowest overall performance. This is somewhat expected since this space only has 10 features to represent documents.

In Table 6 we provide further evidence about the role of different resolutions. In this complementary analysis we study each chunk at test data. For this we use the MulR learned in training to represent test documents, then we compute the Information Gain using Weka Hall et al. (2009) at each test chunk. In Table 6 we show the number of features in each resolution \(R_{i}\) that are present in the top ten meta-words of the MulR(TVT). The analysis complements the evidence, lower resolutions have higher IG at early chunks, whereas higher resolutions are more necessary in late chunks.

## 7 Conclusions

In this paper we proposed a multi resolution representation that allows to generate multiple "views" of the document. Intuitively these views expose different semantic meanings for words and documents along different resolutions. The different resolutions allow to effectively represent the content of short and large texts at different early stages. The MulR obtained the best results reported so far on the early Sexual Predator Detection task dataset Inches and Crestani (2012). For Depression Detection the chunk by chunk evaluation shows promising results for MulR in early stages. What is more, it was shown that the MulR further improves the early recognition performance in the two tasks using different word representations. The relevance of the resolutions in these results is a key factor to understand the proposed MulR and future extensions. These results provide solid evidence to further research on this topic and encourage researchers to apply and evaluate the usefulness of multi-resolution features for other related _early_ tasks.

#### Acknowledgments

This research was partially supported by CONACYT-Mexico (project FC-2016/2410).

## References

* Baker and McCallum (1998) Douglas Baker and Andrew Kachites McCallum. 1998. Distributional Clustering of Words for Text Classification. In _Proceedings of the 21st ACM International Conference on Research and Development in Information Retrieval_. pages 96-103.
* Blei et al. (2003) David M. Blei, Andrew Y. Ng, and Michael I. Jordan. 2003. Latent Dirichlet Allocation. _Journal of Machine Learning Research_ 3:993-1022.
* Brown et al. (2012) Peter F. Brown, Peter V. deSouza, Robert L. Mercer, Vincent J. Della Pietra, and Jenifer C. Lai.

\begin{table}
\begin{tabular}{|l||c|c|c|c|} \hline \multicolumn{3}{|c|}{Sexual} & \multicolumn{3}{|c|}{Predator Detection} \\ \hline
**Test set** & \(R_{1}\) & \(R_{2}\) & \(R_{3}\) & \(R_{4}\) & \(R_{5}\) \\ \hline chunk-1 & 4 & 3 & 2 & 1 & 0 \\ \hline chunk-2 & 3 & 2 & 3 & 2 & 0 \\ \hline chunk-3 & 3 & 2 & 2 & 2 & 1 \\ \hline chunk-4 & 3 & 2 & 3 & 1 & 1 \\ \hline chunk-5 & 3 & 2 & 3 & 1 & 1 \\ \hline chunk-6 & 3 & 1 & 1 & 1 \\ \hline chunk-7 & 3 & 1 & 3 & 2 & 1 \\ \hline chunk-8 & 2 & 2 & 1 & 2 & 3 \\ \hline chunk-9 & 3 & 1 & 1 & 2 & 3 \\ \hline chunk-10 & 3 & 0 & 2 & 2 & 3 \\ \hline \end{tabular}
\end{table}
Table 6: Post-analysis in test dataset. Distribution of the top ten meta-words according to each resolution \(R_{i}\) at different chunks. We used Information Gain Hall et al. (2009) to rank meta-words in MulR(TVT).

\begin{table}
\begin{tabular}{|l|c|c|c|c|c|c|c|c|c|c|} \hline \multicolumn{12}{|c|}{Sexual Predator Detection} \\ \hline
**Method** & \(ch_{1}\) & \(ch_{2}\) & \(ch_{3}\) & \(ch_{4}\) & \(ch_{5}\) & \(ch_{6}\) & \(chr\) & \(ch_{8}\) & \(ch_{9}\) & \(ch_{10}\) \\ \hline MulR(TVT)-R1 & 59.88 & 74.71 & 82.17 & 85.82 & 89.19 & 90.19 & 91.51 & 92.61 & 92.42 & 92.51 \\ MulR(TVT)-R2 & 70.03 & 83.03 & 88.04 & 90.49 & 93.00 & 94.01 & 94.85 & 94.70 & 94.90 & 95.02 \\ MulR(TVT)-R3 & 67.87 & 82.41 & 87.23 & 90.17 & 92.07 & 92.88 & 93.91 & 94.89 & 95.49 & 96.20 \\ MulR(TVT)-R4 & 66.10 & 80.33 & 85.89 & 88.76 & 90.48 & 91.86 & 93.14 & 93.85 & 94.73 & 95.38 \\ MulR(TVT)-R5 & 62.34 & 77.73 & 83.05 & 86.72 & 88.64 & 90.32 & 92.16 & 93.04 & 93.00 & 93.76 \\ \hline MulR(TVT) & **71.15** & **84.00** & **88.56** & **91.66** & **94.11** & **94.92** & **95.31** & **96.50** & **97.16** & **97.43** \\ \hline  & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} \\ \end{tabular}
\end{table}
Table 5: \(F_{1}\) results for the chunk by chunk evaluation of different approaches in Sexual Predator Detection. The best MulR(TVT) is separately evaluated under each resolution.

 

1992. Class-based n-gram models of natural language. _Comput. Linguist._ 18(4):467-479.
* Deerwester et al. (1990) S. Deerwester, S. Dumais, T. Landauer, G. Furnas, and R. Harshman. 1990. Indexing by Latent Semantic Analysis. _Journal of the American Society of Information Science_ 41(6):391-407.
* Dulac-Arnold et al. (2011) G. Dulac-Arnold, L. Denoyer, and P. Gallinari. 2011. Text classification: A sequential reading approach. In _Advances in Information Retrieval, Proc. of 33rd European Conference on IR Research, (ECIR'11)_. Springers, volume 6611 of _LNCS_, pages 411-423.
* Errecalde et al. (2017) Marcelo L. Errecalde, Ma. Paula Villegas, Dario G. Funez, M. Jose Garciarena Ucelay, and Leticia C. Cagnina. 2017. Temporal variation of terms as concept space for early risk prediction. In _CLEF (Working Notes)_.
* Escalante et al. (2013) H. J. Escalante, A. Juarez, E. Villatoro, M. Montesy-Gomez, and L. Villasenor. 2013. Sexual predator detection in chats with chained classifiers. In _Proceedings of NAACL-HLT 2013, 4th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis_. page 46.
* Escalante et al. (2016) H. J. Escalante, M. Montes-y-Gomez, L. Villasenor, and M. L. Errecalde. 2016. Early text classification: a naive solution. In _Proceedings of NAACL-HLT 2016, 7th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis_. pages 91-99.

* Hall et al. (2009) M. Hall, E. Frank, G. Holmes, B. Pfahringer, P Reutemann, and I. Witten. 2009. The weka data mining software: An update. _SIGKDD Explorations_ 11.
* Inches and Crestani (2012) G. Inches and F. Crestani. 2012. Overview of the international sexual predator identification competition at pan-2012. In _CEUR Workshop Proceedings, Working Notes for CLEF 2012 Conference_. CEUR, volume 1178.
* Joachims (1998) Thorsten Joachims. 1998. Text categorization with support vector machines: Learning with many relevant features. In Claire Nedellec and Celine Rouveiroul, editors, _Machine Learning: ECML-98_, Springer Berlin Heidelberg, volume 1398 of _Lecture Notes in Computer Science_, pages 137-142.
* Lavelli et al. (2004) Alberto Lavelli, Fabrizio Sebastiani, and Roberto Zanoli. 2004. Distributional term representations: an experimental comparison. In _CIKM_. ACM, pages 615-624.
* Lazebnik et al. (2006) Svetlana Lazebnik, Cordelia Schmid, and Jean Ponce. 2006. Beyond bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories. In _Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition_. volume 2, pages 2169-2178.
* Lewis (1992) David D. Lewis. 1992. An Evaluation of Phrasal and Clustered Representations on a Text Categorization Task. In _Proceedings of the 15th International ACM/SIGIR Conference on Research & Development in Information Retrieval_. June, pages 37-50.
* Li and Jain (1998) Yong Li and Anil Jain. 1998. Classification of Text Documents. _The Computer Journal_ 41(8):537 -546.
* Lopez-Monroy et al. (2015) A. Pastor Lopez-Monroy, Manuel Montes-y-Gomez, Hugo Jair Escalante, Luis Villasenor-Pineda, and Efstathios Stamatatos. 2015. Discriminative subprofile-specific representations for author profiling in social media. _Knowledge-based Systems_ 89:134-147.
* Losada et al. (2017) David E. Losada, Fabio Crestani, and Javier Parapar. 2017. _eRISX 2017: CLEF Lab on Early Risk Prediction on the Internet: Experimental Foundations_, Springer International Publishing, Cham, pages 346-360.
* Mikolov et al. (2013) Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Distributed Representations of Words and Phrases and their Compositionality. In _Proceedings of Advances in Neural Information Processing Systems 26 (NIPS 2013)_. pages 1-9.
* Sivic and Zisserman (2004) J. Sivic and A. Zisserman. 2004. Video data mining using configurations of viewpoint invariant regions. In _CVPR_. IEEE, volume 1, pages I-488.
* Slonim and Tishby (2001) Noam Slonim and Naftali Tishby. 2001. The Power of Word Clusters for Text Classification. In _Proceedings of the 23rd European Colloquium on Information Retrieval Research_. volume 1, pages 1-12.
* Villatoro-Tello et al. (2012) Esau Villatoro-Tello, Antonio Juarez-Gonzalez, Hugo Jair Escalante, Manuel Montes-y Gomez, and Luis Villasenor Pineda. 2012. A two-step approach for effective detection of misbehaving users in chats. In _CLEF (Online Working Notes/Labs/Workshop)_.