prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. What are the results from these proposed strategies?
Context 1: Introduction and Background
Many reinforcement learning algorithms are designed for relatively small discrete or continuous action spaces and so have trouble scaling  Text-adventure games—or interaction fictions—are simulations in which both an agents' state and action spaces are in textual natural language  An example of a one turn agent interaction in the popular text-game Zork1 can be seen in Fig  FIGREF1  Text-adventure games provide us with multiple challenges in the form of partial observability, commonsense reasoning, and a combinatorially-sized state-action space  Text-adventure games are structured as long puzzles or quests, interspersed with bottlenecks  The quests can usually be completed through multiple branching paths  However, games can also feature one or more bottlenecks  Bottlenecks are areas that an agent must pass through in order to progress to the next section of the game regardless of what path the agent has taken to complete that section of the quest BIBREF0  In this work, we focus on more effectively exploring this space and surpassing these bottlenecks—building on prior work that focuses on tackling the other problems 
Formally, we use the definition of text-adventure games as seen in BIBREF1 and BIBREF2  These games are partially observable Markov decision processes (POMDPs), represented as a 7-tuple of $\langle S,T,A,\Omega , O,R, \gamma \rangle $ representing the set of environment states, mostly deterministic conditional transition probabilities between states, the vocabulary or words used to compose text commands, observations returned by the game, observation conditional probabilities, reward function, and the discount factor respectively  For our purposes, understanding the exact state and action spaces we use in this work is critical and so we define each of these in relative depth 
Action-Space  To solve Zork1, the cannonical text-adventure games, requires the generation of actions consisting of up to five-words from a relatively modest vocabulary of 697 words recognized by the game’s parser  This results in $\mathcal {O}(697^5)={1 64e14}$ possible actions at every step  To facilitate text-adventure game playing, BIBREF2 introduce Jericho, a framework for interacting with text-games  They propose a template-based action space in which the agent first selects a template, consisting of an action verb and preposition, and then filling that in with relevant entities $($e g  $[get]$ $ [from] $ $)$  Zork1 has 237 templates, each with up to two blanks, yielding a template-action space of size $\mathcal {O}(237 \times 697^2)={1 15e8}$  This space is still far larger than most used by previous approaches applying reinforcement learning to text-based games 
State-Representation  Prior work has shown that knowledge graphs are effective in terms of dealing with the challenges of partial observability $($BIBREF3 BIBREF3; BIBREF4$)$  A knowledge graph is a set of 3-tuples of the form $\langle subject, relation, object \rangle $  These triples are extracted from the observations using Stanford's Open Information Extraction (OpenIE) BIBREF5  Human-made text-adventure games often contain relatively complex semi-structured information that OpenIE is not designed to parse and so they add additional rules to ensure that the correct information is parsed  The graph itself is more or less a map of the world, with information about objects' affordances and attributes linked to the rooms that they are place in a map  The graph also makes a distinction with respect to items that are in the agent's possession or in their immediate surrounding environment  An example of what the knowledge graph looks like and specific implementation details can be found in Appendix SECREF14 
BIBREF6 introduce the KG-A2C, which uses a knowledge graph based state-representation to aid in the section of actions in a combinatorially-sized action-space—specifically they use the knowledge graph to constrain the kinds of entities that can be filled in the blanks in the template action-space  They test their approach on Zork1, showing the combination of the knowledge graph and template action selection resulted in improvements over existing methods  They note that their approach reaches a score of 40 which corresponds to a bottleneck in Zork1 where the player is eaten by a “grue” (resulting in negative reward) if the player has not first lit a lamp  The lamp must be lit many steps after first being encountered, in a different section of the game; this action is necessary to continue exploring but doesn’t immediately produce any positive reward  That is, there is a long term dependency between actions that is not immediately rewarded, as seen in Figure FIGREF1  Others using artificially constrained action spaces also report an inability to pass through this bottleneck BIBREF7, BIBREF8  They pose a significant challenge for these methods because the agent does not see the correct action sequence to pass the bottleneck enough times  This is in part due to the fact that for that sequence to be reinforced, the agent needs to reach the next possible reward beyond the bottleneck 
More efficient exploration strategies are required to pass bottlenecks  Our contributions are two-fold  We first introduce a method that detects bottlenecks in text-games using the overall reward gained and the knowledge graph state  This method freezes the policy used to reach the bottleneck and restarts the training from there on out, additionally conducting a backtracking search to ensure that a sub-optimal policy has not been frozen  The second contribution explore how to leverage knowledge graphs to improve existing exploration algorithms for dealing with combinatorial action-spaces such as Go-Explore BIBREF9  We additionally present a comparative ablation study analyzing the performance of these methods on the popular text-game Zork1 
Exploration Methods
In this section, we describe methods to explore combinatorially sized action spaces such as text-games—focusing especially on methods that can deal with their inherent bottleneck structure  We first describe our method that explicitly attempts to detect bottlenecks and then describe how an exploration algorithm such as Go Explore BIBREF9 can leverage knowledge graphs 
KG-A2C-chained An example of a bottleneck can be seen in Figure FIGREF1  We extend the KG-A2C algorithm as follows  First, we detect bottlenecks as states where the agent is unable to progress any further  We set a patience parameter and if the agent has not seen a higher score in patience steps, the agent assumes it has been limited by a bottleneck  Second, when a bottleneck is found, we freeze the policy that gets the agent to the state with the highest score  The agent then begins training a new policy from that particular state 
Simply freezing the policy that led to the bottleneck, however, can potentially result in a policy one that is globally sub-optimal  We therefore employ a backtracking strategy that restarts exploration from each of the $n$ previous steps—searching for a more optimal policy that reaches that bottleneck  At each step, we keep track of a buffer of $n$ states and admissible actions that led up to that locally optimal state  We force the agent to explore from this state to attempt to drive it out of the local optima  If it is further unable to find itself out of this local optima, we refresh the training process again, but starting at the state immediately before the agent reaches the local optima  If this continues to fail, we continue to iterate through this buffer of seen states states up to that local optima until we either find a more optimal state or we run out of states to refresh from, in which we terminate the training algorithm 
KG-A2C-Explore Go-Explore BIBREF9 is an algorithm that is designed to keep track of sub-optimal and under-explored states in order to allow the agent to explore upon more optimal states that may be a result of sparse rewards  The Go-Explore algorithm consists of two phases, the first to continuously explore until a set of promising states and corresponding trajectories are found on the basis of total score, and the second to robustify this found policy against potential stochasticity in the game  Promising states are defined as those states when explored from will likely result in higher reward trajectories  Since the text games we are dealing with are mostly deterministic, with the exception of Zork in later stages, we only focus on using Phase 1 of the Go-Explore algorithm to find an optimal policy  BIBREF10 look at applying Go-Explore to text-games on a set of simpler games generated using the game generation framework TextWorld BIBREF1  Instead of training a policy network in parallel to generate actions used for exploration, they use a small set of “admissible actions”—actions guaranteed to change the world state at any given step during Phase 1—to explore and find high reward trajectories  This space of actions is relatively small (of the order of $10^2$ per step) and so finding high reward trajectories in larger action-spaces such as in Zork would be infeasible
Go-Explore maintains an archive of cells—defined as a set of states that map to a single representation—to keep track of promising states  BIBREF9 simply encodes each cell by keeping track of the agent's position and BIBREF10 use the textual observations encoded by recurrent neural network as a cell representation  We improve on this implementation by training the KG-A2C network in parallel, using the snapshot of the knowledge graph in conjunction with the game state to further encode the current state and use this as a cell representation  At each step, Go-Explore chooses a cell to explore at random (weighted by score to prefer more advanced cells)  The KG-A2C will run for a number of steps, starting with the knowledge graph state and the last seen state of the game from the cell  This will generate a trajectory for the agent while further training the KG-A2C at each iteration, creating a new representation for the knowledge graph as well as a new game state for the cell  After expanding a cell, Go-Explore will continue to sample cells by weight to continue expanding its known states  At the same time, KG-A2C will benefit from the heuristics of selecting preferred cells and be trained on promising states more often 
Evaluation
We compare our two exploration strategies to the following baselines and ablations:
KG-A2C This is the exact same method presented in BIBREF6 with no modifications 
A2C Represents the same approach as KG-A2C but with all the knowledge graph components removed  The state representation is text only encoded using recurrent networks 
A2C-chained Is a variation on KG-A2C-chained where we use our policy chaining approach with the A2C method to train the agent instead of KG-A2C 
A2C-Explore Uses A2C in addition to the exploration strategy seen in KG-A2C-Explore  The cell representations here are defined in terms of the recurrent network based encoding of the textual observation 
Figure FIGREF10 shows that agents utilizing knowledge-graphs in addition to either enhanced exploration method far outperform the baseline A2C and KG-A2C  KG-A2C-chained and KG-A2C-Explore both pass the bottleneck of a score of 40, whereas A2C-Explore gets to the bottleneck but cannot surpass it 
There are a couple of key insights that can be drawn from these results The first is that the knowledge graph appears to be critical; it is theorized to help with partial observability  However the knowledge graph representation isn't sufficient in that the knowledge graph representation without enhanced exploration methods cannot surpass the bottleneck  A2C-chained—which explores without a knowledge graph—fails to even outperform the baseline A2C  We hypothesize that this is due to the knowledge graph aiding implicitly in the sample efficiency of bottleneck detection and subsequent exploration  That is, exploring after backtracking from a potentially detected bottleneck is much more efficient in the knowledge graph based agent 
The Go-Explore based exploration algorithm sees less of a difference between agents  A2C-Explore converges more quickly, but to a lower reward trajectory that fails to pass the bottleneck, whereas KG-A2C-Explore takes longer to reach a similar reward but consistently makes it through the bottleneck  The knowledge graph cell representation appears to thus be a better indication of what a promising state is as opposed to just the textual observation 
Comparing the advanced exploration methods when using the knowledge graph, we see that both agents successfully pass the bottleneck corresponding to entering the cellar and lighting the lamp and reach comparable scores within a margin of error  KG-A2C-chained is significantly more sample efficient and converges faster  We can infer that chaining policies by explicitly detecting bottlenecks lets us pass it more quickly than attempting to find promising cell representations with Go-Explore  This form of chained exploration with backtracking is particularly suited to sequential decision making problems that can be represented as acyclic directed graphs as in Figure FIGREF1 
Appendix ::: Zork1
Zork1 is one of the first text-adventure games and heavily influences games released later in terms of narrative style and game structure  It is a dungeon crawler where the player must explore a vast world and collect a series of treasures  It was identified by BIBREF2 as a moonshot game and has been the subject of much work in leaning agents BIBREF12, BIBREF7, BIBREF11, BIBREF8  Rewards are given to the player when they collect treasures as well as when important intermediate milestones needed to further explore the world are passed  Figure FIGREF15 and Figure FIGREF1 show us a map of the world of Zork1 and the corresponding quest structure 
The bottleneck seen at a score of around 40 is when the player first enters the cellar on the right side of the map  The cellar is dark and you need to immediately light the lamp to see anything  Attempting to explore the cellar in the dark results in you being instantly killed by a monster known as a “grue” 
Appendix ::: Knowledge Graph Rules
We make no changes from the graph update rules used by BIBREF6  Candidate interactive objects are identified by performing part-of-speech tagging on the current observation, identifying singular and proper nouns as well as adjectives, and are then filtered by checking if they can be examined using the command $examine$ $OBJ$  Only the interactive objects not found in the inventory are linked to the node corresponding to the current room and the inventory items are linked to the “you” node  The only other rule applied uses the navigational actions performed by the agent to infer the relative positions of rooms, e g  $\langle kitchen,down,cellar \rangle $ when the agent performs $go$ $down$ when in the kitchen to move to the cellar 
Appendix ::: Hyperparameters
Hyperparameters used for our agents are given below  Patience and buffer size are used for the policy chaining method as described in Section SECREF2  Cell step size is a parameter used for Go-Explore and describes how many steps are taken when exploring in a given cell state  Base hyperparameters for KG-A2C are taken from BIBREF6 and the same parameters are used for A2C 
Context 2: Introduction
This work is licensed under a Creative Commons Attribution 4 0 International License  License details: http://creativecommons org/licenses/by/4 0/
Entity linking (EL), mapping entity mentions in texts to a given knowledge base (KB), serves as a fundamental role in many fields, such as question answering BIBREF0 , semantic search BIBREF1 , and information extraction BIBREF2 , BIBREF3   However, this task is non-trivial because entity mentions are usually ambiguous  As shown in Figure FIGREF1 , the mention England refers to three entities in KB, and an entity linking system should be capable of identifying the correct entity as England cricket team rather than England and England national football team 
Entity linking is typically broken down into two main phases: (i) candidate generation obtains a set of referent entities in KB for each mention, and (ii) named entity disambiguation selects the possible candidate entity by solving a ranking problem  The key challenge lies in the ranking model that computes the relevance between candidates and the corresponding mentions based on the information both in texts and KBs BIBREF4   In terms of the features used for ranking, we classify existing EL models into two groups: local models to resolve mentions independently relying on textual context information from the surrounding words BIBREF5 , BIBREF6 , BIBREF7 , BIBREF8 , and global (collective) models, which are the main focus of this paper, that encourage the target entities of all mentions in a document to be topically coherent BIBREF9 , BIBREF10 , BIBREF11 , BIBREF12 , BIBREF13 , BIBREF14  
Global models usually build an entity graph based on KBs to capture coherent entities for all identified mentions in a document, where the nodes are entities, and edges denote their relations  The graph provides highly discriminative semantic signals (e g , entity relatedness) that are unavailable to local model BIBREF15   For example (Figure FIGREF1 ), an EL model seemly cannot find sufficient disambiguation clues for the mention England from its surrounding words, unless it utilizes the coherence information of consistent topic “cricket" among adjacent mentions England, Hussain, and Essex  Although the global model has achieved significant improvements, its limitation is threefold:
To mitigate the first limitation, recent EL studies introduce neural network (NN) models due to its amazing feature abstraction and generalization ability  In such models, words/entities are represented by low dimensional vectors in a continuous space, and features for mention as well as candidate entities are automatically learned from data BIBREF4   However, existing NN-based methods for EL are either local models BIBREF16 , BIBREF17 or merely use word/entity embeddings for feature extraction and rely on another modules for collective disambiguation, which thus cannot fully utilize the power of NN models for collective EL BIBREF18 , BIBREF19 , BIBREF20  
The second drawback of the global approach has been alleviated through approximate optimization techniques, such as PageRank/random walks BIBREF21 , graph pruning BIBREF22 , ranking SVMs BIBREF23 , or loopy belief propagation (LBP) BIBREF18 , BIBREF24   However, these methods are not differentiable and thus difficult to be integrated into neural network models (the solution for the first limitation) 
To overcome the third issue of inadequate training data, BIBREF17 has explored a massive amount of hyperlinks in Wikipedia, but these potential annotations for EL contain much noise, which may distract a naive disambiguation model BIBREF6  
In this paper, we propose a novel Neural Collective Entity Linking model (NCEL), which performs global EL combining deep neural networks with Graph Convolutional Network (GCN) BIBREF25 , BIBREF26 that allows flexible encoding of entity graphs  It integrates both local contextual information and global interdependence of mentions in a document, and is efficiently trainable in an end-to-end fashion  Particularly, we introduce attention mechanism to robustly model local contextual information by selecting informative words and filtering out the noise  On the other hand, we apply GCNs to improve discriminative signals of candidate entities by exploiting the rich structure underlying the correct entities  To alleviate the global computations, we propose to convolute on the subgraph of adjacent mentions  Thus, the overall coherence shall be achieved in a chain-like way via a sliding window over the document  To the best of our knowledge, this is the first effort to develop a unified model for neural collective entity linking 
In experiments, we first verify the efficiency of NCEL via theoretically comparing its time complexity with other collective alternatives  Afterwards, we train our neural model using collected Wikipedia hyperlinks instead of dataset-specific annotations, and perform evaluations on five public available benchmarks  The results show that NCEL consistently outperforms various baselines with a favorable generalization ability  Finally, we further present the performance on a challenging dataset WW BIBREF19 as well as qualitative results, investigating the effectiveness of each key module 
Preliminaries and Framework
We denote INLINEFORM0 as a set of entity mentions in a document INLINEFORM1 , where INLINEFORM2 is either a word INLINEFORM3 or a mention INLINEFORM4   INLINEFORM5 is the entity graph for document INLINEFORM6 derived from the given knowledge base, where INLINEFORM7 is a set of entities, INLINEFORM8 denotes the relatedness between INLINEFORM9 and higher values indicate stronger relations  Based on INLINEFORM10 , we extract a subgraph INLINEFORM11 for INLINEFORM12 , where INLINEFORM13 denotes the set of candidate entities for INLINEFORM14   Note that we don't include the relations among candidates of the same mention in INLINEFORM15 because these candidates are mutually exclusive in disambiguation 
Formally, we define the entity linking problem as follows: Given a set of mentions INLINEFORM0 in a document INLINEFORM1 , and an entity graph INLINEFORM2 , the goal is to find an assignment INLINEFORM3  
To collectively find the best assignment, NCEL aims to improve the discriminability of candidates' local features by using entity relatedness within a document via GCN, which is capable of learning a function of features on the graph through shared parameters over all nodes  Figure FIGREF10 shows the framework of NCEL including three main components:
Example As shown in Figure FIGREF10 , for the current mention England, we utilize its surrounding words as local contexts (e g , surplus), and adjacent mentions (e g , Hussian) as global information  Collectively, we utilize the candidates of England INLINEFORM0 as well as those entities of its adjacencies INLINEFORM1 to construct feature vectors for INLINEFORM2 and the subgraph of relatedness as inputs of our neural model  Let darker blue indicate higher probability of being predicted, the correct candidate INLINEFORM3 becomes bluer due to its bluer neighbor nodes of other mentions INLINEFORM4   The dashed lines denote entity relations that have indirect impacts through the sliding adjacent window , and the overall structure shall be achieved via multiple sub-graphs by traversing all mentions 
Before introducing our model, we first describe the component of candidate generation 
Candidate Generation
Similar to previous work BIBREF24 , we use the prior probability INLINEFORM0 of entity INLINEFORM1 conditioned on mention INLINEFORM2 both as a local feature and to generate candidate entities: INLINEFORM3   We compute INLINEFORM4 based on statistics of mention-entity pairs from: (i) Wikipedia page titles, redirect titles and hyperlinks, (ii) the dictionary derived from a large Web Corpus BIBREF27 , and (iii) the YAGO dictionary with a uniform distribution BIBREF22   We pick up the maximal prior if a mention-entity pair occurs in different resources  In experiments, to optimize for memory and run time, we keep only top INLINEFORM5 entities based on INLINEFORM6   In the following two sections, we will present the key components of NECL, namely feature extraction and neural network for collective entity linking 
Feature Extraction
The main goal of NCEL is to find a solution for collective entity linking using an end-to-end neural model, rather than to improve the measurements of local textual similarity or global mention/entity relatedness  Therefore, we use joint embeddings of words and entities at sense level BIBREF28 to represent mentions and its contexts for feature extraction  In this section, we give a brief description of our embeddings followed by our features used in the neural model 
Learning Joint Embeddings of Word and Entity
Following BIBREF28 , we use Wikipedia articles, hyperlinks, and entity outlinks to jointly learn word/mention and entity embeddings in a unified vector space, so that similar words/mentions and entities have similar vectors  To address the ambiguity of words/mentions, BIBREF28 represents each word/mention with multiple vectors, and each vector denotes a sense referring to an entity in KB  The quality of the embeddings is verified on both textual similarity and entity relatedness tasks 
Formally, each word/mention has a global embedding INLINEFORM0 , and multiple sense embeddings INLINEFORM1   Each sense embedding INLINEFORM2 refers to an entity embedding INLINEFORM3 , while the difference between INLINEFORM4 and INLINEFORM5 is that INLINEFORM6 models the co-occurrence information of an entity in texts (via hyperlinks) and INLINEFORM7 encodes the structured entity relations in KBs  More details can be found in the original paper 
Local Features
Local features focus on how compatible the entity is mentioned in a piece of text (i e , the mention and the context words)  Except for the prior probability (Section SECREF9 ), we define two types of local features for each candidate entity INLINEFORM0 :
String Similarity Similar to BIBREF16 , we define string based features as follows: the edit distance between mention's surface form and entity title, and boolean features indicating whether they are equivalent, whether the mention is inside, starts with or ends with entity title and vice versa 
Compatibility We also measure the compatibility of INLINEFORM0 with the mention's context words INLINEFORM1 by computing their similarities based on joint embeddings: INLINEFORM2 and INLINEFORM3 , where INLINEFORM4 is the context embedding of INLINEFORM5 conditioned on candidate INLINEFORM6 and is defined as the average sum of word global vectors weighted by attentions: INLINEFORM7
where INLINEFORM0 is the INLINEFORM1 -th word's attention from INLINEFORM2   In this way, we automatically select informative words by assigning higher attention weights, and filter out irrelevant noise through small weights  The attention INLINEFORM3 is computed as follows: INLINEFORM4
where INLINEFORM0 is the similarity measurement, and we use cosine similarity in the presented work  We concatenate the prior probability, string based similarities, compatibility similarities and the embeddings of contexts as well as the entity as the local feature vectors 
Global Features
The key idea of collective EL is to utilize the topical coherence throughout the entire document  The consistency assumption behind it is that: all mentions in a document shall be on the same topic  However, this leads to exhaustive computations if the number of mentions is large  Based on the observation that the consistency attenuates along with the distance between two mentions, we argue that the adjacent mentions might be sufficient for supporting the assumption efficiently 
Formally, we define neighbor mentions as INLINEFORM0 adjacent mentions before and after current mention INLINEFORM1 : INLINEFORM2 , where INLINEFORM3 is the pre-defined window size  Thus, the topical coherence at document level shall be achieved in a chain-like way  As shown in Figure FIGREF10 ( INLINEFORM4 ), mentions Hussain and Essex, a cricket player and the cricket club, provide adequate disambiguation clues to induce the underlying topic “cricket" for the current mention England, which impacts positively on identifying the mention surrey as another cricket club via the common neighbor mention Essex 
A degraded case happens if INLINEFORM0 is large enough to cover the entire document, and the mentions used for global features become the same as the previous work, such as BIBREF21   In experiments, we heuristically found a suitable INLINEFORM1 which is much smaller than the total number of mentions  The benefits of efficiency are in two ways: (i) to decrease time complexity, and (ii) to trim the entity graph into a fixed size of subgraph that facilitates computation acceleration through GPUs and batch techniques, which will be discussed in Section SECREF24  
Given neighbor mentions INLINEFORM0 , we extract two types of vectorial global features and structured global features for each candidate INLINEFORM1 :
Neighbor Mention Compatibility Suppose neighbor mentions are topical coherent, a candidate entity shall also be compatible with neighbor mentions if it has a high compatibility score with the current mention, otherwise not  That is, we extract the vectorial global features by computing the similarities between INLINEFORM0 and all neighbor mentions: INLINEFORM1 , where INLINEFORM2 is the mention embedding by averaging the global vectors of words in its surface form: INLINEFORM3 , where INLINEFORM4 are tokenized words of mention INLINEFORM5  
Subgraph Structure The above features reflect the consistent semantics in texts (i e , mentions)  We now extract structured global features using the relations in KB, which facilitates the inference among candidates to find the most topical coherent subset
Context 3: Introduction
Recently, people have started looking at online forums either as a primary or secondary source of counseling services BIBREF0  BIBREF1 reported that over the first five years of operation (2011-2016), ReachOut com – Ireland's online youth mental health service – 62% of young people would visit a website for support when going through a tough time  With the expansion of the Internet, there has been a substantial growth in the number of users looking for psychological support online 
The importance of the on-line life of patients has been recognized in research as well  BIBREF2 stated that the online life of patients constitutes a major influence on their self-definition  Furthermore, according to BIBREF3, the social networking activities of an individual, offer an important reflection of their personality  While dealing with patients suffering from psychological problems, it is important that therapists do not ignore this pivotal source of information which can provide deep insights into their patients' mental conditions 
Acceptance of on-line support groups (OSG) by Mental Health Professionals is still not established BIBREF4  Since OSG can have double-edged effects on patients and the presence of professionals is often limited, we argue that their properties should be further studied  According to BIBREF5 OSG effectiveness is hard to assess, while some studies showed OSG's potential to change participants' attitudes, no such effect was observed in other studies (see Related Work Section for more details)  Furthermore the scope of previous work on analysis of users' behaviour in OSG has been limited by the fact that they relied on expert annotation of posts and comments BIBREF6 
We present a novel approach for automatically analysing online conversations for the presence of therapeutic factors of group therapy defined by BIBREF7 as “the actual mechanisms of effecting change in the patient”  The authors have identified 11 therapeutic factors in group therapy: Universality, Altruism, Instillation of Hope, Guidance, Imparting information, Developing social skills, Interpersonal learning, Cohesion, Catharsis, Existential factors, Imitative behavior and Corrective recapitulation of family of origin issues  In this paper, we focus on 3 therapeutic factors: Universality, Altruism and Instillation of Hope (listed below), as we believe that these can be approximated by using established NLP techniques (e g  Sentiment Analysis, Dialogue Act tagging etc ) 
Universality: the disconfirmation of a user's feelings of uniqueness of their mental health condition 
Altruism: others offer support, reassurance, suggestions and insight 
Instillation of Hope: inspiration provided to participants by their peers 
The selected therapeutic factors are analysed in terms of illocutionary force and attitude  Due to the multi-party and asynchronous nature of on-line social media conversations, prior to the analysis, we extract conversation threads among users – an essential prerequisite for any kind of higher-level dialogue analysis BIBREF10  Afterwards, the illocutionary force is identified using Dialogue Act tagging, whereas the attitude by using Sentiment Analysis  The quantitative analysis is then performed on these processed conversations 
Ideally, the analysis would require experts to annotate each post and comment on the presence of therapeutic factors  However, due to time and cost demands of this task, it is feasible to analyse only a small fraction of the available data  Compared to previous studies (e g  BIBREF6) that analysed few tens of conversations and several thousand lines of chat; using the proposed approach – application of Dialogue Acts and Sentiment Analysis – we were able to automatically analyse approximately 300 thousands conversations (roughly 1 5 million comments)  The rest of the paper is structured as follows  In Section 2 we introduce related work  Next, in Section 3 we describe the pre-processing pipeline and the methodology to perform thread extraction on asynchronous multi-party conversations  In Section 4 we provide the describe the final dataset used for the analysis, and in Section 5 we present the results of our analysis  Finally, in Section 6 we provide concluding remarks and future research directions 
Related Work
On-line support groups have been analyzed for various factors before  For instance, BIBREF11 analysed stress reduction in on-line support group chat-rooms, and the effects of on-line social interactions  Such studies mostly relied on questionnaires and were based on a small number of users  Nevertheless, in BIBREF11, the author showed that social support facilitates coping with distress, improves mood and expedites recovery from it  These findings highlight that, overall, on-line discussion boards appear to be therapeutic and constructive for individuals suffering alcohol-abuse 
Application of NLP to the analysis of mental health-related conversation has been studied as well (e g  BIBREF12, BIBREF13)  BIBREF6 applied sentiment-analysis combined with extensive turn-level annotation to investigate stress reduction in on-line support group chat-rooms, showing that sentiment-analysis is a good predictor of entrance stress level  Furthermore, similar to our setting, they applied automatic thread-extraction to determine conversation threads 
BIBREF14 have shown that on-line support group therapy increased the quality of life of patients with metastatic breast cancer  Since many original posters reported the benefits of group therapy on patients BIBREF15, BIBREF2, BIBREF16, BIBREF17, BIBREF18, BIBREF7, we evaluate the effect of the user interaction using sentiment scores of comments in on-line support groups 
According to BIBREF6, users with high incoming stress tend to request less information from others, as a percentage of their time, and share much more information, in absolute terms  In addition, high information sharing has been shown to be a good predictor of stress reduction at the end of the chat BIBREF6  Regarding information sharing, we rely on Dialogue Acts BIBREF19 to model the speaker's intention in producing an utterance  In particular, we are interested in Dialogue Act label that is defined to represent descriptive, narrative, or personal information – the statement 
Dialogue Acts have been applied to the analysis of spoken BIBREF20, BIBREF21 as well as on-line written synchronous conversations BIBREF22  We apply Dialogue Act tag set defined in BIBREF22 to the analysis of our on-line asynchronous conversations  We argue that Dialogue Acts can be used to analyse user behaviour in social media and verify the presence of therapeutic factors 
Methodology
We select the three therapeutic factors – Universality, Altruism and Instillation of Hope – that can be best approximated using NLP techniques: Sentiment Analysis and Dialogue Act tagging  We discuss each one of the selected therapeutic factors and the identified necessary conditions  The listed conditions, however, are not sufficient to attribute the presence of a therapeutic factor with high confidence, which only can be obtained using expert annotation  Our analysis focuses on the structure of conversations; though content plays an important role as well 
Universality consists in the disconfirmation of patients' belief of uniqueness of their disease  This therapeutic factor is shown to be a powerful source of relief for the patient, according to BIBREF7  From this definition, we can draw the following conditions that are applicable to our environment:
improvement of original poster's sentiment: we hypothesize that the discovery that other people passed through similar issues leads to a higher sentiment score;
posts containing negative personal experiences: to disconfirm the belief of uniqueness users have to share their story;
comments containing negative statements: to disconfirm the patient's feelings of uniqueness, the commenting user must tell a similar negative personal experience  This condition requires two sub-conditions: high presence of statements in comments and the presence of negative comments replying to negative posts 
Instillation of Hope is based on inspiration provided to participants by their peers  Through the inspiration provided by their peers, patients can increase their expectation on the therapy outcome  BIBREF7 in several studies have demonstrated that a high expectation of help before the start of a therapy is significantly correlated with a positive therapy outcome  The author states that many patients pointed out the importance of having observed the improvement of others

ground_truth: Reward of 11.8 for the A2C-chained model, 41.8 for the KG-A2C-chained model, 40 for A2C-Explore and 44 for KG-A2C-Explore.

KG-A2C-chained and KG-A2C-Explore both pass the bottleneck of a score of 40

prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. How effective is their NCEL approach overall?
Context 2:  For each document, we obtain the entity graph INLINEFORM0 by taking candidate entities of all mentions INLINEFORM1 as nodes, and using entity embeddings to compute their similarities as edges INLINEFORM2   Then, we extract the subgraph structured features INLINEFORM3 for each entity INLINEFORM4 for efficiency 
Formally, we define the subgraph as: INLINEFORM0 , where INLINEFORM1   For example (Figure FIGREF1 ), for entity England cricket team, the subgraph contains the relation from it to all candidates of neighbor mentions: England cricket team, Nasser Hussain (rugby union), Nasser Hussain, Essex, Essex County Cricket Club and Essex, New York  To support batch-wise acceleration, we represent INLINEFORM2 in the form of adjacency table based vectors: INLINEFORM3 , where INLINEFORM4 is the number of candidates per mention 
Finally, for each candidate INLINEFORM0 , we concatenate local features and neighbor mention compatibility scores as the feature vector INLINEFORM1 , and construct the subgraph structure representation INLINEFORM2 as the inputs of NCEL 
Neural Collective Entity Linking
NCEL incorporates GCN into a deep neural network to utilize structured graph information for collectively feature abstraction, while differs from conventional GCN in the way of applying the graph  Instead of the entire graph, only a subset of nodes is “visible" to each node in our proposed method, and then the overall structured information shall be reached in a chain-like way  Fixing the size of the subset, NCEL is further speeded up by batch techniques and GPUs, and is efficient to large-scale data 
Graph Convolutional Network
GCNs are a type of neural network model that deals with structured data  It takes a graph as an input and output labels for each node  As a simplification of spectral graph convolutions, the main idea of BIBREF26 is similar to a propagation model: to enhance the features of a node according to its neighbor nodes  The formulation is as follows: INLINEFORM0
where INLINEFORM0 is a normalized adjacent matrix of the input graph with self-connection, INLINEFORM1 and INLINEFORM2 are the hidden states and weights in the INLINEFORM3 -th layer, and INLINEFORM4 is a non-linear activation, such as ReLu 
Model Architecture
As shown in Figure FIGREF10 , NCEL identifies the correct candidate INLINEFORM0 for the mention INLINEFORM1 by using vectorial features as well as structured relatedness with candidates of neighbor mentions INLINEFORM2   Given feature vector INLINEFORM3 and subgraph representation INLINEFORM4 of each candidate INLINEFORM5 , we stack them as inputs for mention INLINEFORM6 : INLINEFORM7 , and the adjacent matrix INLINEFORM8 , where INLINEFORM9 denotes the subgraph with self-connection  We normalize INLINEFORM10 such that all rows sum to one, denoted as INLINEFORM11 , avoiding the change in the scale of the feature vectors 
Given INLINEFORM0 and INLINEFORM1 , the goal of NCEL is to find the best assignment: INLINEFORM2
where INLINEFORM0 is the output variable of candidates, and INLINEFORM1 is a probability function as follows: INLINEFORM2
where INLINEFORM0 is the score function parameters by INLINEFORM1   NCEL learns the mapping INLINEFORM2 through a neural network including three main modules: encoder, sub-graph convolution network (sub-GCN) and decoder  Next, we introduce them in turn 
Encoder The function of this module is to integrate different features by a multi-layer perceptron (MLP): INLINEFORM0
where INLINEFORM0 is the hidden states of the current mention, INLINEFORM1 and INLINEFORM2 are trainable parameters and bias  We use ReLu as the non-linear activation INLINEFORM3  
Sub-Graph Convolution Network Similar to GCN, this module learns to abstract features from the hidden state of the mention itself as well as its neighbors  Suppose INLINEFORM0 is the hidden states of the neighbor INLINEFORM1 , we stack them to expand the current hidden states of INLINEFORM2 as INLINEFORM3 , such that each row corresponds to that in the subgraph adjacent matrix INLINEFORM4   We define sub-graph convolution as: INLINEFORM5
where INLINEFORM0 is a trainable parameter 
Decoder After INLINEFORM0 iterations of sub-graph convolution, the hidden states integrate both features of INLINEFORM1 and its neighbors  A fully connected decoder maps INLINEFORM2 to the number of candidates as follows: INLINEFORM3
where INLINEFORM0  
Training
The parameters of network are trained to minimize cross-entropy of the predicted and ground truth INLINEFORM0 : INLINEFORM1
Suppose there are INLINEFORM0 documents in training corpus, each document has a set of mentions INLINEFORM1 , leading to totally INLINEFORM2 mention sets  The overall objective function is as follows: INLINEFORM3
Experiments
To avoid overfitting with some dataset, we train NCEL using collected Wikipedia hyperlinks instead of specific annotated data  We then evaluate the trained model on five different benchmarks to verify the linking precision as well as the generalization ability  Furthermore, we investigate the effectiveness of key modules in NCEL and give qualitative results for comprehensive analysis 
Baselines and Datasets
We compare NCEL with the following state-of-the-art EL methods including three local models and three types of global models:
Local models: He BIBREF29 and Chisholm BIBREF6 beat many global models by using auto-encoders and web links, respectively, and NTEE BIBREF16 achieves the best performance based on joint embeddings of words and entities 
Iterative model: AIDA BIBREF22 links entities by iteratively finding a dense subgraph 
Loopy Belief Propagation: Globerson BIBREF18 and PBoH BIBREF30 introduce LBP BIBREF31 techniques for collective inference, and Ganea BIBREF24 solves the global training problem via truncated fitting LBP 
PageRank/Random Walk: Boosting BIBREF32 , AGDISTISG BIBREF33 , Babelfy BIBREF34 , WAT BIBREF35 , xLisa BIBREF36 and WNED BIBREF19 performs PageRank BIBREF37 or random walk BIBREF38 on the mention-entity graph and use the convergence score for disambiguation 
For fairly comparison, we report the original scores of the baselines in the papers  Following these methods, we evaluate NCEL on the following five datasets: (1) CoNLL-YAGO BIBREF22 : the CoNLL 2003 shared task including testa of 4791 mentions in 216 documents, and testb of 4485 mentions in 213 documents  (2) TAC2010 BIBREF39 : constructed for the Text Analysis Conference that comprises 676 mentions in 352 documents for testing  (3) ACE2004 BIBREF23 : a subset of ACE2004 co-reference documents including 248 mentions in 35 documents, which is annotated by Amazon Mechanical Turk  (4) AQUAINT BIBREF40 : 50 news articles including 699 mentions from three different news agencies  (5) WW BIBREF19 : a new benchmark with balanced prior distributions of mentions, leading to a hard case of disambiguation  It has 6374 mentions in 310 documents automatically extracted from Wikipedia 
Training Details and Running Time Analysis
Training We collect 50,000 Wikipedia articles according to the number of its hyperlinks as our training data  For efficiency, we trim the articles to the first three paragraphs leading to 1,035,665 mentions in total  Using CoNLL-Test A as the development set, we evaluate the trained NCEL on the above benchmarks  We set context window to 20, neighbor mention window to 6, and top INLINEFORM0 candidates for each mention  We use two layers with 2000 and 1 hidden units in MLP encoder, and 3 layers in sub-GCN  We use early stop and fine tune the embeddings  With a batch size of 16, nearly 3 epochs cost less than 15 minutes on the server with 20 core CPU and the GeForce GTX 1080Ti GPU with 12Gb memory  We use standard Precision, Recall and F1 at mention level (Micro) and at the document level (Macro) as measurements 
Complexity Analysis Compared with local methods, the main disadvantage of collective methods is high complexity and expensive costs  Suppose there are INLINEFORM0 mentions in documents on average, among these global models, NCEL not surprisingly has the lowest time complexity INLINEFORM1 since it only considers adjacent mentions, where INLINEFORM2 is the number of sub-GCN layers indicating the iterations until convergence  AIDA has the highest time complexity INLINEFORM3 in worst case due to exhaustive iteratively finding and sorting the graph  The LBP and PageRank/random walk based methods achieve similar high time complexity of INLINEFORM4 mainly because of the inference on the entire graph 
Results on GERBIL
GERBIL BIBREF41 is a benchmark entity annotation framework that aims to provide a unified comparison among different EL methods across datasets including ACE2004, AQUAINT and CoNLL  We compare NCEL with the global models that report the performance on GERBIL 
As shown in Table TABREF26 , NCEL achieves the best performance in most cases with an average gain of 2% on Micro F1 and 3% Macro F1  The baseline methods also achieve competitive results on some datasets but fail to adapt to the others  For example, AIDA and xLisa perform quite well on ACE2004 but poorly on other datasets, or WAT, PBoH, and WNED have a favorable performance on CoNLL but lower values on ACE2004 and AQUAINT  Our proposed method performs consistently well on all datasets that demonstrates the good generalization ability 
Results on TAC2010 and WW
In this section, we investigate the effectiveness of NCEL in the “easy" and “hard" datasets, respectively  Particularly, TAC2010, which has two mentions per document on average (Section SECREF19 ) and high prior probabilities of correct candidates (Figure FIGREF28 ), is regarded as the “easy" case for EL, and WW is the “hard" case since it has the most mentions with balanced prior probabilities BIBREF19   Besides, we further compare the impact of key modules by removing the following part from NCEL: global features (NCEL-local), attention (NCEL-noatt), embedding features (NCEL-noemb), and the impact of the prior probability (prior) 
The results are shown in Table FIGREF28 and Table FIGREF28   We can see the average linking precision (Micro) of WW is lower than that of TAC2010, and NCEL outperforms all baseline methods in both easy and hard cases  In the “easy" case, local models have similar performance with global models since only little global information is available (2 mentions per document)  Besides, NN-based models, NTEE and NCEL-local, perform significantly better than others including most global models, demonstrating that the effectiveness of neural models deals with the first limitation in the introduction 
Impact of NCEL Modules
As shown in Figure FIGREF28 , the prior probability performs quite well in TAC2010 but poorly in WW  Compared with NCEL-local, the global module in NCEL brings more improvements in the “hard" case than that for “easy" dataset, because local features are discriminative enough in most cases of TAC2010, and global information becomes quite helpful when local features cannot handle  That is, our propose collective model is robust and shows a good generalization ability to difficult EL  The improvements by each main module are relatively small in TAC2010, while the modules of attention and embedding features show non-negligible impacts in WW (even worse than local model), mainly because WW contains much noise, and these two modules are effective in improving the robustness to noise and the ability of generalization by selecting informative words and providing more accurate semantics, respectively 
Qualitative Analysis
The results of example in Figure FIGREF1 are shown in Table TABREF30 , which is from CoNLL testa dataset  For mention Essex, although both NCEL and NCEL-local correctly identify entity Essex County Cricket Club, NCEL outputs higher probability due to the enhancement of neighbor mentions  Moreover, for mention England, NCEL-local cannot find enough disambiguation clues from its context words, such as surplus and requirements, and thus assigns a higher probability of 0 42 to the country England according to the prior probability  Collectively, NCEL correctly identifies England cricket team with a probability of 0 72 as compared with 0 20 in NCEL-local with the help of its neighbor mention Essex 
Conclusion
In this paper, we propose a neural model for collective entity linking that is end-to-end trainable  It applies GCN on subgraphs instead of the entire entity graph to efficiently learn features from both local and global information  We design an attention mechanism that endows NCEL robust to noisy data  Trained on collected Wikipedia hyperlinks, NCEL outperforms the state-of-the-art collective methods across five different datasets  Besides, further analysis of the impacts of main modules as well as qualitative results demonstrates its effectiveness 
In the future, we will extend our method into cross-lingual settings to help link entities in low-resourced languages by exploiting rich knowledge from high-resourced languages, and deal with NIL entities to facilitate specific applications 
Acknowledgments
The work is supported by National Key Research and Development Program of China (2017YFB1002101), NSFC key project (U1736204, 61661146007), and THUNUS NExT Co-Lab 
Context 3:  Therefore, the three main conditions are the following:
improvement of original poster's sentiment: we hypothesize that instillation of hope leads to a higher sentiment score;
posts containing negative personal experiences: hope can be instilled in someone who shares a negative personal experience;
comments containing positive personal experiences: in order to instill hope, commenting posters must show to original posters an overall positive personal experience  To detect positive personal experience, we require the presence of statements in comments and a positive sentiment of comments replying to negative posts 
Altruism consists of peers offering support, reassurance, suggestions and insight, since they share similar problems with one another BIBREF7  The experience of finding that a patient can be of value to others is refreshing and boosts self-esteem BIBREF7  However, in the current study we focus on testing whether commenting posters are altruists or not  We do not test whether the altruistic behavior leads to an improvement on the altruist itself  For these reasons, we define three main conditions:
improvement of original poster's sentiment: we hypothesize that supportive and reassuring statements improve the sentiment score of the original poster;
posts contains negative personal experiences: users offer support, reassurance and suggestion when facing a negative personal experience of the original poster;
comments containing positive statements: either supportive or reassuring statements show by definition a positive intended emotional communication  Thus comments to the post should consist of positive sentiment statements 
Consequently, a conversation containing the aforementioned therapeutic factors should satisfy the following conditions in terms of NLP: Sentiment Analysis and Dialogue Acts 
original posters have a higher sentiment score at the end of the thread than at the beginning;
the original post consists mostly of polarised statements;
the presence of a significant amount of statements in comments, since both support and sharing similar negative experiences can be represented as statements;
both negative and positive statements in comments lead to higher final sentiment score of the original poster 
Datasets
We verify the presence of therapeutic factors in two social media datasets: OSG and Twitter  The first dataset is crawled from an on-line support groups website, and the second dataset consists of a small sample of Twitter conversation threads  Since the former consists of multi-threaded conversations, we apply a pre-processing to extract conversation threads to provide a fair comparison with the Twitter dataset  An example conversation from each data source is presented in Figure FIGREF19 
Datasets ::: Twitter
We have downloaded 1,873 Twitter conversation threads, roughly 14k tweets, from a publicly available resource that were previously pre-processed and have conversation threads extracted  A conversation in the dataset consists of at least 4 tweets  Even though, according to BIBREF23, Twitter is broadly applicable to public health research, our expectation is that it contains less therapeutic conversations in comparison to specialized on-line support forums 
Datasets ::: OSG
Our data has been developed by crawling and pre-processing an OSG web forum  The forum has a great variety of different groups such as depression, anxiety, stress, relationship, cancer, sexually transmitted diseases, etc  Each conversation starts with one post and can contain multiple comments  Each post or comment is represented by a poster, a timestamp, a list of users it is referencing to, thread id, a comment id and a conversation id  The thread id is the same for comments replying to each other, otherwise it is different  The thread id is increasing with time  Thus, it provides ordering among threads; whereas the timestamp provides ordering in the thread 
Each conversation can belong to multiple groups  Consequently, the dataset needs to be processed to remove duplicates  The dataset resulting after de-duplication contains 295 thousand conversations, each conversation contains on average 6 comments  In total, there are 1 5 million comments  Since the created dataset is multi-threaded, we need to extract conversation threads, to eliminate paths not relevant to the original post 
Datasets ::: OSG ::: Conversation Thread Extraction
The thread extraction algorithm is heuristic-based and consists of two steps: (1) creation of a tree, based on a post written by a user and the related comments and (2) transformation of the tree into a list of threads 
The tree creation is an extension of the approach of BIBREF24, where first a graph of conversation is constructed  In the approach, direct replies to a post are attached to the first nesting level and subsequent comments to increasing nesting levels  In our approach, we also exploit comments' features 
The tree creation is performed without processing the content of comments, which allows us to process posts and comments of any length efficiently  The heuristic used in the process is based on three simplifying assumptions:
Unless there is a specific reference to another comment or a user, comments are attached to the original post 
When replying, the commenting poster is always replying to the original post or some other comment  Unless specified otherwise, it is assumed that it is a response to the previous (in time) post/comment 
Subsequent comments by the same poster are part of the same thread 
To evaluate the performance of the thread extraction algorithm, 2 annotators have manually constructed the trees for 100 conversations  The performance of the algorithm on this set of 100 conversations is evaluated using accuracy and standard Information Retrieval evaluation metrics of precision, recall, and F$_1$ measure  The results are reported in Table TABREF28 together with random and majority baselines  The turn-level percent agreement between the 2 annotators is 97 99% and Cohen's Kappa Coefficient is 83 80% 
Datasets ::: Data Representation
For both data sources, Twitter and OSG with extracted threads, posts and comments are tokenized and sentence split  Each sentence is passed through Sentiment Analysis and Dialogue Act tagging  Since a post or a comment can contain multiple sentences, therefore multiple Dialogue Acts, it is represented as as a one-hot encoding, where each position represents a Dialogue Act 
For Sentiment Analysis we use a lexicon-based sentiment analyser introduced by BIBREF25  For Dialogue Act tagging, on the other hand, we make use of a model trained on NPSChat corpus BIBREF22 following the approach of BIBREF26 
Analysis
As we mentioned in Section 3, the presence of each of the therapeutic conditions under analysis is a necessary for a conversation to be considered to have therapeutic factors  In this section we present the results of our analysis with respect to these conditions 
Analysis ::: Change in Sentiment score of Original Posters
The first condition which we test is the sentiment change in conversation threads, comparing the initial and final sentiment scores (i e  posts' scores) of the original poster  The results of the analysis are presented in Figure FIGREF33  In the figure we can observe that the distribution of the sentiment change in the two datasets is different  While in Twitter the amount of conversations that lead to the increase of sentiment score is roughly equal to the amount of conversations that lead to the decrease of sentiment score; the situation is different for OSG  In OSG, the amount of conversations that lead to the increase of sentiment score is considerably higher 
Figure FIGREF34 provides a more fine grained analysis, where we additionally analyse the sentiment change in nominal polarity terms – negative and positive  In OSG, the number of users that changed polarity from negative to positive is more than the double of the users that have changed the polarity from positive to negative  In Twitter, on the other hand, the users mostly changed polarity from positive to negative  Results of the analysis suggest that in OSG , sentiment increases and users tend to change polarity from negative to positive, whereas in Twitter sentiment tends to decrease  Verification of this condition alone indicates that the ratio of potentially therapeutic conversations in Twitter is lower 
Analysis ::: Structure of Posts and Comments
Table TABREF36 presents the distribution of automatically predicted per-sentence Dialogue Acts in the datasets  The most frequent tag is statement in both  In Table TABREF37, on the other hand, we present the distribution of post and comment structures in terms of automatically predicted Dialogue Act tags

ground_truth: NCEL consistently outperforms various baselines with a favorable generalization ability

prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. How large is the Twitter dataset?
Context 3:  The structure is an unordered set of tags in the post or comment  From the table we can observe that the distribution of tag sets is similar between posts and comments  In both cases the most common set is statement only  However, conversations containing only statement, emphasis or question posts and comments predominantly appear in Twitter  Which is expected due to the shorter length of Twitter posts and comments 
We can also observe that the original posters tend to ask more questions than the commenting posters – 19 83% for posts vs  11 21% for comments (summed)  This suggests that the original posters frequently ask either for suggestion or confirmation of their points of view or their disconfirmation  However, the high presence of personal experiences is supported by the high number of posts containing only statements 
High number of statement tags in comments suggests that users reply either with supporting or empathic statements or personal experience  However, 6 39% of comments contain accept and reject tags, which mark the degree to which a speaker accepts some previous proposal, plan, opinion, or statement BIBREF20  The described Dialogue Act tags are often used when commenting posters discuss original poster's point of view  For instance, “It's true  I felt the same ” – {Accept, Statement} or “Well no  You're not alone” – {Reject, Statement}  The datasets differ with respect to the distribution of these Dialogue Acts tags, they appear more frequently in OSG 
Analysis ::: Sentiment of Posts and Comments
Table TABREF39 presents the distribution of sentiment polarity in post and comment statements (i e  sentences tagged as statement)  For OSG, the predominant sentiment label of statements is positive and it is the highest for both posts and comments  However, the difference between the amounts of positive and negative statements is higher for the replying comments (34 5% vs  42 5%)  For Twitter, on the other hand, the predominant sentiment label of statements is neutral and the polarity distribution between posts and comments is very close  One particular observation is that the ratio of negative statements is higher in OSG for both posts and comments than in Twitter, which supports the idea of sharing negative experiences 
Further we analyze whether the sentiment of a comment (i e  the replying user) is affected by the sentiment of the original post (i e  the user being replied to), which will imply that the users adapt their behaviour with respect to the post's sentiment  For the analysis, we split the datasets into three buckets according to the posts' sentiment score – negative, neutral, or positive, and represent each conversation in terms of percentages of comments (replies) with each sentiment label  The buckets are then compared using t-test for statistically significant differences 
Table TABREF40 presents the distribution of sentiment labels with respect to the post's sentiment score  The patterns of distribution are similar across the datasets  We can observe that overall, replies tend to have a positive sentiment, which suggests that replying posters tend to have a positive attitude  However, the ratio of positive comments is higher for OSG than for Twitter 
The results of the Welch's t-test on OSG data reveal that there are statistically significant differences in the distribution of replying comments' sentiment between conversations with positive and negative starting posts  A positive post tends to get significantly more positive replies  Similarly, a negative post tends to get significantly more negative replies (both with $p < 0 01$) 
Table TABREF41 presents the distribution of the sentiment labels of the final text provided by the original poster with respect to the sentiment polarity of the comments  The results indicate that OSG participants are more supportive, as the majority of conversations end in a positive final sentiment regardless of the sentiment of comments  We can also observe that negative comments in OSG lead to positive sentiment, which supports the idea of sharing the negative experiences, thus presence of therapeutic factors  For Twitter, on the other hand, only positive comments lead to the positive final sentiments, whereas other comments lead predominantly to neutral final sentiments 
Our analysis in terms of sentiment and Dialogue Acts supports the presence of the three selected therapeutic factors – Universality, Altruism and Instillation of Hope – in OSG more than in Twitter  The main contributors to this conclusion are the facts that there is more positive change in the sentiment of the original posters in OSG (people seeking support) and that in OSG even negative and neutral comments are likely to lead to positive changes 
Conclusion
In this work, we propose a methodology to automatically analyse online social platforms for the presence of therapeutic factors (i e  Universality, Altruism and Instillation of Hope)  We evaluate our approach on two on-line platforms, Twitter and an OSG web forum  We apply NLP techniques of Sentiment Analysis and Dialogue Act tagging to automatically verify the presence of therapeutic factors, which allows us to analyse larger amounts of conversational data (as compared to previous studies) 
Our analysis indicates that OSG conversations satisfy higher number of conditions approximating therapeutic factors than Twitter conversations  Given this outcome, we postulate that users who join support group websites spontaneously seem to benefit from it  Indeed, as shown in Section SECREF5, the original posters who interact with others by replying to comments, have benefited from an improvement of their emotional state 
We would like to reemphasise that the conditions for the therapeutic factors are necessary but not sufficient; since our analysis focuses on the structure of conversations, being agnostic to the content  NLP, however, allows us to strengthen our approximations even further  Thus, the further extension of our work is also augmentation of our study with other language analysis metrics and their correlation with human annotation 
It should be noted that the proposed approach is an approximation of the tedious tasks of annotation of conversations by experts versed in the therapeutic factors and their associated theories  Even though we can use Sentiment Analysis to detect the existence of therapeutic factors, we cannot differentiate between Altruism and Instillation of Hope, as this requires differentiation between emotional state of the user and the intended emotional communication  Thus, the natural extensions of this work are differentiation between different therapeutic factors and comparison of the proposed analysis to the human evaluation 
Although we acknowledge that the proposed methodology does not serve as a replacement of manual analysis of OSG for the presence of therapeutic factors, we believe that it could facilitate and supplement this process  The method can serve as a tool for general practitioners and psychologists who can use it as an additional source of information regarding their patients’ condition and, in turn, offer a more personalised support that is better tailored to individual therapeutic needs 

ground_truth: 1,873 Twitter conversation threads, roughly 14k tweets

1,873 Twitter conversation threads, roughly 14k tweets