prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. Which methods are considered to find examples of biases and unwarranted inferences??
Context 1: Introduction
The Flickr30K dataset BIBREF0 is a collection of over 30,000 images with 5 crowdsourced descriptions each  It is commonly used to train and evaluate neural network models that generate image descriptions (e g  BIBREF2 )  An untested assumption behind the dataset is that the descriptions are based on the images, and nothing else  Here are the authors (about the Flickr8K dataset, a subset of Flickr30K):
“By asking people to describe the people, objects, scenes and activities that are shown in a picture without giving them any further information about the context in which the picture was taken, we were able to obtain conceptual descriptions that focus only on the information that can be obtained from the image alone ” BIBREF1
What this assumption overlooks is the amount of interpretation or recontextualization carried out by the annotators  Let us take a concrete example  Figure FIGREF1 shows an image from the Flickr30K dataset 
This image comes with the five descriptions below  All but the first one contain information that cannot come from the image alone  Relevant parts are highlighted in bold:
We need to understand that the descriptions in the Flickr30K dataset are subjective descriptions of events  This can be a good thing: the descriptions tell us what are the salient parts of each image to the average human annotator  So the two humans in Figure FIGREF1 are relevant, but the two soap dispensers are not  But subjectivity can also result in stereotypical descriptions, in this case suggesting that the male is more likely to be the manager, and the female is more likely to be the subordinate  rashtchian2010collecting do note that some descriptions are speculative in nature, which they say hurts the accuracy and the consistency of the descriptions  But the problem is not with the lack of consistency here  Quite the contrary: the problem is that stereotypes may be pervasive enough for the data to be consistently biased  And so language models trained on this data may propagate harmful stereotypes, such as the idea that women are less suited for leadership positions 
This paper aims to give an overview of linguistic bias and unwarranted inferences resulting from stereotypes and prejudices  I will build on earlier work on linguistic bias in general BIBREF3 , providing examples from the Flickr30K data, and present a taxonomy of unwarranted inferences  Finally, I will discuss several methods to analyze the data in order to detect biases 
Stereotype-driven descriptions
Stereotypes are ideas about how other (groups of) people commonly behave and what they are likely to do  These ideas guide the way we talk about the world  I distinguish two kinds of verbal behavior that result from stereotypes: (i) linguistic bias, and (ii) unwarranted inferences  The former is discussed in more detail by beukeboom2014mechanisms, who defines linguistic bias as “a systematic asymmetry in word choice as a function of the social category to which the target belongs ” So this bias becomes visible through the distribution of terms used to describe entities in a particular category  Unwarranted inferences are the result of speculation about the image; here, the annotator goes beyond what can be glanced from the image and makes use of their knowledge and expectations about the world to provide an overly specific description  Such descriptions are directly identifiable as such, and in fact we have already seen four of them (descriptions 2–5) discussed earlier 
Linguistic bias
Generally speaking, people tend to use more concrete or specific language when they have to describe a person that does not meet their expectations  beukeboom2014mechanisms lists several linguistic `tools' that people use to mark individuals who deviate from the norm  I will mention two of them 
[leftmargin=0cm]
One well-studied example BIBREF4 , BIBREF5 is sexist language, where the sex of a person tends to be mentioned more frequently if their role or occupation is inconsistent with `traditional' gender roles (e g  female surgeon, male nurse)  Beukeboom also notes that adjectives are used to create “more narrow labels [or subtypes] for individuals who do not fit with general social category expectations” (p  3)  E g  tough woman makes an exception to the `rule' that women aren't considered to be tough 
can be used when prior beliefs about a particular social category are violated, e g  The garbage man was not stupid  See also BIBREF6  
These examples are similar in that the speaker has to put in additional effort to mark the subject for being unusual  But they differ in what we can conclude about the speaker, especially in the context of the Flickr30K data  Negations are much more overtly displaying the annotator's prior beliefs  When one annotator writes that A little boy is eating pie without utensils (image 2659046789), this immediately reveals the annotator's normative beliefs about the world: pie should be eaten with utensils  But when another annotator talks about a girls basketball game (image 8245366095), this cannot be taken as an indication that the annotator is biased about the gender of basketball players; they might just be helpful by providing a detailed description  In section 3 I will discuss how to establish whether or not there is any bias in the data regarding the use of adjectives 
Unwarranted inferences
Unwarranted inferences are statements about the subject(s) of an image that go beyond what the visual data alone can tell us  They are based on additional assumptions about the world  After inspecting a subset of the Flickr30K data, I have grouped these inferences into six categories (image examples between parentheses):
[leftmargin=0cm]
We've seen an example of this in the introduction, where the `manager' was said to be talking about job performance and scolding [a worker] in a stern lecture (8063007) 
Many dark-skinned individuals are called African-American regardless of whether the picture has been taken in the USA or not (4280272)  And people who look Asian are called Chinese (1434151732) or Japanese (4834664666) 
In image 4183120 (Figure FIGREF16 ), people sitting at a gym are said to be watching a game, even though there could be any sort of event going on  But since the location is so strongly associated with sports, crowdworkers readily make the assumption 
Quite a few annotations focus on explaining the why of the situation  For example, in image 3963038375 a man is fastening his climbing harness in order to have some fun  And in an extreme case, one annotator writes about a picture of a dancing woman that the school is having a special event in order to show the american culture on how other cultures are dealt with in parties (3636329461)  This is reminiscent of the Stereotypic Explanatory Bias BIBREF7 , which refers to “the tendency to provide relatively more explanations in descriptions of stereotype inconsistent, compared to consistent behavior” BIBREF6   So in theory, odd or surprising situations should receive more explanations, since a description alone may not make enough sense in those cases, but it is beyond the scope of this paper to test whether or not the Flickr30K data suffers from the SEB 
Older people with children around them are commonly seen as parents (5287405), small children as siblings (205842), men and women as lovers (4429660), groups of young people as friends (36979) 
Annotators will often guess the status or occupation of people in an image  Sometimes these guesses are relatively general (e g  college-aged people being called students in image 36979), but other times these are very specific (e g  a man in a workshop being called a graphics designer, 5867606) 
Detecting stereotype-driven descriptions
In order to get an idea of the kinds of stereotype-driven descriptions that are in the Flickr30K dataset, I made a browser-based annotation tool that shows both the images and their associated descriptions  You can simply leaf through the images by clicking `Next' or `Random' until you find an interesting pattern 
Ethnicity/race
One interesting pattern is that the ethnicity/race of babies doesn't seem to be mentioned unless the baby is black or asian  In other words: white seems to be the default, and others seem to be marked  How can we tell whether or not the data is actually biased?
We don't know whether or not an entity belongs to a particular social class (in this case: ethnic group) until it is marked as such  But we can approximate the proportion by looking at all the images where the annotators have used a marker (in this case: adjectives like black, white, asian), and for those images count how many descriptions (out of five) contain a marker  This gives us an upper bound that tells us how often ethnicity is indicated by the annotators  Note that this upper bound lies somewhere between 20% (one description) and 100% (5 descriptions)  Figure TABREF22 presents count data for the ethnic marking of babies  It includes two false positives (talking about a white baby stroller rather than a white baby)  In the Asian group there is an additional complication: sometimes the mother gets marked rather than the baby  E g  An Asian woman holds a baby girl  I have counted these occurrences as well 
The numbers in Table TABREF22 are striking: there seems to be a real, systematic difference in ethnicity marking between the groups  We can take one step further and look at all the 697 pictures with the word `baby' in it  If there turn out to be disproportionately many white babies, this strengthens the conclusion that the dataset is biased 
I have manually categorized each of the baby images  There are 504 white, 66 asian, and 36 black babies  73 images do not contain a baby, and 18 images do not fall into any of the other categories  While this does bring down the average number of times each category was marked, it also increases the contrast between white babies (who get marked in less than 1% of the images) and asian/black babies (who get marked much more often)  A next step would be to see whether these observations also hold for other age groups, i e  children and adults  INLINEFORM0
Other methods
It may be difficult to spot patterns by just looking at a collection of images  Another method is to tag all descriptions with part-of-speech information, so that it becomes possible to see e g  which adjectives are most commonly used for particular nouns  One method readers may find particularly useful is to leverage the structure of Flickr30K Entities BIBREF8   This dataset enriches Flickr30K by adding coreference annotations, i e  which phrase in each description refers to the same entity in the corresponding image  I have used this data to create a coreference graph by linking all phrases that refer to the same entity  Following this, I applied Louvain clustering BIBREF9 to the coreference graph, resulting in clusters of expressions that refer to similar entities  Looking at those clusters helps to get a sense of the enormous variation in referring expressions  To get an idea of the richness of this data, here is a small sample of the phrases used to describe beards (cluster 268): a scruffy beard; a thick beard; large white beard; a bubble beard; red facial hair; a braided beard; a flaming red beard  In this case, `red facial hair' really stands out as a description; why not choose the simpler `beard' instead?
Discussion
In the previous section, I have outlined several methods to manually detect stereotypes, biases, and odd phrases  Because there are many ways in which a phrase can be biased, it is difficult to automatically detect bias from the data  So how should we deal with stereotype-driven descriptions?
Conclusion
This paper provided a taxonomy of stereotype-driven descriptions in the Flickr30K dataset  I have divided these descriptions into two classes: linguistic bias and unwarranted inferences  The former corresponds to the annotators' choice of words when confronted with an image that may or may not match their stereotypical expectancies  The latter corresponds to the tendency of annotators to go beyond what the physical data can tell us, and expand their descriptions based on their past experiences and knowledge of the world  Acknowledging these phenomena is important, because on the one hand it helps us think about what is learnable from the data, and on the other hand it serves as a warning: if we train and evaluate language models on this data, we are effectively teaching them to be biased 
I have also looked at methods to detect stereotype-driven descriptions, but due to the richness of language it is difficult to find an automated measure  Depending on whether your goal is production or interpretation, it may either be useful to suppress or to emphasize biases in human language  Finally, I have discussed stereotyping behavior as the addition of a contextual layer on top of a more basic description  This raises the question what kind of descriptions we would like our models to produce 
Acknowledgments
Thanks to Piek Vossen and Antske Fokkens for discussion, and to Desmond Elliott and an anonymous reviewer for comments on an earlier version of this paper  This research was supported by the Netherlands Organization for Scientific Research (NWO) via the Spinoza-prize awarded to Piek Vossen (SPI 30-673, 2014-2019) 
Context 2: Introduction
This paper describes our approach and results for Task 2 of the CoNLL–SIGMORPHON 2018 shared task on universal morphological reinflection BIBREF0   The task is to generate an inflected word form given its lemma and the context in which it occurs 
Morphological (re)inflection from context is of particular relevance to the field of computational linguistics: it is compelling to estimate how well a machine-learned system can capture the morphosyntactic properties of a word given its context, and map those properties to the correct surface form for a given lemma 
There are two tracks of Task 2 of CoNLL–SIGMORPHON 2018: in Track 1 the context is given in terms of word forms, lemmas and morphosyntactic descriptions (MSD); in Track 2 only word forms are available  See Table TABREF1 for an example  Task 2 is additionally split in three settings based on data size: high, medium and low, with high-resource datasets consisting of up to 70K instances per language, and low-resource datasets consisting of only about 1K instances 
The baseline provided by the shared task organisers is a seq2seq model with attention (similar to the winning system for reinflection in CoNLL–SIGMORPHON 2016, BIBREF1 ), which receives information about context through an embedding of the two words immediately adjacent to the target form  We use this baseline implementation as a starting point and achieve the best overall accuracy of 49 87 on Task 2 by introducing three augmentations to the provided baseline system: (1) We use an LSTM to encode the entire available context; (2) We employ a multi-task learning approach with the auxiliary objective of MSD prediction; and (3) We train the auxiliary component in a multilingual fashion, over sets of two to three languages 
In analysing the performance of our system, we found that encoding the full context improves performance considerably for all languages: 11 15 percentage points on average, although it also highly increases the variance in results  Multi-task learning, paired with multilingual training and subsequent monolingual finetuning, scored highest for five out of seven languages, improving accuracy by another 9 86% on average 
System Description
Our system is a modification of the provided CoNLL–SIGMORPHON 2018 baseline system, so we begin this section with a reiteration of the baseline system architecture, followed by a description of the three augmentations we introduce 
Baseline
The CoNLL–SIGMORPHON 2018 baseline is described as follows:
The system is an encoder-decoder on character sequences  It takes a lemma as input and generates a word form  The process is conditioned on the context of the lemma [   ] The baseline treats the lemma, word form and MSD of the previous and following word as context in track 1  In track 2, the baseline only considers the word forms of the previous and next word  [   ] The baseline system concatenates embeddings for context word forms, lemmas and MSDs into a context vector  The baseline then computes character embeddings for each character in the input lemma  Each of these is concatenated with a copy of the context vector  The resulting sequence of vectors is encoded using an LSTM encoder  Subsequently, an LSTM decoder generates the characters in the output word form using encoder states and an attention mechanism 
To that we add a few details regarding model size and training schedule:
the number of LSTM layers is one;
embedding size, LSTM layer size and attention layer size is 100;
models are trained for 20 epochs;
on every epoch, training data is subsampled at a rate of 0 3;
LSTM dropout is applied at a rate 0 3;
context word forms are randomly dropped at a rate of 0 1;
the Adam optimiser is used, with a default learning rate of 0 001; and
trained models are evaluated on the development data (the data for the shared task comes already split in train and dev sets) 
Our system
Here we compare and contrast our system to the baseline system  A diagram of our system is shown in Figure FIGREF4  
The idea behind this modification is to provide the encoder with access to all morpho-syntactic cues present in the sentence  In contrast to the baseline, which only encodes the immediately adjacent context of a target word, we encode the entire context  All context word forms, lemmas, and MSD tags (in Track 1) are embedded in their respective high-dimensional spaces as before, and their embeddings are concatenated  However, we now reduce the entire past context to a fixed-size vector by encoding it with a forward LSTM, and we similarly represent the future context by encoding it with a backwards LSTM 
We introduce an auxiliary objective that is meant to increase the morpho-syntactic awareness of the encoder and to regularise the learning process—the task is to predict the MSD tag of the target form  MSD tag predictions are conditioned on the context encoding, as described in UID15   Tags are generated with an LSTM one component at a time, e g  the tag PRO;NOM;SG;1 is predicted as a sequence of four components, INLINEFORM0 PRO, NOM, SG, 1 INLINEFORM1  
For every training instance, we backpropagate the sum of the main loss and the auxiliary loss without any weighting 
As MSD tags are only available in Track 1, this augmentation only applies to this track 
The parameters of the entire MSD (auxiliary-task) decoder are shared across languages 
Since a grouping of the languages based on language family would have left several languages in single-member groups (e g  Russian is the sole representative of the Slavic family), we experiment with random groupings of two to three languages  Multilingual training is performed by randomly alternating between languages for every new minibatch  We do not pass any information to the auxiliary decoder as to the source language of the signal it is receiving, as we assume abstract morpho-syntactic features are shared across languages 
After 20 epochs of multilingual training, we perform 5 epochs of monolingual finetuning for each language  For this phase, we reduce the learning rate to a tenth of the original learning rate, i e  0 0001, to ensure that the models are indeed being finetuned rather than retrained 
We keep all hyperparameters the same as in the baseline
Context 3: Introduction
From a group of small users at the time of its inception in 2009, Quora has evolved in the last few years into one of the largest community driven Q&A sites with diverse user communities  With the help of efficient content moderation/review policies and active in-house review team, efficient Quora bots, this site has emerged into one of the largest and reliable sources of Q&A on the Internet  On Quora, users can post questions, follow questions, share questions, tag them with relevant topics, follow topics, follow users apart from answering, commenting, upvoting/downvoting etc  The integrated social structure at the backbone of it and the topical organization of its rich content have made Quora unique with respect to other Q&A sites like Stack Overflow, Yahoo! Answers etc  and these are some of the prime reasons behind its popularity in recent times  Quality question posting and getting them answered are the key objectives of any Q&A site  In this study we focus on the answerability of questions on Quora, i e , whether a posted question shall eventually get answered  In Quora, the questions with no answers are referred to as “open questions”  These open questions need to be studied separately to understand the reason behind their not being answered or to be precise, are there any characteristic differences between `open' questions and the answered ones  For example, the question “What are the most promising advances in the treatment of traumatic brain injuries?” was posted on Quora on 23rd June, 2011 and got its first answer after almost 2 years on 22nd April, 2013  The reason that this question remained open so long might be the hardness of answering it and the lack of visibility and experts in the domain  Therefore, it is important to identify the open questions and take measures based on the types - poor quality questions can be removed from Quora and the good quality questions can be promoted so that they get more visibility and are eventually routed to topical experts for better answers 
Characterization of the questions based on question quality requires expert human interventions often judging if a question would remain open based on factors like if it is subjective, controversial, open-ended, vague/imprecise, ill-formed, off-topic, ambiguous, uninteresting etc  Collecting judgment data for thousands of question posts is a very expensive process  Therefore, such an experiment can be done only for a small set of questions and it would be practically impossible to scale it up for the entire collection of posts on the Q&A site  In this work, we show that appropriate quantification of various linguistic activities can naturally correspond to many of the judgment factors mentioned above (see table 2 for a collection of examples)  These quantities encoding such linguistic activities can be easily measured for each question post and thus helps us to have an alternative mechanism to characterize the answerability on the Q&A site 
There are several research works done in Q&A focusing on content of posts  BIBREF0 exploit community feedback to identify high quality content on Yahoo! Answers  BIBREF1 use textual features to predict answer quality on Yahoo! Answers  BIBREF2 , investigate predictors of answer quality through a comparative, controlled field study of user responses  BIBREF3 study the problem of how long questions remain unanswered  BIBREF4 propose a prediction model on how many answers a question shall receive

ground_truth: spot patterns by just looking at a collection of images, tag all descriptions with part-of-speech information, I applied Louvain clustering

Looking for adjectives marking the noun "baby" and also looking for most-common adjectives related to certain nouns using POS-tagging

prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. What architecture does the encoder have?
Context 2:  Training data is split 90:10 for training and validation  We train our models for 50 epochs, adding early stopping with a tolerance of five epochs of no improvement in the validation loss  We do not subsample from the training data 
We train models for 50 different random combinations of two to three languages in Track 1, and 50 monolingual models for each language in Track 2  Instead of picking the single model that performs best on the development set and thus risking to select a model that highly overfits that data, we use an ensemble of the five best models, and make the final prediction for a given target form with a majority vote over the five predictions 
Results and Discussion
Test results are listed in Table TABREF17   Our system outperforms the baseline for all settings and languages in Track 1 and for almost all in Track 2—only in the high resource setting is our system not definitively superior to the baseline 
Interestingly, our results in the low resource setting are often higher for Track 2 than for Track 1, even though contextual information is less explicit in the Track 2 data and the multilingual multi-tasking approach does not apply to this track  We interpret this finding as an indicator that a simpler model with fewer parameters works better in a setting of limited training data  Nevertheless, we focus on the low resource setting in the analysis below due to time limitations  As our Track 1 results are still substantially higher than the baseline results, we consider this analysis valid and insightful 
Ablation Study
We analyse the incremental effect of the different features in our system, focusing on the low-resource setting in Track 1 and using development data 
Encoding the entire context with an LSTM highly increases the variance of the observed results  So we trained fifty models for each language and each architecture  Figure FIGREF23 visualises the means and standard deviations over the trained models  In addition, we visualise the average accuracy for the five best models for each language and architecture, as these are the models we use in the final ensemble prediction  Below we refer to these numbers only 
The results indicate that encoding the full context with an LSTM highly enhances the performance of the model, by 11 15% on average  This observation explains the high results we obtain also for Track 2 
Adding the auxiliary objective of MSD prediction has a variable effect: for four languages (de, en, es, and sv) the effect is positive, while for the rest it is negative  We consider this to be an issue of insufficient data for the training of the auxiliary component in the low resource setting we are working with 
We indeed see results improving drastically with the introduction of multilingual training, with multilingual results being 7 96% higher than monolingual ones on average 
We studied the five best models for each language as emerging from the multilingual training (listed in Table TABREF27 ) and found no strong linguistic patterns  The en–sv pairing seems to yield good models for these languages, which could be explained in terms of their common language family and similar morphology  The other natural pairings, however, fr–es, and de–sv, are not so frequent among the best models for these pairs of languages 
Finally, monolingual finetuning improves accuracy across the board, as one would expect, by 2 72% on average 
The final observation to be made based on this breakdown of results is that the multi-tasking approach paired with multilingual training and subsequent monolingual finetuning outperforms the other architectures for five out of seven languages: de, en, fr, ru and sv  For the other two languages in the dataset, es and fi, the difference between this approach and the approach that emerged as best for them is less than 1%  The overall improvement of the multilingual multi-tasking approach over the baseline is 18 30% 
Error analysis
Here we study the errors produced by our system on the English test set to better understand the remaining shortcomings of the approach  A small portion of the wrong predictions point to an incorrect interpretation of the morpho-syntactic conditioning of the context, e g  the system predicted plan instead of plans in the context Our _ include raising private capital  The majority of wrong predictions, however, are nonsensical, like bomb for job, fify for fixing, and gnderrate for understand  This observation suggests that generally the system did not learn to copy the characters of lemma into inflected form, which is all it needs to do in a large number of cases  This issue could be alleviated with simple data augmentation techniques that encourage autoencoding BIBREF2  
MSD prediction
Figure FIGREF32 summarises the average MSD-prediction accuracy for the multi-tasking experiments discussed above  Accuracy here is generally higher than on the main task, with the multilingual finetuned setup for Spanish and the monolingual setup for French scoring best: 66 59% and 65 35%, respectively  This observation illustrates the added difficulty of generating the correct surface form even when the morphosyntactic description has been identified correctly 
We observe some correlation between these numbers and accuracy on the main task: for de, en, ru and sv, the brown, pink and blue bars here pattern in the same way as the corresponding INLINEFORM0 's in Figure FIGREF23   One notable exception to this pattern is fr where inflection gains a lot from multilingual training, while MSD prediction suffers greatly  Notice that the magnitude of change is not always the same, however, even when the general direction matches: for ru, for example, multilingual training benefits inflection much more than in benefits MSD prediction, even though the MSD decoder is the only component that is actually shared between languages  This observation illustrates the two-fold effect of multi-task training: an auxiliary task can either inform the main task through the parameters the two tasks share, or it can help the main task learning through its regularising effect 
Related Work
Our system is inspired by previous work on multi-task learning and multi-lingual learning, mainly building on two intuitions: (1) jointly learning related tasks tends to be beneficial BIBREF3 , BIBREF4 , BIBREF5 , BIBREF6 , BIBREF7 ; and (2) jointly learning related languages in an MTL-inspired framework tends to be beneficial BIBREF8 , BIBREF9 , BIBREF10   In the context of computational morphology, multi-lingual approaches have previously been employed for morphological reinflection BIBREF2 and for paradigm completion BIBREF11   In both of these cases, however, the available datasets covered more languages, 40 and 21, respectively, which allowed for linguistically-motivated language groupings and for parameter sharing directly on the level of characters  BIBREF10 explore parameter sharing between related languages for dependency parsing, and find that sharing is more beneficial in the case of closely related languages 
Conclusions
In this paper we described our system for the CoNLL–SIGMORPHON 2018 shared task on Universal Morphological Reinflection, Task 2, which achieved the best performance out of all systems submitted, an overall accuracy of 49 87  We showed in an ablation study that this is due to three core innovations, which extend a character-based encoder-decoder model: (1) a wide context window, encoding the entire available context; (2) multi-task learning with the auxiliary task of MSD prediction, which acts as a regulariser; (3) a multilingual approach, exploiting information across languages  In future work we aim to gain better understanding of the increase in variance of the results introduced by each of our modifications and the reasons for the varying effect of multi-task learning for different languages 
Acknowledgements
We gratefully acknowledge the support of the NVIDIA Corporation with the donation of the Titan Xp GPU used for this research 
Context 3:  BIBREF5 analyze and predict unanswered questions on Yahoo Answers  BIBREF6 study question quality in Yahoo! Answers 
Dataset description
We obtained our Quora dataset BIBREF7 through web-based crawls between June 2014 to August 2014  This crawling exercise has resulted in the accumulation of a massive Q&A dataset spanning over a period of over four years starting from January 2010 to May 2014  We initiated crawling with 100 questions randomly selected from different topics so that different genre of questions can be covered  The crawling of the questions follow a BFS pattern through the related question links  We obtained 822,040 unique questions across 80,253 different topics with a total of 1,833,125 answers to these questions  For each question, we separately crawl their revision logs that contain different types of edit information for the question and the activity log of the question asker 
Linguistic activities on Quora
In this section, we identify various linguistic activities on Quora and propose quantifications of the language usage patterns in this Q&A site  In particular, we show that there exists significant differences in the linguistic structure of the open and the answered questions  Note that most of the measures that we define are simple, intuitive and can be easily obtained automatically from the data (without manual intervention)  Therefore the framework is practical, inexpensive and highly scalable 
Content of a question text is important to attract people and make them engage more toward it  The linguistic structure (i e , the usage of POS tags, the use of Out-of-Vocabulary words, character usage etc ) one adopts are key factors for answerability of questions  We shall discuss the linguistic structure that often represents the writing style of a question asker 
In fig 1 (a), we observe that askers of open questions generally use more no  of words compared to answered questions  To understand the nature of words (standard English words or chat-like words frequently used in social media) used in the text, we compare the words with GNU Aspell dictionary to see whether they are present in the dictionary or not  We observe that both open questions and answered questions follow similar distribution (see fig 1 (b))  Part-of-Speech (POS) tags are indicators of grammatical aspects of texts  To observe how the Part-of-Speech tags are distributed in the question texts, we define a diversity metric  We use the standard CMU POS tagger BIBREF8 for identifying the POS tags of the constituent words in the question

ground_truth: LSTM

LSTM

prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. Do the answered questions measure for the usefulness of the answer?
Context 3:  We define the POS tag diversity (POSDiv) of a question $q_i$ as follows: $POSDiv(q_i) = -\sum _{j \in pos_{set}}p_j\times \log (p_j)$ where $p_j$ is the probability of the $j^{th}$ POS in the set of POS tags  Fig 1 (c) shows that the answered questions have lower POS tag diversity compared to open questions  Question texts undergo several edits so that their readability and the engagement toward them are enhanced  It is interesting to identify how far such edits can make the question different from the original version of it  To capture this phenomena, we have adopted ROUGE-LCS recall BIBREF9 from the domain of text summarization  Higher the recall value, lesser are the changes in the question text  From fig 1 (d), we observe that open questions tend to have higher recall compared to the answered ones which suggests that they have not gone through much of text editing thus allowing for almost no scope of readability enhancement 
Psycholinguistic analysis:
The way an individual talks or writes, give us clue to his/her linguistic, emotional, and cognitive states  A question asker's linguistic, emotional, cognitive states are also revealed through the language he/she uses in the question text  In order to capture such psycholinguistic aspects of the asker, we use Linguistic Inquiry and Word Count (LIWC) BIBREF10 that analyzes various emotional, cognitive, and structural components present in individuals' written texts  LIWC takes a text document as input and outputs a score for the input for each of the LIWC categories such as linguistic (part-of-speech of the words, function words etc ) and psychological categories (social, anger, positive emotion, negative emotion, sadness etc ) based on the writing style and psychometric properties of the document  In table 1 , we perform a comparative analysis of the asker's psycholinguistic state while asking an open question and an answered question 
Askers of open questions use more function words, impersonal pronouns, articles on an average whereas asker of answered questions use more personal pronouns, conjunctions and adverbs to describe their questions  Essentially, open questions lack content words compared to answered questions which, in turn, affects the readability of the question  As far as the psychological aspects are concerned, answered question askers tend to use more social, family, human related words on average compared to an open question asker  The open question askers express more positive emotions whereas the answered question asker tend to express more negative emotions in their texts  Also, answered question askers are more emotionally involved and their questions reveal higher usage of anger, sadness, anxiety related words compared to that of open questions  Open questions, on the other hand, contains more sexual, body, health related words which might be reasons why they do not attract answers 
In table 2 , we show a collection of examples of open questions to illustrate that many of the above quantities based on the linguistic activities described in this section naturally correspond to the factors that human judges consider responsible for a question remaining unanswered  This is one of the prime reasons why these quantities qualify as appropriate indicators of answerability 
Prediction model
In this section, we describe the prediction framework in detail  Our goal is to predict whether a given question after a time period $t$ will be answered or not 
Linguistic styles of the question asker
The content and way of posing a question is important to attract answers  We have observed in the previous section that these linguistic as well as psycholinguistic aspects of the question asker are discriminatory factors  For the prediction, we use the following features:

ground_truth: No