prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. Which sports clubs are the targets?
Context 1: Introduction
Stance detection (also called stance identification or stance classification) is one of the considerably recent research topics in natural language processing (NLP)  It is usually defined as a classification problem where for a text and target pair, the stance of the author of the text for that target is expected as a classification output from the set: {Favor, Against, Neither} BIBREF0  
Stance detection is usually considered as a subtask of sentiment analysis (opinion mining) BIBREF1 topic in NLP  Both are mostly performed on social media texts, particularly on tweets, hence both are important components of social media analysis  Nevertheless, in sentiment analysis, the sentiment of the author of a piece of text usually as Positive, Negative, and Neutral is explored while in stance detection, the stance of the author of the text for a particular target (an entity, event, etc ) either explicitly or implicitly referred to in the text is considered  Like sentiment analysis, stance detection systems can be valuable components of information retrieval and other text analysis systems BIBREF0  
Previous work on stance detection include BIBREF2 where a stance classifier based on sentiment and arguing features is proposed in addition to an arguing lexicon automatically compiled  The ultimate approach performs better than distribution-based and uni-gram-based baseline systems BIBREF2   In BIBREF3 , the authors show that the use of dialogue structure improves stance detection in on-line debates  In BIBREF4 , Hasan and Ng carry out stance detection experiments using different machine learning algorithms, training data sets, features, and inter-post constraints in on-line debates, and draw insightful conclusions based on these experiments  For instance, they find that sequence models like HMMs perform better at stance detection when compared with non-sequence models like Naive Bayes (NB) BIBREF4   In another related study BIBREF5 , the authors conclude that topic-independent features can be exploited for disagreement detection in on-line dialogues  The employed features include agreement, cue words, denial, hedges, duration, polarity, and punctuation BIBREF5   Stance detection on a corpus of student essays is considered in BIBREF6   After using linguistically-motivated feature sets together with multivalued NB and SVM as the learning models, the authors conclude that they outperform two baseline approaches BIBREF6   In BIBREF7 , the author claims that Wikipedia can be used to determine stances about controversial topics based on their previous work regarding controversy extraction on the Web 
Among more recent related work, in BIBREF8 stance detection for unseen targets is studied and bidirectional conditional encoding is employed  The authors state that their approach achieves state-of-the art performance rates BIBREF8 on SemEval 2016 Twitter Stance Detection corpus BIBREF0   In BIBREF9 , a stance-community detection approach called SCIFNET is proposed  SCIFNET creates networks of people who are stance targets, automatically from the related document collections BIBREF9 using stance expansion and refinement techniques to arrive at stance-coherent networks  A tweet data set annotated with stance information regarding six predefined targets is proposed in BIBREF10 where this data set is annotated through crowdsourcing  The authors indicate that the data set is also annotated with sentiment information in addition to stance, so it can help reveal associations between stance and sentiment BIBREF10   Lastly, in BIBREF0 , SemEval 2016's aforementioned shared task on Twitter Stance Detection is described  Also provided are the results of the evaluations of 19 systems participating in two subtasks (one with training data set provided and the other without an annotated data set) of the shared task BIBREF0  
In this paper, we present a tweet data set in Turkish annotated with stance information, where the corresponding annotations are made publicly available  The domain of the tweets comprises two popular football clubs which constitute the targets of the tweets included  We also provide the evaluation results of SVM classifiers (for each target) on this data set using unigram, bigram, and hashtag features 
To the best of our knowledge, the current study is the first one to target at stance detection in Turkish tweets  Together with the provided annotated data set and the corresponding evaluations with the aforementioned SVM classifiers which can be used as baseline systems, our study will hopefully help increase social media analysis studies on Turkish content 
The rest of the paper is organized as follows: In Section SECREF2 , we describe our tweet data set annotated with the target and stance information  Section SECREF3 includes the details of our SVM-based stance classifiers and their evaluation results with discussions  Section SECREF4 includes future research topics based on the current study, and finally Section SECREF5 concludes the paper with a summary 
A Stance Detection Data Set
We have decided to consider tweets about popular sports clubs as our domain for stance detection  Considerable amounts of tweets are being published for sports-related events at every instant  Hence we have determined our targets as Galatasaray (namely Target-1) and Fenerbahçe (namely, Target-2) which are two of the most popular football clubs in Turkey  As is the case for the sentiment analysis tools, the outputs of the stance detection systems on a stream of tweets about these clubs can facilitate the use of the opinions of the football followers by these clubs 
In a previous study on the identification of public health-related tweets, two tweet data sets in Turkish (each set containing 1 million random tweets) have been compiled where these sets belong to two different periods of 20 consecutive days BIBREF11   We have decided to use one of these sets (corresponding to the period between August 18 and September 6, 2015) and firstly filtered the tweets using the possible names used to refer to the target clubs  Then, we have annotated the stance information in the tweets for these targets as Favor or Against  Within the course of this study, we have not considered those tweets in which the target is not explicitly mentioned, as our initial filtering process reveals 
For the purposes of the current study, we have not annotated any tweets with the Neither class  This stance class and even finer-grained classes can be considered in further annotation studies  We should also note that in a few tweets, the target of the stance was the management of the club while in some others a particular footballer of the club is praised or criticised  Still, we have considered the club as the target of the stance in all of the cases and carried out our annotations accordingly 
At the end of the annotation process, we have annotated 700 tweets, where 175 tweets are in favor of and 175 tweets are against Target-1, and similarly 175 tweets are in favor of and 175 are against Target-2  Hence, our data set is a balanced one although it is currently limited in size  The corresponding stance annotations are made publicly available at http://ceng metu edu tr/ INLINEFORM0 e120329/ Turkish_Stance_Detection_Tweet_Dataset csv in Comma Separated Values (CSV) format  The file contains three columns with the corresponding headers  The first column is the tweet id of the corresponding tweet, the second column contains the name of the stance target, and the last column includes the stance of the tweet for the target as Favor or Against 
To the best of our knowledge, this is the first publicly-available stance-annotated data set for Turkish  Hence, it is a significant resource as there is a scarcity of annotated data sets, linguistic resources, and NLP tools available for Turkish  Additionally, to the best of our knowledge, it is also significant for being the first stance-annotated data set including sports-related tweets, as previous stance detection data sets mostly include on-line texts on political/ethical issues 
Stance Detection Experiments Using SVM Classifiers
It is emphasized in the related literature that unigram-based methods are reliable for the stance detection task BIBREF2 and similarly unigram-based models have been used as baseline models in studies such as BIBREF0   In order to be used as a baseline and reference system for further studies on stance detection in Turkish tweets, we have trained two SVM classifiers (one for each target) using unigrams as features  Before the extraction of unigrams, we have employed automated preprocessing to filter out the stopwords in our annotated data set of 700 tweets  The stopword list used is the list presented in BIBREF12 which, in turn, is the slightly extended version of the stopword list provided in BIBREF13  
We have used the SVM implementation available in the Weka data mining application BIBREF14 where this particular implementation employs the SMO algorithm BIBREF15 to train a classifier with a linear kernel  The 10-fold cross-validation results of the two classifiers are provided in Table TABREF1 using the metrics of precision, recall, and F-Measure 
The evaluation results are quite favorable for both targets and particularly higher for Target-1, considering the fact that they are the initial experiments on the data set  The performance of the classifiers is better for the Favor class for both targets when compared with the performance results for the Against class  This outcome may be due to the common use of some terms when expressing positive stance towards sports clubs in Turkish tweets  The same percentage of common terms may not have been observed in tweets during the expression of negative stances towards the targets  Yet, completely the opposite pattern is observed in stance detection results of baseline systems given in BIBREF0 , i e , better F-Measure rates have been obtained for the Against class when compared with the Favor class BIBREF0   Some of the baseline systems reported in BIBREF0 are SVM-based systems using unigrams and ngrams as features similar to our study, but their data sets include all three stance classes of Favor, Against, and Neither, while our data set comprises only tweets classified as belonging to Favor or Against classes  Another difference is that the data sets in BIBREF0 have been divided into training and test sets, while in our study we provide 10-fold cross-validation results on the whole data set  On the other hand, we should also note that SVM-based sentiment analysis systems (such as those given in BIBREF16 ) have been reported to achieve better F-Measure rates for the Positive sentiment class when compared with the results obtained for the Negative class  Therefore, our evaluation results for each stance class seem to be in line with such sentiment analysis systems  Yet, further experiments on the extended versions of our data set should be conducted and the results should again be compared with the stance detection results given in the literature 
We have also evaluated SVM classifiers which use only bigrams as features, as ngram-based classifiers have been reported to perform better for the stance detection problem BIBREF0   However, we have observed that using bigrams as the sole features of the SVM classifiers leads to quite poor results  This observation may be due to the relatively limited size of the tweet data set employed  Still, we can conclude that unigram-based features lead to superior results compared to the results obtained using bigrams as features, based on our experiments on our data set  Yet, ngram-based features may be employed on the extended versions of the data set to verify this conclusion within the course of future work 
With an intention to exploit the contribution of hashtag use to stance detection, we have also used the existence of hashtags in tweets as an additional feature to unigrams  The corresponding evaluation results of the SVM classifiers using unigrams together the existence of hashtags as features are provided in Table TABREF2  
When the results given in Table TABREF2 are compared with the results in Table TABREF1 , a slight decrease in F-Measure (0 5%) for Target-1 is observed, while the overall F-Measure value for Target-2 has increased by 1 8%  Although we could not derive sound conclusions mainly due to the relatively small size of our data set, the increase in the performance of the SVM classifier Target-2 is an encouraging evidence for the exploitation of hashtags in a stance detection system  We leave other ways of exploiting hashtags for stance detection as a future work 
To sum up, our evaluation results are significant as reference results to be used for comparison purposes and provides evidence for the utility of unigram-based and hashtag-related features in SVM classifiers for the stance detection problem in Turkish tweets 
Future Prospects
Future work based on the current study includes the following:
Conclusion
Stance detection is a considerably new research area in natural language processing and is considered within the scope of the well-studied topic of sentiment analysis  It is the detection of stance within text towards a target which may be explicitly specified in the text or not  In this study, we present a stance-annotated tweet data set in Turkish where the targets of the annotated stances are two popular sports clubs in Turkey  The corresponding annotations are made publicly-available for research purposes  To the best of our knowledge, this is the first stance detection data set for the Turkish language and also the first sports-related stance-annotated data set  Also presented in this study are SVM classifiers (one for each target) utilizing unigram and bigram features in addition to using the existence of hashtags as another feature  10-fold cross validation results of these classifiers are presented which can be used as reference results by prospective systems  Both the annotated data set and the classifiers with evaluations are significant since they are the initial contributions to stance detection problem in Turkish tweets 
Context 2: Introduction
Word Sense Disambiguation (WSD) is a fundamental task and long-standing challenge in Natural Language Processing (NLP), which aims to find the exact sense of an ambiguous word in a particular context BIBREF0  Previous WSD approaches can be grouped into two main categories: knowledge-based and supervised methods 
Knowledge-based WSD methods rely on lexical resources like WordNet BIBREF1 and usually exploit two kinds of lexical knowledge  The gloss, which defines a word sense meaning, is first utilized in Lesk algorithm BIBREF2 and then widely taken into account in many other approaches BIBREF3, BIBREF4  Besides, structural properties of semantic graphs are mainly used in graph-based algorithms BIBREF5, BIBREF6 
Traditional supervised WSD methods BIBREF7, BIBREF8, BIBREF9 focus on extracting manually designed features and then train a dedicated classifier (word expert) for every target lemma 
Although word expert supervised WSD methods perform better, they are less flexible than knowledge-based methods in the all-words WSD task BIBREF10  Recent neural-based methods are devoted to dealing with this problem  BIBREF11 present a supervised classifier based on Bi-LSTM, which shares parameters among all word types except the last layer  BIBREF10 convert WSD task to a sequence labeling task, thus building a unified model for all polysemous words  However, neither of them can totally beat the best word expert supervised methods 
More recently, BIBREF12 propose to leverage the gloss information from WordNet and model the semantic relationship between the context and gloss in an improved memory network  Similarly, BIBREF13 introduce a (hierarchical) co-attention mechanism to generate co-dependent representations for the context and gloss  Their attempts prove that incorporating gloss knowledge into supervised WSD approach is helpful, but they still have not achieved much improvement, because they may not make full use of gloss knowledge 
In this paper, we focus on how to better leverage gloss information in a supervised neural WSD system  Recently, the pre-trained language models, such as ELMo BIBREF14 and BERT BIBREF15, have shown their effectiveness to alleviate the effort of feature engineering  Especially, BERT has achieved excellent results in question answering (QA) and natural language inference (NLI)  We construct context-gloss pairs from glosses of all possible senses (in WordNet) of the target word, thus treating WSD task as a sentence-pair classification problem  We fine-tune the pre-trained BERT model and achieve new state-of-the-art results on WSD task  In particular, our contribution is two-fold:
1  We construct context-gloss pairs and propose three BERT-based models for WSD 
2  We fine-tune the pre-trained BERT model, and the experimental results on several English all-words WSD benchmark datasets show that our approach significantly outperforms the state-of-the-art systems 
Methodology
In this section, we describe our method in detail 
Methodology ::: Task Definition
In WSD, a sentence $s$ usually consists of a series of words: $\lbrace w_1,\cdots ,w_m\rbrace $, and some of the words $\lbrace w_{i_1},\cdots ,w_{i_k}\rbrace $ are targets $\lbrace t_1,\cdots ,t_k\rbrace $ need to be disambiguated  For each target $t$, its candidate senses $\lbrace c_1,\cdots ,c_n\rbrace $ come from entries of its lemma in a pre-defined sense inventory (usually WordNet)  Therefore, WSD task aims to find the most suitable entry (symbolized as unique sense key) for each target in a sentence  See a sentence example in Table TABREF1 
Methodology ::: BERT
BERT BIBREF15 is a new language representation model, and its architecture is a multi-layer bidirectional Transformer encoder  BERT model is pre-trained on a large corpus and two novel unsupervised prediction tasks, i e , masked language model and next sentence prediction tasks are used in pre-training  When incorporating BERT into downstream tasks, the fine-tuning procedure is recommended  We fine-tune the pre-trained BERT model on WSD task 
Methodology ::: BERT ::: BERT(Token-CLS)
Since every target in a sentence needs to be disambiguated to find its exact sense, WSD task can be regarded as a token-level classification task  To incorporate BERT to WSD task, we take the final hidden state of the token corresponding to the target word (if more than one token, we average them) and add a classification layer for every target lemma, which is the same as the last layer of the Bi-LSTM model BIBREF11 
Methodology ::: GlossBERT
BERT can explicitly model the relationship of a pair of texts, which has shown to be beneficial to many pair-wise natural language understanding tasks  In order to fully leverage gloss information, we propose GlossBERT to construct context-gloss pairs from all possible senses of the target word in WordNet, thus treating WSD task as a sentence-pair classification problem 
We describe our construction method with an example (See Table TABREF1)  There are four targets in this sentence, and here we take target word research as an example:
Methodology ::: GlossBERT ::: Context-Gloss Pairs
The sentence containing target words is denoted as context sentence  For each target word, we extract glosses of all $N$ possible senses (here $N=4$) of the target word (research) in WordNet to obtain the gloss sentence  [CLS] and [SEP] marks are added to the context-gloss pairs to make it suitable for the input of BERT model  A similar idea is also used in aspect-based sentiment analysis BIBREF16 
Methodology ::: GlossBERT ::: Context-Gloss Pairs with Weak Supervision
Based on the previous construction method, we add weak supervised signals to the context-gloss pairs (see the highlighted part in Table TABREF1)  The signal in the gloss sentence aims to point out the target word, and the signal in the context sentence aims to emphasize the target word considering the situation that a target word may occur more than one time in the same sentence 
Therefore, each target word has $N$ context-gloss pair training instances ($label\in \lbrace yes, no\rbrace $)  When testing, we output the probability of $label=yes$ of each context-gloss pair and choose the sense corresponding to the highest probability as the prediction label of the target word  We experiment with three GlossBERT models:
Methodology ::: GlossBERT ::: GlossBERT(Token-CLS)
We use context-gloss pairs as input  We highlight the target word by taking the final hidden state of the token corresponding to the target word (if more than one token, we average them) and add a classification layer ($label\in \lbrace yes, no\rbrace $) 
Methodology ::: GlossBERT ::: GlossBERT(Sent-CLS)
We use context-gloss pairs as input  We take the final hidden state of the first token [CLS] as the representation of the whole sequence and add a classification layer ($label\in \lbrace yes, no\rbrace $), which does not highlight the target word 
Methodology ::: GlossBERT ::: GlossBERT(Sent-CLS-WS)
We use context-gloss pairs with weak supervision as input  We take the final hidden state of the first token [CLS] and add a classification layer ($label\in \lbrace yes, no\rbrace $), which weekly highlight the target word by the weak supervision 
Experiments ::: Datasets
The statistics of the WSD datasets are shown in Table TABREF12 
Experiments ::: Datasets ::: Training Dataset
Following previous work BIBREF13, BIBREF12, BIBREF10, BIBREF17, BIBREF9, BIBREF7, we choose SemCor3 0 as training corpus, which is the largest corpus manually annotated with WordNet sense for WSD 
Experiments ::: Datasets ::: Evaluation Datasets
We evaluate our method on several English all-words WSD datasets  For a fair comparison, we use the benchmark datasets proposed by BIBREF17 which include five standard all-words fine-grained WSD datasets from the Senseval and SemEval competitions: Senseval-2 (SE2), Senseval-3 (SE3), SemEval-2007 (SE07), SemEval-2013 (SE13) and SemEval-2015 (SE15)  Following BIBREF13, BIBREF12 and BIBREF10, we choose SE07, the smallest among these test sets, as the development set 
Experiments ::: Datasets ::: WordNet
Since BIBREF17 map all the sense annotations in these datasets from their original versions to WordNet 3 0, we extract word sense glosses from WordNet 3 0 
Experiments ::: Settings
We use the pre-trained uncased BERT$_\mathrm {BASE}$ model for fine-tuning, because we find that BERT$_\mathrm {LARGE}$ model performs slightly worse than BERT$_\mathrm {BASE}$ in this task  The number of Transformer blocks is 12, the number of the hidden layer is 768, the number of self-attention heads is 12, and the total number of parameters of the pre-trained model is 110M  When fine-tuning, we use the development set (SE07) to find the optimal settings for our experiments  We keep the dropout probability at 0 1, set the number of epochs to 4  The initial learning rate is 2e-5, and the batch size is 64 
Experiments ::: Results
Table TABREF19 shows the performance of our method on the English all-words WSD benchmark datasets  We compare our approach with previous methods 
The first block shows the MFS baseline, which selects the most frequent sense in the training corpus for each target word 
The second block shows two knowledge-based systems  Lesk$_{ext+emb}$ BIBREF4 is a variant of Lesk algorithm BIBREF2 by calculating the gloss-context overlap of the target word  Babelfy BIBREF6 is a unified graph-based approach which exploits the semantic network structure from BabelNet 
The third block shows two word expert traditional supervised systems  IMS BIBREF7 is a flexible framework which trains SVM classifiers and uses local features  And IMS$_{+emb}$ BIBREF9 is the best configuration of the IMS framework, which also integrates word embeddings as features 
The fourth block shows several recent neural-based methods  Bi-LSTM BIBREF11 is a baseline for neural models  Bi-LSTM$_{+ att  + LEX + POS}$ BIBREF10 is a multi-task learning framework for WSD, POS tagging, and LEX with self-attention mechanism, which converts WSD to a sequence learning task  GAS$_{ext}$ BIBREF12 is a variant of GAS which is a gloss-augmented variant of the memory network by extending gloss knowledge  CAN$^s$ and HCAN BIBREF13 are sentence-level and hierarchical co-attention neural network models which leverage gloss knowledge 
In the last block, we report the performance of our method  BERT(Token-CLS) is our baseline, which does not incorporate gloss information, and it performs slightly worse than previous traditional supervised methods and recent neural-based methods  It proves that directly using BERT cannot obtain performance growth  The other three methods outperform other models by a substantial margin, which proves that the improvements come from leveraging BERT to better exploit gloss information  It is worth noting that our method achieves significant improvements in SE07 and Verb over previous methods, which have the highest ambiguity level among all datasets and all POS tags respectively according to BIBREF17 
Moreover, GlossBERT(Token-CLS) performs better than GlossBERT(Sent-CLS), which proves that highlighting the target word in the sentence is important  However, the weakly highlighting method GlossBERT(Sent-CLS-WS) performs best in most circumstances, which may result from its combination of the advantages of the other two methods 
Experiments ::: Discussion
There are two main reasons for the great improvements of our experimental results  First, we construct context-gloss pairs and convert WSD problem to a sentence-pair classification task which is similar to NLI tasks and train only one classifier, which is equivalent to expanding the corpus  Second, we leverage BERT BIBREF15 to better exploit the gloss information  BERT model shows its advantage in dealing with sentence-pair classification tasks by its amazing improvement on QA and NLI tasks  This advantage comes from both of its two novel unsupervised prediction tasks 
Compared with traditional word expert supervised methods, our GlossBERT shows its effectiveness to alleviate the effort of feature engineering and does not require training a dedicated classifier for every target lemma  Up to now, it can be said that the neural network method can totally beat the traditional word expert method  Compared with recent neural-based methods, our solution is more intuitive and can make better use of gloss knowledge  Besides, our approach demonstrates that when we fine-tune BERT on a downstream task, converting it into a sentence-pair classification task may be a good choice 
Conclusion
In this paper, we seek to better leverage gloss knowledge in a supervised neural WSD system  We propose a new solution to WSD by constructing context-gloss pairs and then converting WSD to a sentence-pair classification task  We fine-tune the pre-trained BERT model and achieve new state-of-the-art results on WSD task 
Acknowledgments
We would like to thank the anonymous reviewers for their valuable comments  The research work is supported by National Natural Science Foundation of China (No  61751201 and 61672162), Shanghai Municipal Science and Technology Commission (16JC1420401 and 17JC1404100), Shanghai Municipal Science and Technology Major Project (No 2018SHZDZX01) and ZJLab 
Context 3: Introduction
The cognitive processes involved in human language comprehension are complex and only partially identified  According to the dual-stream model of speech comprehension BIBREF1 , sound waves are first converted to phoneme-like features and further processed by a ventral stream that maps those features onto words and semantic structures, and a dorsal stream that (among other things) supports audio-short term memory  The mapping of words onto meaning is thought to be subserved by widely distributed regions of the brain that specialize in particular modalities — for example visual aspects of the word banana reside in the occipital lobe of the brain and are activated when the word banana is heard BIBREF2 — and the different representation modalities are thought to be integrated into a single coherent latent representation in the anterior temporal lobe BIBREF3   While this part of meaning representation in human language comprehension is somewhat understood, much less is known about how the meanings of words are integrated together to form the meaning of sentences and discourses  One tool researchers use to study the integration of meaning across words is electroencephelography (EEG), which measures the electrical activity of large numbers of neurons acting in concert  EEG has the temporal resolution necessary to study the processes involved in meaning integration, and certain stereotyped electrical responses to word presentations, known as event-related potentials (ERPs), have been identified with some of the processes thought to contribute to comprehension 
In this work, we consider six ERP components that have been associated in the cognitive neuroscience and psycholinguistics literature with language processing and which we analyze in the data from BIBREF0 (see Figure FIGREF1 for spatial and temporal definitions of these ERP components)  Three of these — the N400, EPNP, and PNP responses — are primarily considered markers for semantic processing, while the other three — the P600, ELAN, and LAN responses — are primarily considered markers for syntactic processing  However, the neat division of the ERP responses into either semantic or syntactic categories is controversial  The N400 response has been very well studied (for an overview see BIBREF4 ) and it is well established that it is associated with semantic complexity, but the features of language that trigger the other ERP responses we consider here are poorly understood  We propose to use a neural network pretrained as a language model to probe what features of language drive these ERP responses, and in turn to probe what features of language mediate the cognitive processes that underlie human language comprehension, and especially the integration of meaning across words 
Background
While a full discussion of each ERP component and the features of language thought to trigger each are beyond the scope of this document (for reviews see e g  BIBREF0 , BIBREF2 , BIBREF4 , BIBREF5 , and BIBREF6 ), we introduce some basic features of ERP components to help in the discussion later  ERP components are electrical potential responses measured with respect to a baseline that are triggered by an event (in our case the presentation of a new word to a participant in an experiment)  The name of each ERP component reflects whether the potential is positive or negative relative to the baseline  The N400 is so-named because it is Negative relative to a baseline (the baseline is typically recorded just before a word is presented at an electrode that is not affected by the ERP response) and because it peaks in magnitude at about 400ms after a word is presented to a participant in an experiment  The P600 is Positive relative to a baseline and peaks around 600ms after a word is presented to a participant (though its overall duration is much longer and less specific in time than the N400)  The post-N400 positivity is so-named because it is part of a biphasic response; it is a positivity that occurs after the negativity associated with the N400  The early post-N400 positivity (EPNP) is also part of a biphasic response, but the positivity has an eariler onset than the standard PNP  Finally, the LAN and ELAN are the left-anterior negativity and early left-anterior negativity respectively  These are named for their timing, spatial distribution on the scalp, and direction of difference from the baseline  It is important to note that ERP components can potentially cancel and mask each other, and that it is difficult to precisely localize the neural activity that causes the changes in electrical potential at the electrodes where those changes are measured 
Related Work
This work is most closely related to the paper from which we get the ERP data: BIBREF0 

ground_truth: Galatasaray, Fenerbahçe

Galatasaray , Fenerbahçe 

prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. Do they use large or small BERT?
Context 3:  In that work, the authors relate the surprisal of a word, i e  the (negative log) probability of the word appearing in its context, to each of the ERP signals we consider here  The authors do not directly train a model to predict ERPs  Instead, models of the probability distribution of each word in context are used to compute a surprisal for each word, which is input into a mixed effects regression along with word frequency, word length, word position in the sentence, and sentence position in the experiment  The effect of the surprisal is assessed using a likelihood-ratio test  In BIBREF7 , the authors take an approach similar to BIBREF0   The authors compare the explanatory power of surprisal (as computed by an LSTM or a Recurrent Neural Network Grammar (RNNG) language model) to a measure of syntactic complexity they call “distance" that counts the number of parser actions in the RNNG language model  The authors find that surprisal (as predicted by the RNNG) and distance are both significant factors in a mixed effects regression which predicts the P600, while the surprisal as computed by an LSTM is not  Unlike BIBREF0 and BIBREF7 , we do not use a linking function (e g  surprisal) to relate a language model to ERPs  We thus lose the interpretability provided by the linking function, but we are able to predict a significant proportion of the variance for all of the ERP components, where prior work could not  We interpret our results through characterization of the ERPs in terms of how they relate to each other and to eye-tracking data rather than through a linking function  The authors in BIBREF8 also use a recurrent neural network to predict neural activity directly  In that work the authors predict magnetoencephalography (MEG) activity, a close cousin to EEG, recorded while participants read a chapter of Harry Potter and the Sorcerer’s Stone BIBREF9   Their approach to characterization of processing at each MEG sensor location is to determine whether it is best predicted by the context vector of the recurrent network (prior to the current word being processed), the embedding of the current word, or the probability of the current word given the context  In future work we also intend to add these types of studies to the ERP predictions 
Discussion
In this work we find that all six of the ERP components from BIBREF0 can be predicted above chance by a model which has been pretrained using a language modeling objective and then directly trained to predict the components  This is in contrast to prior work which has successfully linked language models to the N400 BIBREF0 and P600 BIBREF7 but not the other ERP components  We also note that contrary to BIBREF7 , we find that an LSTM does contain information that can be used to predict EEG data, and in particular that it can predict the P600  We speculate that the analysis used in BIBREF7 did not find reliable effects because the language models were related to the EEG data through functions chosen a priori (the surprisal, and the `distance' metric)  These functions, though interpretable, might be interpretable at the cost of losing much of the information in the representations learned by the network 
In addition, we show through our multitask learning analysis that information is shared between ERP components, and between ERP components and behavioral data

ground_truth: small BERT

small BERT

prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. What datasets are used?
Context 3:  Although these relationships must be viewed with caution until they can be verified across multiple datasets and with more variation in neural network architectures, here we consider some potential reasons for our findings  The broad point we wish to make is that by better understanding which ERP components share information with each other and with behavioral data through the type of analysis we present here (multitask learning) or other means, we can better understand what drives each ERP component and in turn the processes involved in human language comprehension 
Conclusion
We have shown that ERP components can be predicted from neural networks pretrained as language models and fine-tuned to directly predict those components  To the best of our knowledge, prior work has not successfully used statistical models to predict all of these components  Furthermore, we have shown that multitask learning benefits the prediction of ERP components and can suggest how components relate to each other  At present, these joint-training benefit relationships are only suggestive, but if these relationships ultimately lead to insights about what drives each ERP component, then the components become more useful tools for studying human language comprehension  By using multitask learning as a method of characterization, we have found some expected relationships (LAN+P600 and ELAN+P600) and several more surprising relationships  We believe that this is exactly the kind of finding that makes multitask learning an interesting exploratory technique in this area  Additionally, we have shown that information can be shared between heterogeneous types of data (eye-tracking, self-paced reading, and ERP components) in the domain of human language processing prediction, and in particular between behavioral and neural data  Given the small datasets associated with human language processing, using heterogeneous data is a potentially major advantage of a multitask approach  In future work, we will further explore what information is encoded into the model representations when neural and behavioral data are used to train neural networks, and how these representations differ from the representations in a model trained on language alone 
Acknowledgments
We thank our reviewers for their valuable feedback  This work is supported in part by National Institutes of Health grant number U01NS098969 
Appendix
Here we present a visualization (Figure FIGREF21 ) of the results presented in Table TABREF9 of the main paper, and a visualization (Figure FIGREF22 ) of a more complete set of results from which the information in Table TABREF16 of the main paper is drawn  We also show supplemental results for variants of our primary analysis on multitask learning with eye-tracking, self-paced reading time and ERP data  In the variants we modify the input representation to our decoder network to see whether the relationships between the behavioral data and neural activity appear to be consistent with different choices of encoder architectures  Additional (and more varied) choices or architectures are left to future work  The results in Table TABREF23 reflect using only the forward-encoder (rather than the bi-LSTM) in the encoder network, while the results in Table TABREF24 reflect using only the word embeddings (i e  bypassing the LSTM entirely)  While the results are clearly worse for each of these choices of architecture than for using a bi-LSTM encoder, the relationships between the behavioral data and the ERP signals is qualitatively similar  Finally, TABREF25 shows the Pearson correlation coefficient between different measures  We note that the patterns of correlation are different than the patterns of which measures benefit from joint training with each other 

ground_truth: Answer with content missing: (Whole Method and Results sections) The primary dataset we use is the ERP data collected and computed by Frank et al. (2015), and we also use behavioral data (eye-tracking data and self-paced reading times) from Frank et al. (2013) which were collected on the same set of 205 sentences.
Select:
- ERP data collected and computed by Frank et al. (2015)
- behavioral data (eye-tracking data and self-paced reading times) from Frank et al. (2013)

the ERP data: BIBREF0