prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. Which toolkits do they use?
Context 1: Introduction
As social media, specially Twitter, takes on an influential role in presidential elections in the U S , natural language processing of political tweets BIBREF0 has the potential to help with nowcasting and forecasting of election results as well as identifying the main issues with a candidate – tasks of much interest to journalists, political scientists, and campaign organizers BIBREF1  As a methodology to obtain training data for a machine learning system that analyzes political tweets, BIBREF2 devised a crowdsourcing scheme with variable crowdworker numbers based on the difficulty of the annotation task  They provided a dataset of tweets where the sentiments towards political candidates were labeled both by experts in political communication and by crowdworkers who were likely not domain experts  BIBREF2 revealed that crowdworkers can match expert performance relatively accurately and in a budget-efficient manner  Given this result, the authors envisioned future work in which groundtruth labels would be crowdsourced for a large number of tweets and then used to design an automated NLP tool for political tweet analysis 
The question we address here is: How accurate are existing NLP tools for political tweet analysis? These tools would provide a baseline performance that any new machine learning system for political tweet analysis would compete against  We here explore whether existing NLP systems can answer the questions "What sentiment?" and "Towards whom?" accurately for the dataset of political tweets provided by BIBREF2  In our analysis, we include NLP tools with publicly-available APIs, even if the tools were not specifically designed for short texts like tweets, and, in particular, political tweets 
Our experiments reveal that the task of entity-level sentiment analysis is difficult for existing tools to answer accurately while the recognition of the entity, here, which politician, was easier 
NLP Toolkits
NLP toolkits typically have the following capabilities: tokenization, part-of-speech (PoS) tagging, chunking, named entity recognition and sentiment analysis  In a study by BIBREF3, it is shown that the well-known NLP toolkits NLTK BIBREF4, Stanford CoreNLP BIBREF5, and TwitterNLP BIBREF6 have tokenization, PoS tagging and NER modules in their pipelines  There are two main approaches for NER: (1) rule-based and (2) statistical or machine learning based  The most ubiquitous algorithms for sequence tagging use Hidden Markov Models BIBREF7, Maximum Entropy Markov Models BIBREF7, BIBREF8, or Conditional Random Fields BIBREF9  Recent works BIBREF10, BIBREF11 have used recurrent neural networks with attention modules for NER 
Sentiment detection tools like SentiStrength BIBREF12 and TensiStrength BIBREF13 are rule-based tools, relying on various dictionaries of emoticons, slangs, idioms, and ironic phrases, and set of rules that can detect the sentiment of a sentence overall or a targeted sentiment  Given a list of keywords, TensiStrength (similar to SentiStrength) reports the sentiment towards selected entities in a sentence, based on five levels of relaxation and five levels of stress 
Among commercial NLP toolkits (e g , BIBREF14, BIBREF15, BIBREF16), we selected BIBREF17 and BIBREF18 for our experiments, which, to the best of our knowledge, are the only publicly accessible commercial APIs for the task of entity-level sentiment analysis that is agnostic to the text domain  We also report results of TensiStrength BIBREF13, TwitterNLP BIBREF6, BIBREF19, CogComp-NLP BIBREF20, and Stanford NLP NER BIBREF21 
Dataset and Analysis Methodology
We used the 1,000-tweet dataset by BIBREF2 that contains the named-entities labels and entity-level sentiments for each of the four 2016 presidential primary candidates Bernie Sanders, Donald Trump, Hillary Clinton, and Ted Cruz, provided by crowdworkers, and by experts in political communication, whose labels are considered groundtruth  The crowdworkers were located in the US and hired on the BIBREF22 platform  For the task of entity-level sentiment analysis, a 3-scale rating of "negative," "neutral," and "positive" was used by the annotators 
BIBREF2 proposed a decision tree approach for computing the number of crowdworkers who should analyze a tweet based on the difficulty of the task  Tweets are labeled by 2, 3, 5, or 7 workers based on the difficulty of the task and the level of disagreement between the crowdworkers  The model computes the number of workers based on how long a tweet is, the presence of a link in a tweet, and the number of present sarcasm signals  Sarcasm is often used in political tweets and causes disagreement between the crowdworkers  The tweets that are deemed to be sarcastic by the decision tree model, are expected to be more difficult to annotate, and hence are allocated more crowdworkers to work on 
We conducted two sets of experiments  In the first set, we used BIBREF23, BIBREF17, and BIBREF18, for entity-level sentiment analysis; in the second set, BIBREF17, BIBREF19, BIBREF24, BIBREF25, and BIBREF26, BIBREF18 for named-entity recognition 
In the experiments that we conducted with TwitterNLP for named-entity recognition, we worked with the default values of the model  Furthermore, we selected the 3-class Stanford NER model, which uses the classes “person,” “organization,” and “location” because it resulted in higher accuracy compared to the 7-class model  For CogComp-NLP NER we used Ontonotes 5 0 NER model BIBREF27  For spaCy NER we used the `en_core_web_lg' model 
We report the experimental results for our two tasks in terms of the correct classification rate (CCR)  For sentiment analysis, we have a three-class problem (positive, negative, and neutral), where the classes are mutually exclusive  The CCR, averaged for a set of tweets, is defined to be the number of correctly-predicted sentiments over the number of groundtruth sentiments in these tweets  For NER, we consider that each tweet may reference up to four candidates, i e , targeted entities  The CCR, averaged for a set of tweets, is the number of correctly predicted entities (candidates) over the number of groundtruth entities (candidates) in this set 
Results and Discussion
The dataset of 1,000 randomly selected tweets contains more than twice as many tweets about Trump than about the other candidates  In the named-entity recognition experiment, the average CCR of crowdworkers was 98 6%, while the CCR of the automated systems ranged from 77 2% to 96 7%  For four of the automated systems, detecting the entity Trump was more difficult than the other entities (e g , spaCy 72 7% for the entity Trump vs  above 91% for the other entities)  An example of incorrect NER is shown in Figure FIGREF1 top  The difficulties the automated tools had in NER may be explained by the fact that the tools were not trained on tweets, except for TwitterNLP, which was not in active development when the data was created BIBREF1 
In the sentiment analysis experiments, we found that a tweet may contain multiple sentiments  The groundtruth labels contain 210 positive sentiments, 521 neutral sentiments, and 305 negative sentiments to the candidates  We measured the CCR, across all tweets, to be 31 7% for Rosette Text Analytics, 43 2% for Google Cloud, 44 2% for TensiStrength, and 74 7% for the crowdworkers  This means the difference between the performance of the tools and the crowdworkers is significant – more than 30 percent points 
Crowdworkers correctly identified 62% of the neutral, 85% of the positive, and 92% of the negative sentiments  Google Cloud correctly identified 88% of the neutral sentiments, but only 3% of the positive, and 19% of the negative sentiments  TensiStrength correctly identified 87 2% of the neutral sentiments, but 10 5% of the positive, and 8 1% of the negative sentiments  Rosette Text Analytics correctly identified 22 7% of neutral sentiments, 38 1% of negative sentiments and 40 9% of positive sentiments  The lowest and highest CCR pertains to tweets about Trump and Sanders for both Google Cloud and TensiStrength, Trump and Clinton for Rosette Text Analytics, and Clinton and Cruz for crowdworkers  An example of incorrect ELS analysis is shown in Figure FIGREF1 bottom 
Conclusions and Future Work
Our results show that existing NLP systems cannot accurately perform sentiment analysis of political tweets in the dataset we experimented with  Labeling by humans, even non-expert crowdworkers, yields accuracy results that are well above the results of existing automated NLP systems  In future work we will therefore use a crowdworker-labeled dataset to train a new machine-learning based NLP system for tweet analysis  We will ensure that the training data is balanced among classes  Our plan is to use state-of-the-art deep neural networks and compare their performance for entity-level sentiment analysis of political tweets 
Acknowledgments
Partial support of this work by the Hariri Institute for Computing and Computational Science & Engineering at Boston University (to L G ) and a Google Faculty Research Award (to M B  and L G ) is gratefully acknowledged  Additionally, we would like to thank Daniel Khashabi for his help in running the CogComp-NLP Python API and Mike Thelwal for his help with TensiStrength  We are also grateful to the Stanford NLP group for clarifying some of the questions we had with regards to the Stanford NER tool 
Context 2: Introduction
Neural networks have been successfully used to describe images with text using sequence-to-sequence models BIBREF0  However, the results are simple and dry captions which are one or two phrases long  Humans looking at a painting see more than just objects  Paintings stimulate sentiments, metaphors and stories as well  Therefore, our goal is to have a neural network describe the painting artistically in a style of choice  As a proof of concept, we present a model which generates Shakespearean prose for a given painting as shown in Figure FIGREF1  Accomplishing this task is difficult with traditional sequence to sequence models since there does not exist a large collection of Shakespearean prose which describes paintings: Shakespeare's works describes a single painting shown in Figure FIGREF3  Fortunately we have a dataset of modern English poems which describe images BIBREF1 and line-by-line modern paraphrases of Shakespeare's plays BIBREF2  Our solution is therefore to combine two separately trained models to synthesize Shakespearean prose for a given painting 
Introduction ::: Related work
A general end-to-end approach to sequence learning BIBREF3 makes minimal assumptions on the sequence structure  This model is widely used in tasks such as machine translation, text summarization, conversational modeling, and image captioning  A generative model using a deep recurrent architecture BIBREF0 has also beeen used for generating phrases describing an image  The task of synthesizing multiple lines of poetry for a given image BIBREF1 is accomplished by extracting poetic clues from images  Given the context image, the network associates image attributes with poetic descriptions using a convolutional neural net  The poem is generated using a recurrent neural net which is trained using multi-adversarial training via policy gradient 
Transforming text from modern English to Shakespearean English using text "style transfer" is challenging  An end to end approach using a sequence-to-sequence model over a parallel text corpus BIBREF2 has been proposed based on machine translation  In the absence of a parallel text corpus, generative adversarial networks (GANs) have been used, which simultaneously train two models: a generative model which captures the data distribution, and a discriminative model which evaluates the performance of the generator  Using a target domain language model as a discriminator has also been employed BIBREF4, providing richer and more stable token-level feedback during the learning process  A key challenge in both image and text style transfer is separating content from style BIBREF5, BIBREF6, BIBREF7  Cross-aligned auto-encoder models have focused on style transfer using non-parallel text BIBREF7  Recently, a fine grained model for text style transfer has been proposed BIBREF8 which controls several factors of variation in textual data by using back-translation  This allows control over multiple attributes, such as gender and sentiment, and fine-grained control over the trade-off between content and style 
Methods
We use a total three datasets: two datasets for generating an English poem from an image, and Shakespeare plays and their English translations for text style transfer 
We train a model for generating poems from images based on two datasets BIBREF1  The first dataset consists of image and poem pairs, namely a multi-modal poem dataset (MultiM-Poem), and the second dataset is a large poem corpus, namely a uni-modal poem dataset (UniM-Poem)  The image and poem pairs are extended by adding the nearest three neighbor poems from the poem corpus without redundancy, and an extended image and poem pair dataset is constructed and denoted as MultiM-Poem(Ex)BIBREF1 
We use a collection of line-by-line modern paraphrases for 16 of Shakespeare’s plays BIBREF2, for training a style transfer network from English poems to Shakespearean prose  We use 18,395 sentences from the training data split  We keep 1,218 sentences in the validation data set and 1,462 sentences in our test set 
Methods ::: Image To Poem Actor-Critic Model
For generating a poem from images we use an existing actor-critic architecture BIBREF1  This involves 3 parallel CNNs: an object CNN, sentiment CNN, and scene CNN, for feature extraction  These features are combined with a skip-thought model which provides poetic clues, which are then fed into a sequence-to-sequence model trained by policy gradient with 2 discriminator networks for rewards  This as a whole forms a pipeline that takes in an image and outputs a poem as shown on the top left of Figure FIGREF4  A CNN-RNN generative model acts as an agent  The parameters of this agent define a policy whose execution determines which word is selected as an action  When the agent selects all words in a poem, it receives a reward  Two discriminative networks, shown on the top right of Figure FIGREF4, are defined to serve as rewards concerning whether the generated poem properly describes the input image and whether the generated poem is poetic  The goal of the poem generation model is to generate a sequence of words as a poem for an image to maximize the expected return 
Methods ::: Shakespearizing Poetic Captions
For Shakespearizing modern English texts, we experimented with various types of sequence to sequence models  Since the size of the parallel translation data available is small, we leverage a dictionary providing a mapping between Shakespearean words and modern English words to retrofit pre-trained word embeddings
Context 3: Introduction
Word Sense Disambiguation (WSD) is a fundamental task and long-standing challenge in Natural Language Processing (NLP), which aims to find the exact sense of an ambiguous word in a particular context BIBREF0  Previous WSD approaches can be grouped into two main categories: knowledge-based and supervised methods 
Knowledge-based WSD methods rely on lexical resources like WordNet BIBREF1 and usually exploit two kinds of lexical knowledge  The gloss, which defines a word sense meaning, is first utilized in Lesk algorithm BIBREF2 and then widely taken into account in many other approaches BIBREF3, BIBREF4  Besides, structural properties of semantic graphs are mainly used in graph-based algorithms BIBREF5, BIBREF6 
Traditional supervised WSD methods BIBREF7, BIBREF8, BIBREF9 focus on extracting manually designed features and then train a dedicated classifier (word expert) for every target lemma 
Although word expert supervised WSD methods perform better, they are less flexible than knowledge-based methods in the all-words WSD task BIBREF10  Recent neural-based methods are devoted to dealing with this problem  BIBREF11 present a supervised classifier based on Bi-LSTM, which shares parameters among all word types except the last layer  BIBREF10 convert WSD task to a sequence labeling task, thus building a unified model for all polysemous words  However, neither of them can totally beat the best word expert supervised methods 
More recently, BIBREF12 propose to leverage the gloss information from WordNet and model the semantic relationship between the context and gloss in an improved memory network  Similarly, BIBREF13 introduce a (hierarchical) co-attention mechanism to generate co-dependent representations for the context and gloss  Their attempts prove that incorporating gloss knowledge into supervised WSD approach is helpful, but they still have not achieved much improvement, because they may not make full use of gloss knowledge 
In this paper, we focus on how to better leverage gloss information in a supervised neural WSD system  Recently, the pre-trained language models, such as ELMo BIBREF14 and BERT BIBREF15, have shown their effectiveness to alleviate the effort of feature engineering  Especially, BERT has achieved excellent results in question answering (QA) and natural language inference (NLI)  We construct context-gloss pairs from glosses of all possible senses (in WordNet) of the target word, thus treating WSD task as a sentence-pair classification problem  We fine-tune the pre-trained BERT model and achieve new state-of-the-art results on WSD task  In particular, our contribution is two-fold:
1  We construct context-gloss pairs and propose three BERT-based models for WSD 
2  We fine-tune the pre-trained BERT model, and the experimental results on several English all-words WSD benchmark datasets show that our approach significantly outperforms the state-of-the-art systems 
Methodology
In this section, we describe our method in detail 
Methodology ::: Task Definition
In WSD, a sentence $s$ usually consists of a series of words: $\lbrace w_1,\cdots ,w_m\rbrace $, and some of the words $\lbrace w_{i_1},\cdots ,w_{i_k}\rbrace $ are targets $\lbrace t_1,\cdots ,t_k\rbrace $ need to be disambiguated  For each target $t$, its candidate senses $\lbrace c_1,\cdots ,c_n\rbrace $ come from entries of its lemma in a pre-defined sense inventory (usually WordNet)  Therefore, WSD task aims to find the most suitable entry (symbolized as unique sense key) for each target in a sentence  See a sentence example in Table TABREF1 
Methodology ::: BERT
BERT BIBREF15 is a new language representation model, and its architecture is a multi-layer bidirectional Transformer encoder  BERT model is pre-trained on a large corpus and two novel unsupervised prediction tasks, i e , masked language model and next sentence prediction tasks are used in pre-training  When incorporating BERT into downstream tasks, the fine-tuning procedure is recommended  We fine-tune the pre-trained BERT model on WSD task 
Methodology ::: BERT ::: BERT(Token-CLS)
Since every target in a sentence needs to be disambiguated to find its exact sense, WSD task can be regarded as a token-level classification task

ground_truth: BIBREF17, BIBREF18, TensiStrength BIBREF13, TwitterNLP BIBREF6, BIBREF19, CogComp-NLP BIBREF20, Stanford NLP NER BIBREF21

BIBREF23, BIBREF17, BIBREF18, BIBREF19, BIBREF24, BIBREF25, BIBREF26

prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. What models are used for painting embedding and what for language style transfer?
Context 2:  Incorporating this extra information improves the translation task  The large number of shared word types between the source and target sentences indicates that sharing the representation between them is beneficial 
Methods ::: Shakespearizing Poetic Captions ::: Seq2Seq with Attention
We use a sequence-to-sequence model which consists of a single layer unidrectional LSTM encoder and a single layer LSTM decoder and pre-trained retrofitted word embeddings shared between source and target sentences  We experimented with two different types of attention: global attention BIBREF9, in which the model makes use of the output from the encoder and decoder for the current time step only, and Bahdanau attention BIBREF10, where computing attention requires the output of the decoder from the prior time step  We found that global attention performs better in practice for our task of text style transfer 
Methods ::: Shakespearizing Poetic Captions ::: Seq2Seq with a Pointer Network
Since a pair of corresponding Shakespeare and modern English sentences have significant vocabulary overlap we extend the sequence-to-sequence model mentioned above using pointer networks BIBREF11 that provide location based attention and have been used to enable copying of tokens directly from the input  Moreover, there are lot of proper nouns and rare words which might not be predicted by a vanilla sequence to sequence model 
Methods ::: Shakespearizing Poetic Captions ::: Prediction
For both seq2seq models, we use the attention matrices returned at each decoder time step during inference, to compute the next word in the translated sequence if the decoder output at the current time step is the UNK token  We replace the UNKs in the target output with the highest aligned, maximum attention, source word  The seq2seq model with global attention gives the best results with an average target BLEU score of 29 65 on the style transfer dataset, compared with an average target BLEU score of 26 97 using the seq2seq model with pointer networks 
Results
We perform a qualitative analysis of the Shakespearean prose generated for the input paintings  We conducted a survey, in which we presented famous paintings including those shown in Figures FIGREF1 and FIGREF10 and the corresponding Shakespearean prose generated by the model, and asked 32 students to rate them on the basis of content, creativity and similarity to Shakespearean style on a Likert scale of 1-5  Figure FIGREF12 shows the result of our human evaluation 
The average content score across the paintings is 3 7 which demonstrates that the prose generated is relevant to the painting  The average creativity score is 3 9 which demonstrates that the model captures more than basic objects in the painting successfully using poetic clues in the scene  The average style score is 3 9 which demonstrates that the prose generated is perceived to be in the style of Shakespeare 
We also perform a quantitative analysis of style transfer by generating BLEU scores for the model output using the style transfer dataset  The variation of the BLEU scores with the source sentence lengths is shown in Figure FIGREF11  As expected, the BLEU scores decrease with increase in source sentence lengths 
Results ::: Implementation
All models were trained on Google Colab with a single GPU using Python 3 6 and Tensorflow 2 0  The number of hidden units for the encoder and decoder is 1,576 and 256 for seq2seq with global attention and seq2seq with pointer networks respectively  Adam optimizer was used with the default learning rate of 0 001  The model was trained for 25 epochs  We use pre-trained retrofitted word embeddings of dimension 192 
Results ::: Limitations
Since we do not have an end-to-end dataset, the generated English poem may not work well with Shakespeare style transfer as shown in Figure FIGREF12 for "Starry Night" with a low average content score  This happens when the style transfer dataset does not have similar words in the training set of sentences  A solution would be to expand the style transfer dataset, for a better representation of the poem data 
Results ::: Conclusions and Future Work
In conclusion, combining two pipelines with an intermediate representation works well in practice  We observe that a CNN-RNN based image-to-poem net combined with a seq2seq model with parallel text corpus for text style transfer synthesizes Shakespeare-style prose for a given painting  For the seq2seq model used, we observe that it performs better in practice using global attention as compared with local attention  We make our models and code publicly available BIBREF12  In future work we would like to experiment with GANs in the absence of non-parallel datasets, so that we can use varied styles for text style transfer  We would also like to experiment with cross aligned auto-encoders, which form a latent content representation, to efficiently separate style and content 
Context 3:  To incorporate BERT to WSD task, we take the final hidden state of the token corresponding to the target word (if more than one token, we average them) and add a classification layer for every target lemma, which is the same as the last layer of the Bi-LSTM model BIBREF11 
Methodology ::: GlossBERT
BERT can explicitly model the relationship of a pair of texts, which has shown to be beneficial to many pair-wise natural language understanding tasks  In order to fully leverage gloss information, we propose GlossBERT to construct context-gloss pairs from all possible senses of the target word in WordNet, thus treating WSD task as a sentence-pair classification problem 
We describe our construction method with an example (See Table TABREF1)  There are four targets in this sentence, and here we take target word research as an example:
Methodology ::: GlossBERT ::: Context-Gloss Pairs
The sentence containing target words is denoted as context sentence  For each target word, we extract glosses of all $N$ possible senses (here $N=4$) of the target word (research) in WordNet to obtain the gloss sentence  [CLS] and [SEP] marks are added to the context-gloss pairs to make it suitable for the input of BERT model  A similar idea is also used in aspect-based sentiment analysis BIBREF16 
Methodology ::: GlossBERT ::: Context-Gloss Pairs with Weak Supervision
Based on the previous construction method, we add weak supervised signals to the context-gloss pairs (see the highlighted part in Table TABREF1)  The signal in the gloss sentence aims to point out the target word, and the signal in the context sentence aims to emphasize the target word considering the situation that a target word may occur more than one time in the same sentence 
Therefore, each target word has $N$ context-gloss pair training instances ($label\in \lbrace yes, no\rbrace $)  When testing, we output the probability of $label=yes$ of each context-gloss pair and choose the sense corresponding to the highest probability as the prediction label of the target word  We experiment with three GlossBERT models:
Methodology ::: GlossBERT ::: GlossBERT(Token-CLS)
We use context-gloss pairs as input  We highlight the target word by taking the final hidden state of the token corresponding to the target word (if more than one token, we average them) and add a classification layer ($label\in \lbrace yes, no\rbrace $) 
Methodology ::: GlossBERT ::: GlossBERT(Sent-CLS)
We use context-gloss pairs as input  We take the final hidden state of the first token [CLS] as the representation of the whole sequence and add a classification layer ($label\in \lbrace yes, no\rbrace $), which does not highlight the target word 
Methodology ::: GlossBERT ::: GlossBERT(Sent-CLS-WS)
We use context-gloss pairs with weak supervision as input  We take the final hidden state of the first token [CLS] and add a classification layer ($label\in \lbrace yes, no\rbrace $), which weekly highlight the target word by the weak supervision 
Experiments ::: Datasets
The statistics of the WSD datasets are shown in Table TABREF12 
Experiments ::: Datasets ::: Training Dataset
Following previous work BIBREF13, BIBREF12, BIBREF10, BIBREF17, BIBREF9, BIBREF7, we choose SemCor3 0 as training corpus, which is the largest corpus manually annotated with WordNet sense for WSD 
Experiments ::: Datasets ::: Evaluation Datasets
We evaluate our method on several English all-words WSD datasets  For a fair comparison, we use the benchmark datasets proposed by BIBREF17 which include five standard all-words fine-grained WSD datasets from the Senseval and SemEval competitions: Senseval-2 (SE2), Senseval-3 (SE3), SemEval-2007 (SE07), SemEval-2013 (SE13) and SemEval-2015 (SE15)  Following BIBREF13, BIBREF12 and BIBREF10, we choose SE07, the smallest among these test sets, as the development set 
Experiments ::: Datasets ::: WordNet
Since BIBREF17 map all the sense annotations in these datasets from their original versions to WordNet 3 0, we extract word sense glosses from WordNet 3 0 
Experiments ::: Settings
We use the pre-trained uncased BERT$_\mathrm {BASE}$ model for fine-tuning, because we find that BERT$_\mathrm {LARGE}$ model performs slightly worse than BERT$_\mathrm {BASE}$ in this task  The number of Transformer blocks is 12, the number of the hidden layer is 768, the number of self-attention heads is 12, and the total number of parameters of the pre-trained model is 110M  When fine-tuning, we use the development set (SE07) to find the optimal settings for our experiments  We keep the dropout probability at 0 1, set the number of epochs to 4  The initial learning rate is 2e-5, and the batch size is 64 
Experiments ::: Results
Table TABREF19 shows the performance of our method on the English all-words WSD benchmark datasets  We compare our approach with previous methods

ground_truth: generating a poem from images we use an existing actor-critic architecture, various types of sequence to sequence models

prompt_front: You are given a scientific article and a question. Answer the question as concisely as you can, using a single phrase or sentence if possible. If the question cannot be answered based on the information in the article, write "unanswerable". If the question is a yes/no question, answer "yes", "no", or "unanswerable". Do not provide any explanation.

Please answer the questions based on the text provided below. Is SemCor3.0 reflective of English language data in general?
Context 3: 
The first block shows the MFS baseline, which selects the most frequent sense in the training corpus for each target word 
The second block shows two knowledge-based systems  Lesk$_{ext+emb}$ BIBREF4 is a variant of Lesk algorithm BIBREF2 by calculating the gloss-context overlap of the target word  Babelfy BIBREF6 is a unified graph-based approach which exploits the semantic network structure from BabelNet 
The third block shows two word expert traditional supervised systems  IMS BIBREF7 is a flexible framework which trains SVM classifiers and uses local features  And IMS$_{+emb}$ BIBREF9 is the best configuration of the IMS framework, which also integrates word embeddings as features 
The fourth block shows several recent neural-based methods  Bi-LSTM BIBREF11 is a baseline for neural models  Bi-LSTM$_{+ att  + LEX + POS}$ BIBREF10 is a multi-task learning framework for WSD, POS tagging, and LEX with self-attention mechanism, which converts WSD to a sequence learning task  GAS$_{ext}$ BIBREF12 is a variant of GAS which is a gloss-augmented variant of the memory network by extending gloss knowledge  CAN$^s$ and HCAN BIBREF13 are sentence-level and hierarchical co-attention neural network models which leverage gloss knowledge 
In the last block, we report the performance of our method  BERT(Token-CLS) is our baseline, which does not incorporate gloss information, and it performs slightly worse than previous traditional supervised methods and recent neural-based methods  It proves that directly using BERT cannot obtain performance growth  The other three methods outperform other models by a substantial margin, which proves that the improvements come from leveraging BERT to better exploit gloss information  It is worth noting that our method achieves significant improvements in SE07 and Verb over previous methods, which have the highest ambiguity level among all datasets and all POS tags respectively according to BIBREF17 
Moreover, GlossBERT(Token-CLS) performs better than GlossBERT(Sent-CLS), which proves that highlighting the target word in the sentence is important  However, the weakly highlighting method GlossBERT(Sent-CLS-WS) performs best in most circumstances, which may result from its combination of the advantages of the other two methods 
Experiments ::: Discussion
There are two main reasons for the great improvements of our experimental results  First, we construct context-gloss pairs and convert WSD problem to a sentence-pair classification task which is similar to NLI tasks and train only one classifier, which is equivalent to expanding the corpus  Second, we leverage BERT BIBREF15 to better exploit the gloss information  BERT model shows its advantage in dealing with sentence-pair classification tasks by its amazing improvement on QA and NLI tasks  This advantage comes from both of its two novel unsupervised prediction tasks 
Compared with traditional word expert supervised methods, our GlossBERT shows its effectiveness to alleviate the effort of feature engineering and does not require training a dedicated classifier for every target lemma  Up to now, it can be said that the neural network method can totally beat the traditional word expert method  Compared with recent neural-based methods, our solution is more intuitive and can make better use of gloss knowledge  Besides, our approach demonstrates that when we fine-tune BERT on a downstream task, converting it into a sentence-pair classification task may be a good choice 
Conclusion
In this paper, we seek to better leverage gloss knowledge in a supervised neural WSD system  We propose a new solution to WSD by constructing context-gloss pairs and then converting WSD to a sentence-pair classification task  We fine-tune the pre-trained BERT model and achieve new state-of-the-art results on WSD task 
Acknowledgments
We would like to thank the anonymous reviewers for their valuable comments  The research work is supported by National Natural Science Foundation of China (No  61751201 and 61672162), Shanghai Municipal Science and Technology Commission (16JC1420401 and 17JC1404100), Shanghai Municipal Science and Technology Major Project (No 2018SHZDZX01) and ZJLab 

ground_truth: Yes

Unanswerable