DataHammer/scimrc huggingface dataset:
Question: How many BAE attack modes do the authors introduce?
Evidence: Recent studies have shown the vulnerability of ML models to adversarial attacks, small perturbations which lead to misclassification of inputs. Adversarial example generation in NLP BIBREF0 is more challenging than in common computer vision tasks BIBREF1, BIBREF2, BIBREF3 due to two main reasons: the discrete nature of input space and ensuring semantic coherence with the original sentence. A major bottleneck in applying gradient based BIBREF4 or generator model BIBREF5 based approaches to generate adversarial examples in NLP is the backward propagation of the perturbations from the continuous embedding space to the discrete token space.
Recent works for attacking text models rely on introducing errors at the character level in words BIBREF6, BIBREF7 or adding and deleting words BIBREF8, BIBREF9, BIBREF10, etc. for creating adversarial examples. These techniques often result in adversarial examples which are unnatural looking and lack grammatical correctness, and thus can be easily identified by humans.
TextFooler BIBREF11 is a black-box attack, that uses rule based synonym replacement from a fixed word embedding space to generate adversarial examples. These adversarial examples do not account for the overall semantics of the sentence, and consider only the token level similarity using word embeddings. This can lead to out-of-context and unnaturally complex replacements (see Table ), which can be easily identifiable by humans.
The recent advent of powerful language models BIBREF12, BIBREF13 in NLP has paved the way for using them in various downstream applications. In this paper, we present a simple yet novel technique: BAE (BERT-based Adversarial Examples), which uses a language model (LM) for token replacement to best fit the overall context. We perturb an input sentence by either replacing a token or inserting a new token in the sentence, by means of masking a part of the input and using a LM to fill in the mask (See Figure FIGREF1). BAE relies on the powerful BERT masked LM for ensuring grammatical correctness of the adversarial examples. Our attack beats the previous baselines by a large margin and confirms the inherent vulnerabilities of modern text classification models to adversarial attacks. Moreover, BAE produces more richer and natural looking adversarial examples as it uses the semantics learned by a LM.
To the best of our knowledge, we are the first to use a LM for adversarial example generation. We summarize our major contributions as follows:

We propose BAE, a novel strategy for generating natural looking adversarial examples using a masked language model.
We introduce 4 BAE attack modes, all of which are almost always stronger than previous baselines on 7 text classification datasets.
We show that, surprisingly, just a few replace/insert operations can reduce the accuracy of even a powerful BERT-based classifier by over $80\%$ on some datasets.

We introduce 4 BAE attack modes by replacing and inserting tokens, all of which are almost always stronger than previous baselines on 7 text classification datasets.
Answer: 4.
Question: How is the robustness of the model evaluated?
Evidence: We train our model for a maximal of 6 epochs. We select the best model parameters based on the best $F_1$ score on the development data after each epoch. Note that we split $10\%$ of data from the training data as the development data. The selected model is then applied to the test data for evaluation. During testing, we map words not appearing in the training data to the UNK token. Following the previous works, we perform 10-fold cross validation and report the average results. Our models and variants are implemented using PyTorch BIBREF26.
Following the previous works, we perform 10-fold cross validation and report the average results. 
Answer: The answers are shown as follows:
* 10-fold cross validation
Question: What were the key points of the Treaty of Versailles?
Evidence: The most critical and controversial provision in the treaty was: "The Allied and Associated Governments affirm and Germany accepts the responsibility of Germany and her allies for causing all the loss and damage to which the Allied and Associated Governments and their nationals have been subjected as a consequence of the war imposed upon them by the aggression of Germany and her allies." The other members of the Central Powers signed treaties containing similar articles. This article, Article 231, became known as the War Guilt clause. The treaty required Germany to disarm, make ample territorial concessions, and pay reparations to certain countries that had formed the Entente powers. In 1921 the total cost of these reparations was assessed at 132 billion gold marks (then $31.4 billion or £6.6 billion, roughly equivalent to US$442 billion or UK£284 billion in 2023). Because of the way the deal was structured, the Allied Powers intended Germany would only ever pay a value of 50 billion marks.
Answer: The Treaty of Versailles was a peace treaty to end World War I. The critical point in the treaty between the Germans and the Allied powers was that Germany accepted full responsibility for the loss and damage incurred to the Allies. Germany was required to disarm and pay reparations estimated at $31.4 billion in 1921.
Question: How many subword tokens for the left+right-marked (+m+) and the left-marked (+m)  markings respectively?
Evidence: The Finnish text data used for the language modeling task is provided by BIBREF13. The dataset consists mainly of newspapers and books of around 144 million word tokens and 4.2 million unique tokens. We use a Morfessor 2.0 BIBREF14 using the basic unsupervised Morfessor Baseline algorithm BIBREF15 with a corpus weight parameter ($\alpha $) of 0.001. We have a vocabulary of 34K subword tokens for the left+right-marked (+m+) markings and 19K subword tokens for the left-marked (+m) markings. We also pre-process the data to remove any punctuation marks such that we can use the same data with an ASR system. The input is one sentence per line and we shuffle the sentences at each epoch. The data is randomly divided into- training dataset and a validation dataset. The test dataset consists of 2850 Finnish news articles obtained from the Finnish national broadcaster YLE.
We have a vocabulary of 34K subword tokens for the left+right-marked (+m+) markings and 19K subword tokens for the left-marked (+m) markings.
Answer: 34K subword tokens and 19K subword tokens.
Question: How does the model perform compare with existing systems?
Evidence: In this paper, we present a deep neural network model for the task of fine-grained named entity classification using ELMo embeddings and Wikidata. The proposed model learns representations for entity mentions based on its context and incorporates the rich structure of Wikidata to augment these labels into finer-grained subtypes. We can see comparisons of our model made on Wiki(gold) in Table TABREF20 . We note that the model performs similarly to existing systems without being trained or tuned on that particular dataset. Future work may include refining the clustering method described in Section 2.2 to extend to types other than person, location, organization, and also to include disambiguation of entity types.
We note that the model performs similarly to existing systems without being trained or tuned on that particular dataset.
Answer: The model performs similarly to existing systems without being trained or tuned on that particular dataset.
Question: Given a reference text about the Anglo-Spanish War, tell me what treaty marked the end of the war.
Evidence: The Anglo-Spanish War (1585–1604) was an intermittent conflict between the Habsburg Kingdom of Spain and the Kingdom of England. It was never formally declared. The war included much English privateering against Spanish ships, and several widely separated battles. It began with England's military expedition in 1585 to what was then the Spanish Netherlands under the command of the Earl of Leicester, in support of the Dutch rebellion against Spanish Habsburg rule.

The English enjoyed a victory at Cádiz in 1587, and repelled the Spanish Armada in 1588, but then suffered heavy setbacks: the English Armada (1589), the Drake–Hawkins expedition (1595), and the Essex–Raleigh expedition (1597). Three further Spanish armadas were sent against England and Ireland in 1596, 1597, and 1601, but these likewise ended in failure for Spain, mainly because of adverse weather.

The war became deadlocked around the turn of the 17th century during campaigns in the Netherlands, France, and Ireland. It was brought to an end with the Treaty of London (1604), negotiated between Philip III of Spain and the new king of England, James I. In the treaty, England and Spain agreed to cease their military interventions in the Spanish Netherlands and Ireland, respectively, and the English ended their high seas privateering.
Answer: The Treaty of London marked the end of the Anglo-Spanish War.
Question: What are possible alliances for the future of AI?
Answer: Possible alliances could be based on use case, strategic fit, capability, and alignment. For example, an alliance with Amazon could enhance the use of AI as a personal home assistant, especially for seniors, as the population of many countries ages. Furthermore, alliances with manufacturing groups such as whirlpool could further integrate the home assistance.
Question: What is the research background of this paper?
Evidence: Emotion detection has long been a topic of interest to scholars in natural language processing (NLP) domain. Researchers aim to recognize the emotion behind the text and distribute similar ones into the same group. Establishing an emotion classifier can not only understand each user's feeling but also be extended to various application, for example, the motivation behind a user's interests BIBREF0. Based on releasing of large text corpus on social media and the emotion categories proposed by BIBREF1, BIBREF2, numerous models have provided and achieved fabulous precision so far. For example, DeepMoji BIBREF3 which utilized transfer learning concept to enhance emotions and sarcasm understanding behind the target sentence. CARER BIBREF4 learned contextualized affect representations to make itself more sensitive to rare words and the scenario behind the texts.
As methods become mature, text-based emotion detecting applications can be extended from a single utterance to a dialogue contributed by a series of utterances. Table TABREF2 illustrates the difference between single utterance and dialogue emotion recognition. The same utterances in Table TABREF2, even the same person said the same sentence, the emotion it convey may be various, which may depend on different background of the conversation, tone of speaking or personality. Therefore, for emotion detection, the information from preceding utterances in a conversation is relatively critical.
In SocialNLP 2019 EmotionX, the challenge is to recognize emotions for all utterances in EmotionLines dataset, a dataset consists of dialogues. According to the needs for considering context at the same time, we develop two classification models, inspired by bidirectional encoder representations from transformers (BERT) BIBREF5, FriendsBERT and ChatBERT. In this paper, we introduce our approaches including causal utterance modeling, model pre-training, and fine-turning.
Emotion detection has long been a topic of interest to scholars in natural language processing (NLP) domain. Researchers aim to recognize the emotion behind the text and distribute similar ones into the same group.
Answer: Emotion detection has long been a topic of interest to scholars in natural language processing (NLP) domain. Researchers aim to recognize the emotion behind the text and distribute similar ones into the same group.
Question: What are the top 3 boba places in Burlingame, CA?
Answer: 1. Happy Lemon, 2. Tiger Tea,  3. Boba Guys,
Question: What capability does the model have shown by the baseline results with multilingual BERT?
Evidence: We have presented a new dataset for political stance detection called x-stance. The dataset extends over a broad range of topics and issues regarding national Swiss politics. This diversity of topics opens up an opportunity to further study multi-target learning. Moreover, being partly Swiss Standard German, partly French and Italian, the dataset promotes a multilingual approach to stance detection.
By compiling formal commentary that politicians have written on political questions, we add a new text genre to the field of stance detection. We also propose a question–answer format that allows us to condition stance detection models on a target naturally.
Our baseline results with multilingual Bert show that the model has some capability to perform zero-shot transfer to unseen languages and to unseen targets (both within a topic and to unseen topics). However, there is some gap in performance that future work could address. We expect that the x-stance dataset could furthermore be a valuable resource for fields such as argument mining, argument search or topic classification.
Our baseline results with multilingual BERT show that the model has some capability to perform zero-shot transfer to unseen languages and to unseen targets (both within a topic and to unseen topics).
Answer: The model has some capability to perform zero-shot transfer to unseen languages and to unseen targets (both within a topic and to unseen topics).
Question: What are the words of House Arryn?
Answer: "As High as Honor"
Question: How can one get better at tennis?
Answer: People can improve their tennis skills by playing and practicing alot just like any other sports. Taking lessons or getting coaches is often considered the most efficient way to get better. For tennis, usually it is hard to practice without a hitting partner, but people still can practice by themselves by hitting the balls against the walls. Also tennis serving is one skill that can be practiced alone without a partner
Question: Tell me whether these characters are from Harry Potter or Game of Thrones
Answer: Tyrion Lannister, Luna Lovegood, Sirius Black, Draco Malfoy, Jon Snow, Sansa Stark
Question: Given a reference text about Qamus, when was it attacked and defeated?
Evidence: Qamus (القموص) was one of the fortresses of the Jewish poet Al-Rabi ibn Abu al-Huqayq, and his Jewish tribe called Banu Nadir. The fortress was situated near Khaybar in what is now Saudi Arabia. The fortress was attacked by Muslim forces and defeated circa 629/30 CE. It was after this event that Muhammad married Safiyyah bint Huyayy. The Jewish presence in the region has been attested to the late seventh century who pioneered the cultivation in the area.

It has traditionally been identified with the remains of the Husn al-Qamus (Qamus Fortress) in the old Khaybar oasis, though there's no direct archaeological evidence for this.
Answer: Qamus was attacked and defeated by Muslim forced near the year 630 CE
Question: Identify which instrument is string or percussion: Kemanek, Buzuq
Answer: Buzuq is string, Kemanek is percussion.
Question: What is process manufacturing?
Answer: Process manufacturing is the branch of manufacturing that manufactures goods by combining ingredients that include raw materials and chemicals following a recipe with a specific formulae. This recipe is often kept as a trade secret and often includes variables like heat, moisture content, mixing rates etc. Examples of goods due to process manufacturing include face creams, shampoos, beers and pharmaceutical products.
Question: What is Snow White and the Seven Dwarfs?
Evidence: Snow White and the Seven Dwarfs is a 1937 American animated musical fantasy film produced by Walt Disney Productions and released by RKO Radio Pictures. Based on the 1812 German fairy tale by the Brothers Grimm, it is the first full-length traditionally animated feature film and the first Disney animated feature film. The production was supervised by David Hand, and the film's sequences were directed by Perce Pearce, William Cottrell, Larry Morey, Wilfred Jackson, and Ben Sharpsteen.
Answer: Snow White and the Seven Dwarfs is an animated musical fantasy film produced in 1937. It was produced by Walt Disney Productions and released by RKO Radio Pictures. It is the first full-length traditionally animated feature film and the first animated feature film from Disney.
Question: What clustering algorithm is used on top of the VerbNet-specialized representations?
Evidence: Given the initial distributional or specialised collection of target language vectors INLINEFORM0 , we apply an off-the-shelf clustering algorithm on top of these vectors in order to group verbs into classes. Following prior work BIBREF56 , BIBREF57 , BIBREF17 , we employ the MNCut spectral clustering algorithm BIBREF58 , which has wide applicability in similar NLP tasks which involve high-dimensional feature spaces BIBREF59 , BIBREF60 , BIBREF18 . Again, following prior work BIBREF17 , BIBREF61 , we estimate the number of clusters INLINEFORM1 using the self-tuning method of Zelnik:2004nips. This algorithm finds the optimal number by minimising a cost function based on the eigenvector structure of the word similarity matrix. We refer the reader to the relevant literature for further details.
Following prior work BIBREF56 , BIBREF57 , BIBREF17 , we employ the MNCut spectral clustering algorithm BIBREF58 , which has wide applicability in similar NLP tasks which involve high-dimensional feature spaces BIBREF59 , BIBREF60 , BIBREF18 .
Answer: The answers are shown as follows:
* MNCut spectral clustering algorithm BIBREF58
Question: How many datasets are used to evaluate the quality of the document embeddings learned by MPAD?
Evidence: We evaluate the quality of the document embeddings learned by MPAD on 10 document classification datasets, covering the topic identification, coarse and fine sentiment analysis and opinion mining, and subjectivity detection tasks. We briefly introduce the datasets next. Their statistics are reported in Table TABREF21.
(1) Reuters. This dataset contains stories collected from the Reuters news agency in 1987. Following common practice, we used the ModApte split and considered only the 10 classes with the highest number of positive training examples. We also removed documents belonging to more than one class and then classes left with no document (2 classes).
(2) BBCSport BIBREF30 contains documents from the BBC Sport website corresponding to 2004-2005 sports news articles.
(3) Polarity BIBREF31 features positive and negative labeled snippets from Rotten Tomatoes.
(4) Subjectivity BIBREF32 contains movie review snippets from Rotten Tomatoes (subjective sentences), and Internet Movie Database plot summaries (objective sentences).
(5) MPQA BIBREF33 is made of positive and negative phrases, annotated as part of the summer 2002 NRRC Workshop on Multi-Perspective Question Answering.
(6) IMDB BIBREF34 is a collection of highly polarized movie reviews from IMDB (positive and negative). There are at most 30 reviews for each movie.
(7) TREC BIBREF35 consists of questions that are classified into 6 different categories.
(8) SST-1 BIBREF36 contains the same snippets as Polarity. The authors used the Stanford Parser to parse the snippets and split them into multiple sentences. They then used Amazon Mechanical Turk to annotate the resulting phrases according to their polarity (very negative, negative, neutral, positive, very positive).
(9) SST-2 BIBREF36 is the same as SST-1 but with neutral reviews removed and snippets classified as positive or negative.
(10) Yelp2013 BIBREF26 features reviews obtained from the 2013 Yelp Dataset Challenge.
We evaluate the quality of the document embeddings learned by MPAD on 10 document classification datasets, covering the topic identification, coarse and fine sentiment analysis and opinion mining, and subjectivity detection tasks.
Answer: Ten.
Question: Please give me a bulleted list of sports teams in Spokane.
Evidence: In sports, the region's professional and semi-professional sports teams include the Spokane Indians in Minor League Baseball and Spokane Chiefs in junior ice hockey. The Gonzaga Bulldogs collegiate basketball team competes at the Division I level. As of 2010, Spokane's major daily newspaper, The Spokesman-Review, had a daily circulation of over 76,000.
Answer: - Spokane Indians
- Spokane Chiefs
- Gonzaga Bulldogs
Question: Is there a movie about Wikipedia
Evidence: Truth in Numbers? Everything, According to Wikipedia is a 2010 American documentary film that explores the history and cultural implications of the online, editable encyclopedia Wikipedia. The film considers the question of whether all individuals or just experts should be tasked with editing an encyclopedia.

The site's history and background is given, along with commentary from Wikipedia founders Jimmy Wales and Larry Sanger. Commentators that appear in the film include author Howard Zinn, Len Downie of The Washington Post, Bob Schieffer of CBS News, former Encyclopædia Britannica chief Robert McHenry and former Central Intelligence Agency director James Woolsey. The documentary discusses incidents that shed a negative light on Wikipedia, including the Essjay controversy and the Wikipedia biography controversy.

The long-delayed film premiered at Wikimania 2010 in Gdańsk in July 2010, and was screened at the Paley Center for Media in New York City on October 20, 2010. It was shown as part of the Savannah Film Festival on November 3, 2010, at Savannah College of Art and Design's Trustees Theater. Truth in Numbers? received a mixed reception, with favorable commentary from author Ted Leonsis, in the AOL publication Urlesque, and coverage at the Savannah Film Festival by Carlos Serrano of District.
Answer: Yes. Truth in Numbers? Everything, According to Wikipedia is a 2010 American documentary film that explores the history and cultural implications of the online, editable encyclopedia Wikipedia. The film considers the question of whether all individuals or just experts should be tasked with editing an encyclopedia.
Question: What are the top boba places in San Mateo County??
Answer: There are a few great boba places in San Mateo 1. Yifang, 2. TP Tea, 3.  Urban Ritual 4. Cha Express 5.  Feng Cha
Question: what are the baselines?
Evidence: Our system learns Perceptron models BIBREF37 using the Machine Learning machinery provided by the Apache OpenNLP project with our own customized (local and clustering) features. Our NERC system is publicly available and distributed under the Apache 2.0 License and part of the IXA pipes tools BIBREF38 . Every result reported in this paper is obtained using the conlleval script from the CoNLL 2002 and CoNLL 2003 shared tasks. To guarantee reproducibility of results we also make publicly available the models and the scripts used to perform the evaluations. The system, models and evaluation scripts can be found in the ixa-pipe-nerc website.
The local features constitute our baseline system on top of which the clustering features are added. We implement the following feature set, partially inspired by previous work BIBREF46 :
Our system learns Perceptron models BIBREF37 using the Machine Learning machinery provided by the Apache OpenNLP project with our own customized (local and clustering) features. 
The local features constitute our baseline system on top of which the clustering features are added.
Answer: Perceptron model using the local features.