DataHammer/scimrc huggingface dataset:
Question: Who does the Navy Cross is awarded to?
Evidence: The Navy Cross is the United States Naval Service's second-highest military decoration awarded for sailors and marines who distinguish themselves for extraordinary heroism in combat with an armed enemy force. The medal is equivalent to the Army's Distinguished Service Cross, the Air and Space Forces' Air Force Cross, and the Coast Guard Cross.

The Navy Cross is bestowed by the Secretary of the Navy and may also be awarded to members of the other armed services, and to foreign military personnel while serving with the U.S. Naval Service. The Navy Cross was established by Act of Congress (Public Law 65-253) and approved on February 4, 1919.
Answer: The Navy Cross is awarded to Sailors and Marines who display heroism in combat with an armed enemy force.
Question: What is the baseline?
Evidence: We compare the results of our model to a baseline multi-task architecture inspired by yang2016multi. In our baseline model there are no explicit connections between tasks - the only shared parameters are in the hidden layer.
We compare the results of our model to a baseline multi-task architecture inspired by yang2016multi. In our baseline model there are no explicit connections between tasks - the only shared parameters are in the hidden layer.
Answer: The baseline is a multi-task architecture inspired by another paper.
Question: What dataset do the authors use to test modifications?
Evidence: Extractive multi-document summarization (MDS) aims to summarize a collection of documents by selecting a small number of sentences that represent the original content appropriately. Typical objectives for assembling a summary include information coverage and non-redundancy. A wide variety of methods have been introduced to approach MDS.
Many approaches are based on sentence ranking, i.e. assigning each sentence a score that indicates how well the sentence summarizes the input BIBREF0 , BIBREF1 , BIBREF2 . A summary is created by selecting the top entries of the ranked list of sentences. Since the sentences are often treated separately, these models might allow redundancy in the summary. Therefore, they are often extended by an anti-redundancy filter while de-queuing ranked sentence lists.
Other approaches work at summary-level rather than sentence-level and aim to optimize functions of sets of sentences to find good summaries, such as KL-divergence between probability distributions BIBREF3 or submodular functions that represent coverage, diversity, etc. BIBREF4 
The centroid-based model belongs to the former group: it represents sentences as bag-of-word (BOW) vectors with TF-IDF weighting and uses a centroid of these vectors to represent the whole document collection BIBREF5 . The sentences are ranked by their cosine similarity to the centroid vector. This method is often found as a baseline in evaluations where it usually is outperformed BIBREF0 , BIBREF6 .
This baseline can easily be adapted to work at the summary-level instead the sentence level. This is done by representing a summary as the centroid of its sentence vectors and maximizing the similarity between the summary centroid and the centroid of the document collection. A simple greedy algorithm is used to find the best summary under a length constraint.
In order to keep the method efficient, we outline different methods to select a small number of candidate sentences from each document in the input collection before constructing the summary.
We test these modifications on the DUC2004 dataset for multi-document summarization. The results show an improvement of Rouge scores over the original centroid method. The performance is on par with state-of-the-art methods which shows that the similarity between a summary centroid and the input centroid is a well-suited function for global summary optimization.
The summarization approach presented in this paper is fast, unsupervised and simple to implement. Nevertheless, it performs as well as more complex state-of-the-art approaches in terms of Rouge scores on the DUC2004 dataset. It can be used as a strong baseline for future research or as a fast and easy-to-deploy summarization tool.
We test these modifications on the DUC2004 dataset for multi-document summarization.
Answer: DUC2004 dataset.
Question: Do their contributions contain a dataset?
Evidence: We consider three datasets, two of which are a contribution of this work.
Dataset I consists of 480 concepts previously analyzed in BIBREF1, BIBREF4. 240 are positive examples, titles from the Wikipedia list of controversial issues, and 240 are negative examples chosen at random and exclusive of the positives. Over this dataset, we compare the methodology suggested here to those reported by BIBREF1, BIBREF4. As the latter report overall accuracy of their binary prediction, we convert our controversiality estimates to a binary classification by classifying the higher-scored half as controversial, and the lower half as non-controversial.
Dataset II is based on a more recent version of the Wikipedia list of controversial issues (May 2017). As positive examples we take, from this list, all concepts which appear more than 50 times in Wikipedia. This leaves 608 controversial Wikipedia concepts. For negative examples, we follow BIBREF1, BIBREF4 and select a like number of concepts at random. Here too, since each concept only has a binary label, we convert our estimation into a binary classification, and report accuracy.
Dataset III is extracted from 3561 concepts whose Wikipedia pages are under edit protection, assuming that many of them are likely to be controversial. They were then crowd-annotated, with 10 or more annotators per concept. The annotation instructions were: “Given a topic and its description on Wikipedia, mark if this is a topic that people are likely to argue about.”. Average pairwise kappa agreement on this task was 0.532. Annotations were normalized to controversiality scores on an integer scale of 0 - 10. We used this dataset for testing the models trained on Dataset I.
In all datasets, to obtain the sentence-level context of the concepts (positive and negative), we randomly select two equal-sized sets of Wikipedia sentences, that explicitly reference these concepts – i.e., that contain a hyperlink to the article titled by the concept. Importantly, in each sentence we mask the words that reference the concept – i.e., the surface form of the hyperlink leading to the concept – by a fixed, singular token, thus focusing solely on the context within which the concepts are mentioned.
We consider three datasets, two of which are a contribution of this work.
Answer: Yes.
Question: Which of the following country belongs to Asia and North America? China, Canada, Pakistan, Mexico
Answer: China belongs to Asia, Pakistan belongs to Asia, Canada belongs to  North America, Mexico belongs to  North America
Question: Identify which instrument is string or woodwind: Wheelharp, Koudi
Answer: Koudi is woodwind, Wheelharp is string.
Question: What is the corpus used by authors for training English-Hindi NMT systems?
Evidence: In this section, we describe the languages experimented with, datasets used, the network hyper-parameters used in our experiments.
For training English-Hindi NMT systems, we use the IITB English-Hindi parallel corpus (Kunchukuttan et al., 2018) (1.46M sentences from the training set) and the ILCI English-Hindi parallel corpus (44.7K sentences).
Answer: The authors use the IITB English-Hindi parallel corpus (1.46M sentences from the training set) and the ILCI English-Hindi parallel corpus (44.7K sentences) for training English-Hindi NMT systems.
Question: What are some examples of a woodwind instrument?
Answer: Some examples of woodwind instruments include flute, clarinet, and oboe.
Question: What is the previous work's model?
Evidence: Mathur et al. in their paper for detecting offensive tweets proposed a Ternary Trans-CNN model where they train a model architecture comprising of 3 layers of Convolution 1D having filter sizes of 15, 12 and 10 and kernel size of 3 followed by 2 dense fully connected layer of size 64 and 3. The first dense FC layer has ReLU activation while the last Dense layer had Softmax activation. They were able to train this network on a parallel English dataset provided by Davidson et al. The authors were able to achieve Accuracy of 83.9%, Precision of 80.2%, Recall of 69.8%.
The approach looked promising given that the dataset was merely 3189 sentences divided into three categories and thus we replicated the experiment but failed to replicate the results. The results were poor than what the original authors achieved. But, most of the model hyper-parameter choices where inspired from this work.
Mathur et al. in their paper for detecting offensive tweets proposed a Ternary Trans-CNN model where they train a model architecture comprising of 3 layers of Convolution 1D having filter sizes of 15, 12 and 10 and kernel size of 3 followed by 2 dense fully connected layer of size 64 and 3. The first dense FC layer has ReLU activation while the last Dense layer had Softmax activation. They were able to train this network on a parallel English dataset provided by Davidson et al. The authors were able to achieve Accuracy of 83.9%, Precision of 80.2%, Recall of 69.8%.

The approach looked promising given that the dataset was merely 3189 sentences divided into three categories and thus we replicated the experiment but failed to replicate the results. The results were poor than what the original authors achieved. But, most of the model hyper-parameter choices where inspired from this work.
Answer: The answers are shown as follows:
* Ternary Trans-CNN
Question: What is applied to extract entities from the sentence?
Evidence: Figure FIGREF6 summarizes the proposed model architecture. The model takes character sequence as input and captures contextual features using BERT. A CRF layer is applied to extract entities from the sentence. To effectively transmit information between entity recognition and relation extraction, soft label embedding is built on the top of CRF logits. To solve the problem that one entity belongs to multiple triplets, a multi-sigmoid layer is applied. We find that adding an auxiliary global relation prediction task also improve the performance.
A CRF layer is applied to extract entities from the sentence.
Answer: A CRF layer is applied to extract entities from the sentence.
Question: Which of the following months have 31 days? Classify them as 'yes' or 'no'. January, February, March, April, May, June, July, August, September, October, November and December.
Answer: 'yes', 'no', 'yes', 'no', 'yes', 'no', 'yes', 'yes', 'no', 'yes', 'no', 'yes'.
Question: Name some famous orange cats.
Answer: Some famous orange cats include Tigger, Heathcliff, Garfield, Crookshanks, and Morris.
Question: List all the names of the underground lines in london, in alphabetical order
Evidence: London Underground's eleven lines total 402 kilometres (250 mi) in length, making it the seventh longest metro system in the world. These are made up of the sub-surface network and the deep-tube lines. The Circle, District, Hammersmith & City, and Metropolitan lines form the sub-surface network, with railway tunnels just below the surface and of a similar size to those on British main lines, converging on a circular bi-directional loop around zone 1. The Hammersmith & City and Circle lines share stations and most of their track with each other, as well as with the Metropolitan and District lines. The Bakerloo, Central, Jubilee, Northern, Piccadilly, Victoria and Waterloo & City lines are deep-level tubes, with smaller trains that run in two circular tunnels (tubes) with a diameter about 11 feet 8 inches (3.56 m). These lines have the exclusive use of a pair of tracks, except for the Uxbridge branch of the Piccadilly line, which shares track with the District line between Acton Town and Hanger Lane Junction and with the Metropolitan line between Rayners Lane and Uxbridge; and the Bakerloo line, which shares track with London Overground's Watford DC Line for its aboveground section north of Queen's Park.

Fifty-five per cent of the system runs on the surface. There are 20 miles (32 km) of cut-and-cover tunnel and 93 miles (150 km) of tube tunnel. Many of the central London underground stations on deep-level tube routes are higher than the running lines to assist deceleration when arriving and acceleration when departing. Trains generally run on the left-hand track. In some places, the tunnels are above each other (for example, the Central line east of St Paul's station), or the running tunnels are on the right (for example on the Victoria line between Warren Street and King's Cross St. Pancras, to allow cross-platform interchange with the Northern line at Euston).

The lines are electrified with a four-rail DC system: a conductor rail between the rails is energised at −210 V and a rail outside the running rails at +420 V, giving a potential difference of 630 V. On the sections of line shared with mainline trains, such as the District line from East Putney to Wimbledon and Gunnersbury to Richmond, and the Bakerloo line north of Queen's Park, the centre rail is bonded to the running rails.

The average speed on the Underground is 20.5 mph (33.0 km/h). Outside the tunnels of central London, many lines' trains tend to travel at over 40 mph (64 km/h) in the suburban and countryside areas. The Metropolitan line can reach speeds of 62 mph (100 km/h).
Answer: 1. Bakerloo
2. Central
3. Circle
4. District
5. Hammersmit and City  
6. Jubilee
7. London overground
8. Metropolitan
9. Northern
10. Piccadilly
11. Victoria
12. Waterloo and City
Question: What are the two stages of a QA task?
Evidence: BioASQ is a biomedical document classification, document retrieval, and question answering competition, currently in its seventh year. We provide an overview of our submissions to semantic question answering task (7b, Phase B) of BioASQ 7 (except for 'ideal answer' test, in which we did not participate this year). In this task systems are provided with biomedical questions and are required to submit ideal and exact answers to those questions. We have used BioBERT BIBREF0 based system , see also Bidirectional Encoder Representations from Transformers(BERT) BIBREF1, and we fine tuned it for the biomedical question answering task. Our system scored near the top for factoid questions for all the batches of the challenge. More specifially, in the third test batch set, our system achieved highest ‘MRR’ score for Factoid Question Answering task. Also, for List-type question answering task our system achieved highest recall score in the fourth test batch set. Along with our detailed approach, we present the results for our submissions and also highlight identified downsides for our current approach and ways to improve them in our future experiments. In last test batch results we placed 4th for List-type questions and 3rd for Factoid-type questions.)
The QA task is organized in two phases. Phase A deals with retrieval of the relevant document, snippets, concepts, and RDF triples, and phase B deals with exact and ideal answer generations (which is a paragraph size summary of snippets). Exact answer generation is required for factoid, list, and yes/no type question.
BioASQ organizers provide the training and testing data. The training data consists of questions, gold standard documents, snippets, concepts, and ideal answers (which we did not use in this paper, but we used last year BIBREF2). The test data is split between phases A and B. The Phase A dataset consists of the questions, unique ids, question types. The Phase B dataset consists of the questions, golden standard documents, snippets, unique ids and question types. Exact answers for factoid type questions are evaluated using strict accuracy (the top answer), lenient accuracy (the top 5 answers), and MRR (Mean Reciprocal Rank) which takes into account the ranks of returned answers. Answers for the list type question are evaluated using precision, recall, and F-measure.
Phase A deals with retrieval of the relevant document, snippets, concepts, and RDF triples, and phase B deals with exact and ideal answer generations (which is a paragraph size summary of snippets).
Answer: Phase A deals with retrieval of the relevant document, snippets, concepts, and RDF triples, and phase B deals with exact and ideal answer generations.
Question: Tell me whether each city is on the west coast or east coast of the USA: Seattle, Portland, Miami, Charlotte, Boston, San Diego
Answer: Seattle: West Coast
Portland: West Coast
Miami: East Coast
Charlotte: East Coast
Boston: East Coast
San Diego: West Coast
Question: What are some good things to do while spending a day in the Tamalpais Valley area of Mill Valley, CA?
Answer: Tamalpais Valley is a small neighborhood located in Marin County, California. A number of enjoyable activities can be had there during a short visit. Surrounded by lush hills, you will find a number of well maintained hiking trails. You can also enjoy delicious food at Cafe del Soul or Good Earth. If you need a quick pick-me-up, head over to Equator Coffees for a latte or espresso. Finally, you can cap your night off socializing with friends at The Junction over a pint of beer served from one of their 30 rotating taps.
Question: From the passage provided, extract the average size of a female great white shark.
Evidence: The great white shark (Carcharodon carcharias), also known as the white shark, white pointer, or simply great white, is a species of large mackerel shark which can be found in the coastal surface waters of all the major oceans. It is notable for its size, with the largest preserved female specimen measuring 5.83 m (19.1 ft) in length and around 2,000 kg (4,410 lb) in weight at maturity. However, most are smaller; males measure 3.4 to 4.0 m (11 to 13 ft), and females measure 4.6 to 4.9 m (15 to 16 ft) on average. According to a 2014 study, the lifespan of great white sharks is estimated to be as long as 70 years or more, well above previous estimates, making it one of the longest lived cartilaginous fishes currently known. According to the same study, male great white sharks take 26 years to reach sexual maturity, while the females take 33 years to be ready to produce offspring. Great white sharks can swim at speeds of 25 km/h (16 mph) for short bursts and to depths of 1,200 m (3,900 ft).
Answer: The average female great white shark measures 4.6 to 4.9 m or 15 to 16 ft.
Question: Classify each of the following as either individual sports or team sports: wrestling, boxing, football, basketball, swimming, handball, weightlifting.
Answer: Wrestling: individual sports
Boxing: individual sports
Football: team sports
Basketball: team sports
Swimming: individual sports
Handball: team sports
Weightlifting: individual sports
Question: Identify which animal species is alive or extinct: Velociraptor, Pacific Salmon
Answer: Velociraptor is extinct, Pacific Salmon is alive.
Question: In what year did Columbus reach the Americas?
Answer: Columbus first reached the Americas in 1492
Question: How is the input triple translated to a slot-filling task?
Evidence: We show that it is possible to reduce relation extraction to the problem of answering simple reading comprehension questions. We map each relation type $R(x,y)$ to at least one parametrized natural-language question $q_x$ whose answer is $y$ . For example, the relation $educated\_at(x,y)$ can be mapped to “Where did $x$ study?” and “Which university did $x$ graduate from?”. Given a particular entity $x$ (“Turing”) and a text that mentions $x$ (“Turing obtained his PhD from Princeton”), a non-null answer to any of these questions (“Princeton”) asserts the fact and also fills the slot $y$ . Figure 1 illustrates a few more examples.
We map each relation type $R(x,y)$ to at least one parametrized natural-language question $q_x$ whose answer is $y$ .
Answer: The relation R(x,y) is mapped onto a question q whose answer is y
Question: What frame did they import the final data to?
Evidence: We collected datasets of news articles in English and German language from the news agency Reuters (Table TABREF13 ). After a data cleaning step, which was deleting meta information like author and editor name from the article, title, body and date were stored in a local database and imported to a Pandas data frame BIBREF12 . The English corpus has a dictionary of length 106.848, the German version has a dictionary of length 163.788.
For each article we extracted with the Python library “Spacy” the named entities labeled as person. “Spacy” was used because of its good performance BIBREF13 and it has pre-trained language models for English, German and others. The entity recognition is not perfect, so we have errors in the lists of persons. In a post processing step the terms from a list of common errors are removed. The names of the persons appear in different versions like “Donald Trump” or “Trump”. We map all names to the shorter version i.e. “Trump” in this example.
In Figure FIGREF15 you can see the time series of the mentions of “Trump” in the news, with a peak at the 8th of November 2016 the day of the election. It is also visible that the general level is changing with the election and is on higher level since then.
Taking a look at the histograms of the most frequent persons in some timespan shows the top 20 persons in the English news articles from 2016 to 2018 (Figure FIGREF16 ). As expected the histogram has a distribution that follows Zipfs law BIBREF14 , BIBREF15 .
From the corpus data a dictionary is built, where for each person the number of mentions of this person in the news per day is recorded. This time series data can be used to build a model that covers time as parameter for the relationship to other persons.
After a data cleaning step, which was deleting meta information like author and editor name from the article, title, body and date were stored in a local database and imported to a Pandas9 data frame (McKinney, 2012).
Answer: A Pandas data frame.
Question: What are considered some of the hippest cities in the United States?
Answer: - Austin, Texas
- Portland, Oregon
- Portland, Maine,
- Brooklyn, New York
- Los Angeles, California
- Santa Fe, New Mexico
- Seattle, Washington
- Savannah, Georgia
- Nashville, Tennessee
- Charleston, South Carolina
Question: What does NSAID stand for?
Answer: NSAID is an abbreviation for a class of medicine called non-steroidal anti-inflammatory drugs.
Question: Which state-of-the-art model is surpassed by 9.68% attraction score?
Evidence: FLOAT SELECTED: Table 2: Human evaluation on three aspects: relevance, attraction, and fluency. “None” represents the original headlines in the dataset.
In terms of attraction scores in Table TABREF51, we have three findings: (1) The human-written headlines are more attractive than those from NHG, which agrees with our observation in Section SECREF1. (2) Our TitleStylist can generate more attractive headlines over the NHG and Multitask baselines for all three styles, demonstrating that adapting the model to these styles could improve the attraction and specialization of some parameters in the model for different styles can further enhance the attraction. (3) Adapting the model to the “Clickbait” style could create the most attractive headlines, even out-weighting the original ones, which agrees with the fact that click-baity headlines are better at drawing readers' attention. To be noted, although we learned the “Clickbait” style into our summarization system, we still made sure that we are generating relevant headlines instead of too exaggerated ones, which can be verified by our relevance scores.
FLOAT SELECTED: Table 2: Human evaluation on three aspects: relevance, attraction, and fluency. “None” represents the original headlines in the dataset.
In terms of attraction scores in Table TABREF51, we have three findings: (1) The human-written headlines are more attractive than those from NHG, which agrees with our observation in Section SECREF1.
Answer: The answers are shown as follows:
* pure summarization model NHG