DataHammer/scimrc huggingface dataset:
Question: Do changes in policies of the political actors account for all of the mistakes the model made?
Evidence: In order to investigate the errors the models made confusion matrices were extracted for the predictions on the out-of-domain evaluation data for sentence level predictions (see tab:confusion) as well as topic level predictions (see tab:confusiontopic). One example illustrates that the mistakes the model makes can be associated with changes in the party policy. The green party has been promoting policies for renewable energy and against nuclear energy in their manifestos prior to both legislative periods. Yet the statements of the green party are more often predicted to be from the government parties than from the party that originally promoted these green ideas, reflecting the trend that these legislative periods governing parties took over policies from the green party. This effect is even more pronounced in the topic level predictions: a model trained on data from the 18th Bundestag predicts all manifesto topics of the green party to be from one of the parties of the governing coalition, CDU/CSU or SPD.
One example illustrates that the mistakes the model makes can be associated with changes in the party policy.
Answer: Yes.
Question: What is the origin of Flamenco?
Answer: Flamenco is a form of dance and music commonly associated with the Andalusian Roma community of southern Spain. The roots of flamenco are believed to lie in the Roma migration from Rajasthan, a state in west India, to Spain between the 9th and 14th centuries.
Question: Explain gravity
Answer: Gravity is a fundamental interaction which causes mutual attraction between all things with mass or energy. Gravity is the weakest of the four fundamental interactions namely the strong, weak, electromagnetic and gravity. However, it is the most significant interaction between objects at the macroscopic scale, and it determines the motion of planets, stars, galaxies, and even light. 
Gravity is most accurately described by the general theory of relativity that was proposed by Albert Einstein. The theory describes gravity not as a force, but as the curvature of spacetime, caused by the uneven distribution of mass, and causing masses to move along geodesic lines.
However, for most applications, gravity is well approximated by Newton's law of universal gravitation, which describes gravity as a force causing any two bodies to be attracted toward each other, with magnitude proportional to the product of their masses and inversely proportional to the square of the distance between them.
Question: Given a reference text about Thomas William Dobbie, tell me his nationality.
Evidence: Thomas William Dobbie (November 13, 1829 – April 1, 1908) was a Canadian civil engineer, provincial land surveyor and political figure. He represented Elgin East in the 1st Canadian Parliament as a Conservative member.

He was born in Bayham Township, Upper Canada in 1829, the son of Andrew Dobbie, a Scottish immigrant, and Martha Ann Bowlby. Dobbie was educated in Bayham township and London, Ontario. He served as reeve for the township from 1863 to 1867 and from 1876 to 1877. Dobbie lived in Tillsonburg.

Dobbie was married twice: to Susan Margaret Jones in 1872 and to Mary Hodgson in 1891. He died in Tillsonburg at the age of 78.
Answer: Thomas William Dobbie was Canadian.
Question: Given this paragraph about video games, when was the Legend of Zelda originally released?
Evidence: The Legend of Zelda, originally released in Japan as The Hyrule Fantasy: Zelda no Densetsu, is a 1986 action-adventure game developed and published by Nintendo. The first game of The Legend of Zelda series, it is set in the fantasy land of Hyrule and centers on an elf-like boy named Link, who aims to collect the eight fragments of the Triforce of Wisdom in order to rescue Princess Zelda from the antagonist Ganon. During the course of the game, the player controls Link from a top-down perspective and navigates throughout the overworld and dungeons, collecting weapons, defeating enemies and uncovering secrets along the way.
Answer: 1986
Question: write a story a bout a man walking a dog by the river
Answer: Once upon a time, there was a man called John. John used to get up early and go by the nearby river to walk his dog. one day, while walking by the river John noticed something on the other bank of the river, it was bright and shiny. John started to walk faster to cross the river to the other side, and once he did he found that shiny thing to be dome glass that has a tiny door to the left of it. John never recalled he has seen anything like that before, but once he knocked the door, young beautiful girl open the door, and said Hi. John said Hi back, and he asked is there anyone else living here with you. and the girl said yes, my mom, she is inside, let me call her for you. The girl mom came out and said Hi, how can I help you. and John said, sorry I just noticed this nice shiny dome, and thought to check who is inside. the Lady said yes, we are travellers and we found this nice spot by the river, the dome you see is all made artificially by an AI system, so it look as shiny as possible. we will be camping here by the river for couple of days, then leave after. John said, ok, enjoy time here, and let me know if you need any help. the lady thanked him, and John took his dog back to the other side of the river and continued their walk.
Question: Which of the other locations mentioned in this article is closest to Rudzk Duży?
Evidence: Rudzk Duży [ˈrut͡sk ˈduʐɨ] is a village in the administrative district of Gmina Piotrków Kujawski, within Radziejów County, Kuyavian-Pomeranian Voivodeship, in north-central Poland. It lies approximately 6 kilometres (4 mi) south-west of Piotrków Kujawski, 15 km (9 mi) south-west of Radziejów, and 59 km (37 mi) south of Toruń.
Answer: Piotrków Kujawski. Rudzk Duży is closest to Piotrków Kujawski.
Question: In the paper what does the kernel of a topic mean?
Evidence: To estimate the quality of topic models, we use two main automatic measures: topic coherence and kernel uniqueness. For human content analysis, measures of topic coherence and kernel uniqueness are both important and complement each other. Topics can be coherent but have a lot of repetitions. On the other hand, generated topics can be very diverse, but incoherent within each topic.
Topic coherence is an automatic metric of interpretability. It was shown that the coherence measure has a high correlation with the expert estimates of topic interpretability BIBREF9 , BIBREF17 . Mimno BIBREF9 described an experiment comparing expert evaluation of LDA-generated topics and automatic topic coherence measures. It was found that most "bad" topics consisted of words without clear relations between each other.
Newman et al. BIBREF6 asked users to score topics on a 3-point scale, where 3=“useful” (coherent) and 1=“useless” (less coherent). They instructed the users that one indicator of usefulness is the ease by which one could think of a short label to describe a topic. Then several automatic measures, including WordNet-based measures and corpus co-occurrence measures, were compared. It was found that the best automatic measure having the largest correlation with human evaluation is word co-occurrence calculated as point-wise mutual information (PMI) on Wikipedia articles. Later Lau et al. BIBREF17 showed that normalized poinwise mutual information (NPMI) BIBREF18 calculated on Wikipedia articles correlates even more strongly with human scores.
We calculate automatic topic coherence using two measure variants. The coherence of a topic is the median PMI (NPMI) of word pairs representing the topic, usually it is calculated for INLINEFORM0 most probable elements (in our study ten elements) in the topic. The coherence of the model is the median of the topic coherence. To make this measure more objective, it should be calculated on an external corpus BIBREF17 . In our case, we use Wikipedia dumps. DISPLAYFORM0 
Human-constructed topics usually have unique main words. The measure of kernel uniqueness shows to what extent topics are different from each other and is calculated as the number of unique elements among most probable elements of topics (kernels) in relation to the whole number of elements in kernels. DISPLAYFORM0 
If uniqueness of the topic kernels is closer to zero then many topics are similar to each other, contain the same words in their kernels. In this paper the kernel of a topic means the ten most probable words in the topic. We also calculated perplexity as the measure of language models. We use it for additional checking the model quality.
In this paper the kernel of a topic means the ten most probable words in the topic.
Answer: The ten most probable words in the topic.
Question: How many metrics in question generation are evaluated by the team?
Evidence: We conduct experiments on the SQuAD dataset BIBREF3. It contains 536 Wikipedia articles and 100k crowd-sourced question-answer pairs. The questions are written by crowd-workers and the answers are spans of tokens in the articles. We employ two different data splits by following Zhou2017NeuralQG and Du2017LearningTA . In Zhou2017NeuralQG, the original SQuAD development set is evenly divided into dev and test sets, while Du2017LearningTA treats SQuAD development set as its development set and splits original SQuAD training set into a training set and a test set. We also filter out questions which do not have any overlapped non-stop words with the corresponding sentences and perform some preprocessing steps, such as tokenization and sentence splitting. The data statistics are given in Table TABREF27.
We evaluate with all commonly-used metrics in question generation BIBREF13: BLEU-1 (B1), BLEU-2 (B2), BLEU-3 (B3), BLEU-4 (B4) BIBREF17, METEOR (MET) BIBREF18 and ROUGE-L (R-L) BIBREF19. We use the evaluation script released by Chen2015MicrosoftCC.
We evaluate with all commonly-used metrics in question generation (Du et al., 2017): BLEU- 1 (B1), BLEU-2 (B2), BLEU-3 (B3), BLEU-4 (B4) (Papineni et al., 2002), METEOR (MET) (Denkowski and Lavie, 2014) and ROUGE-L (RL) (Lin, 2004).
Answer: Six.
Question: Given a reference text about Joseph A. Garcia, tell me what high school and college he attended.
Evidence: Joseph A. Garcia (born March 21, 1957) is an American lawyer and politician. He served as the 48th Lieutenant Governor of Colorado from January 2011 to May 2016.

Early life, education and career
Joseph A. Garcia was born on March 21, 1957, in Lafayette, Indiana. He attended Austin E. Lathrop High School in Fairbanks, Alaska from 1973 to 1974 and graduated from T.C. Williams High School in Alexandria, Virginia in 1975. He graduated from the University of Colorado Boulder in 1979 where he received a Bachelor of Science in Business. Garcia then graduated from Harvard Law School in 1983.

After law school, Garcia took a job with the firm of Holme Roberts & Owen, going on to become their first Hispanic partner. Garcia worked with the firm from 1983 to 1993. Garcia was appointed by Governor Romer as the Executive Director of the Department of Regulatory Agencies in 1993. In 1999 Garcia was appointed by the White House as the Regional Director of the Rocky Mountain States for the United States Department of Housing and Urban Development.
Answer: Joseph A. Garcia attended Austin E. Lathrop High School and T.C. Williams High School. After high school, he attended the University of Colorado Boulder and Harvard Law School.
Question: What kind of Youtube video transcripts did they use?
Evidence: We focused evaluation over a small but diversified dataset composed by 10 YouTube videos in the English language in the news context. The selected videos cover different topics like technology, human rights, terrorism and politics with a length variation between 2 and 10 minutes. To encourage the diversity of content format we included newscasts, interviews, reports and round tables.
We focused evaluation over a small but diversified dataset composed by 10 YouTube videos in the English language in the news context. The selected videos cover different topics like technology, human rights, terrorism and politics with a length variation between 2 and 10 minutes. 
Answer: youtube video transcripts on news covering different topics like technology, human rights, terrorism and politics
Question: Is their system a semi-supervised one or an unsupervised one ?
Evidence: Deep learning systems have shown a lot of promise for extractive Question Answering (QA), with performance comparable to humans when large scale data is available. However, practitioners looking to build QA systems for specific applications may not have the resources to collect tens of thousands of questions on corpora of their choice. At the same time, state-of-the-art machine reading systems do not lend well to low-resource QA settings where the number of labeled question-answer pairs are limited (c.f. Table 2 ). Semi-supervised QA methods like BIBREF0 aim to improve this performance by leveraging unlabeled data which is easier to collect.
In this work, we present a semi-supervised QA system which requires the end user to specify a set of base documents and only a small set of question-answer pairs over a subset of these documents. Our proposed system consists of three stages. First, we construct cloze-style questions (predicting missing spans of text) from the unlabeled corpus; next, we use the generated clozes to pre-train a powerful neural network model for extractive QA BIBREF1 , BIBREF2 ; and finally, we fine-tune the model on the small set of provided QA pairs.
Our cloze construction process builds on a typical writing phenomenon and document structure: an introduction precedes and summarizes the main body of the article. Many large corpora follow such a structure, including Wikipedia, academic papers, and news articles. We hypothesize that we can benefit from the un-annotated corpora to better answer various questions – at least ones that are lexically similar to the content in base documents and directly require factual information.
We apply the proposed system on three datasets from different domains – SQuAD BIBREF3 , TriviaQA-Web BIBREF4 and the BioASQ challenge BIBREF5 . We observe significant improvements in a low-resource setting across all three datasets. For SQuAD and TriviaQA, we attain an F1 score of more than 50% by merely using 1% of the training data. Our system outperforms the approaches for semi-supervised QA presented in BIBREF0 , and a baseline which uses the same unlabeled data but with a language modeling objective for pretraining. In the BioASQ challenge, we outperform the best performing system from previous year's challenge, improving over a baseline which does transfer learning from the SQuAD dataset. Our analysis reveals that questions which ask for factual information and match to specific parts of the context documents benefit the most from pretraining on automatically constructed clozes.
In this work, we present a semi-supervised QA system which requires the end user to specify a set of base documents and only a small set of question-answer pairs over a subset of these documents.
Answer: A semi-supervised  QA system.
Question: Extract from the following passage, the periods of time does sunrise refer to.
Evidence: Although the Sun appears to "rise" from the horizon, it is actually the Earth's motion that causes the Sun to appear. The illusion of a moving Sun results from Earth observers being in a rotating reference frame; this apparent motion caused many cultures to have mythologies and religions built around the geocentric model, which prevailed until astronomer Nicolaus Copernicus formulated his heliocentric model in the 16th century.
Astronomically, sunrise occurs for only an instant: the moment at which the upper limb of the Sun appears tangent to the horizon. However, the term sunrise commonly refers to periods of time both before and after this point:
Twilight, the period in the morning during which the sky is brightening, but the Sun is not yet visible. The beginning of morning twilight is called astronomical dawn.
The period after the Sun rises during which striking colors and atmospheric effects are still seen.
The timing of sunrise varies throughout the year and is also affected by the viewer's latitude and longitude, altitude, and time zone. These changes are driven by the axial tilt of Earth, daily rotation of the Earth, the planet's movement in its annual elliptical orbit around the Sun, and the Earth and Moon's paired revolutions around each other. The analemma can be used to make approximate predictions of the time of sunrise.
In late winter and spring, sunrise as seen from temperate latitudes occurs earlier each day, reaching its earliest time near the summer solstice; although the exact date varies by latitude. After this point, the time of sunrise gets later each day, reaching its latest sometime around the winter solstice. The offset between the dates of the solstice and the earliest or latest sunrise time is caused by the eccentricity of Earth's orbit and the tilt of its axis, and is described by the analemma, which can be used to predict the dates.
Variations in atmospheric refraction can alter the time of sunrise by changing its apparent position. Near the poles, the time-of-day variation is exaggerated, since the Sun crosses the horizon at a very shallow angle and thus rises more slowly.
Accounting for atmospheric refraction and measuring from the leading edge slightly increases the average duration of day relative to night. The sunrise equation, however, which is used to derive the time of sunrise and sunset, uses the Sun's physical center for calculation, neglecting atmospheric refraction and the non-zero angle subtended by the solar disc.
Answer: Astronomically, sunrise occurs for only an instant: the moment at which the upper limb of the Sun appears tangent to the horizon. However, the term sunrise commonly refers to periods of time both before and after this point. Twilight, the period in the morning during which the sky is brightening, but the Sun is not yet visible. The beginning of morning twilight is called astronomical dawn. The period after the Sun rises during which striking colors and atmospheric effects are still seen.
Question: What is splitboarding?
Answer: First of all, a splitboard is a snowboard that splits in half length-wise to allow for enhanced mobility while climbing up a slope. When the splitboard is split in half, you adjust the bindings so the setup looks just like a ski setup. After you split the board, you put climbing skins on the bottom of the splitboard to give you traction on the snow. After reaching the top  of your destination, you remove your feet from the bindings, remove the skins and then put the splitboard back together. Once the splitboard and bindings are reassembled, you snowboard down the mountain.
Question: Which is a species of fish? Marlin or Doberman
Answer: Marlin
Question: Given this paragraph about the terrain of the Iberian Peninsula, tell me the highest point in the peninsula and its altitude.
Evidence: The terrain of the Iberian Peninsula is largely mountainous. The major mountain systems are:

The Pyrenees and their foothills, the Pre-Pyrenees, crossing the isthmus of the peninsula so completely as to allow no passage except by mountain road, trail, coastal road or tunnel. Aneto in the Maladeta massif, at 3,404 m, is the highest point

The Mulhacén, the highest peak in the Iberian Peninsula
The Cantabrian Mountains along the northern coast with the massive Picos de Europa. Torre de Cerredo, at 2,648 m, is the highest point
The Galicia/Trás-os-Montes Massif in the Northwest is made up of very old heavily eroded rocks. Pena Trevinca, at 2,127 m, is the highest point
The Sistema Ibérico, a complex system at the heart of the peninsula, in its central/eastern region. It contains a great number of ranges and divides the watershed of the Tagus, Douro and Ebro rivers. Moncayo, at 2,313 m, is the highest point
The Sistema Central, dividing the Iberian Plateau into a northern and a southern half and stretching into Portugal (where the highest point of Continental Portugal (1,993 m) is located in the Serra da Estrela). Pico Almanzor in the Sierra de Gredos is the highest point, at 2,592 m
The Montes de Toledo, which also stretches into Portugal from the La Mancha natural region at the eastern end. Its highest point, at 1,603 m, is La Villuerca in the Sierra de Villuercas, Extremadura
The Sierra Morena, which divides the watershed of the Guadiana and Guadalquivir rivers. At 1,332 m, Bañuela is the highest point
The Baetic System, which stretches between Cádiz and Gibraltar and northeast towards Alicante Province. It is divided into three subsystems:
Prebaetic System, which begins west of the Sierra Sur de Jaén, reaching the Mediterranean Sea shores in Alicante Province. La Sagra is the highest point at 2,382 m.
Subbaetic System, which is in a central position within the Baetic Systems, stretching from Cape Trafalgar in Cádiz Province across Andalusia to the Region of Murcia. The highest point, at 2,027 m (6,650 ft), is Peña de la Cruz in Sierra Arana.
Penibaetic System, located in the far southeastern area stretching between Gibraltar across the Mediterranean coastal Andalusian provinces. It includes the highest point in the peninsula, the 3,478 m high Mulhacén in the Sierra Nevada.
Answer: The highest point in the Iberian Peninsula is Mulhacen, located within the Penibaetic System. Its altitude is 3,478 m, or 11,411 ft
Question: Identify which car manufacturer is Italian or American: Lancia, Lincoln
Answer: Lancia is Italian, Lincoln is American
Question: Write a letter to your friend expressing your feelings about earning your first wage.
Answer: Dear Cooper, 
I hope you have been in good health since we last spoke. I'm excited to inform you that I've just finished my first month of employment with AMG Auto Motors and received my first pay check. The amazing feeling of being paid for my hard work is truly surreal. I feel fortunate to have found a job that pays well for doing something I genuinely enjoy. I hope to save enough money now to buy a car to plan a trip together.
Best regard, 
Simon.
Question: How do they obtain human generated policies?
Evidence: Specifying reward functions in reinforcement learning dialogue agents is often a difficult aspect. We propose to derive rewards from human-human dialogues by assigning positive values to contextualised responses seen in the data, and negative values to randomly chosen responses due to lacking coherence (also referred to as `non-human-like responses') – see example in Tables TABREF29 and TABREF30 . An episode or dialogue reward can thus be computed as INLINEFORM0 , where index INLINEFORM1 refers to the dialogue in focus, index INLINEFORM2 to the dialogue turn in focus, and INLINEFORM3 is given according to DISPLAYFORM0
We propose to derive rewards from human-human dialogues by assigning positive values to contextualised responses seen in the data, and negative values to randomly chosen responses due to lacking coherence (also referred to as `non-human-like responses') – see example in Tables TABREF29 and TABREF30 .
Answer: The answers are shown as follows:
* derive rewards from human-human dialogues by assigning positive values to contextualised responses seen in the data, and negative values to randomly chosen responses due to lacking coherence
Question: Based on this passage, please extract the names of the countries where TSMC has semiconductor fabrication (or "fab") facilities and separate them by commas.
Evidence: Apart from its main base of operations in Hsinchu in Northern Taiwan, where several of its fab facilities are located, it also has leading-edge fabs in Southern Taiwan and Central Taiwan, with other fabs located at its subsidiaries TSMC China in Shanghai, China, WaferTech in Washington state, United States, and SSMC in Singapore, and it has offices in China, Europe, India, Japan, North America, and South Korea.
Answer: Taiwan, China, United States, Singapore