DataHammer/scimrc huggingface dataset:
Question: What are the two existing metrics that are often used for the automatic evaluation of table-to-text generation?
Evidence: The task of generating natural language descriptions of structured data (such as tables) BIBREF2 , BIBREF3 , BIBREF4 has seen a growth in interest with the rise of sequence to sequence models that provide an easy way of encoding tables and generating text from them BIBREF0 , BIBREF1 , BIBREF5 , BIBREF6 .
For text generation tasks, the only gold standard metric is to show the output to humans for judging its quality, but this is too expensive to apply repeatedly anytime small modifications are made to a system. Hence, automatic metrics that compare the generated text to one or more reference texts are routinely used to compare models BIBREF7 . For table-to-text generation, automatic evaluation has largely relied on BLEU BIBREF8 and ROUGE BIBREF9 . The underlying assumption behind these metrics is that the reference text is gold-standard, i.e., it is the ideal target text that a system should generate. In practice, however, when datasets are collected automatically and heuristically, the reference texts are often not ideal. Figure FIGREF2 shows an example from the WikiBio dataset BIBREF0 . Here the reference contains extra information which no system can be expected to produce given only the associated table. We call such reference texts divergent from the table.
We show that existing automatic metrics, including BLEU, correlate poorly with human judgments when the evaluation sets contain divergent references (§ SECREF36 ). For many table-to-text generation tasks, the tables themselves are in a pseudo-natural language format (e.g., WikiBio, WebNLG BIBREF6 , and E2E-NLG BIBREF10 ). In such cases we propose to compare the generated text to the underlying table as well to improve evaluation. We develop a new metric, PARENT (Precision And Recall of Entailed N-grams from the Table) (§ SECREF3 ). When computing precision, PARENT effectively uses a union of the reference and the table, to reward correct information missing from the reference. When computing recall, it uses an intersection of the reference and the table, to ignore extra incorrect information in the reference. The union and intersection are computed with the help of an entailment model to decide if a text n-gram is entailed by the table. We show that this method is more effective than using the table as an additional reference. Our main contributions are:
For table-to-text generation, automatic evaluation has largely relied on BLEU (Papineni et al., 2002) and ROUGE (Lin, 2004).
Answer: BLEU and ROUGE.
Question: What are some indoor sports I can play in winter?
Answer: Here is a list of indoor sports:
- Table tennis
- Indoor basketball
- Futsal
- Trampolining
- Ice hockey
- Yoga
- Badminton
- Fencing
Question: Why do muscle twitch (or twinge)?
Answer: Muscle twitch could be related to too much stress, caffeine, poor diet or lack of exercise.
Question: Why is Grand Canyon a big deal?
Evidence: The Grand Canyon is a river valley in the Colorado Plateau that exposes uplifted Proterozoic and Paleozoic strata, and is also one of the six distinct physiographic sections of the Colorado Plateau province. Even though it is not the deepest canyon in the world (Kali Gandaki Gorge in Nepal is much deeper), the Grand Canyon is known for its visually overwhelming size and its intricate and colorful landscape. Geologically, it is significant because of the thick sequence of ancient rocks that are well preserved and exposed in the walls of the canyon. These rock layers record much of the early geologic history of the North American continent.
Answer: While Gran Canyon is not the deepest canyon on earth, it bears significant geological significance because of its exposed yet well-preserved rock layers. The rock layers record much of the early geologic history of the North American continent
Question: What are three downstream task datasets?
Evidence: To measure the effect of adding Bertram to BERT on downstream tasks, we apply the procedure described in Section SECREF4 to a commonly used textual entailment dataset as well as two text classification datasets: MNLI BIBREF21, AG's News BIBREF22 and DBPedia BIBREF23. For all three datasets, we use BERT$_\text{base}$ as a baseline model and create the substitution dictionary $S$ using the synonym relation of WordNet BIBREF20 and the pattern library BIBREF34 to make sure that all synonyms have consistent parts of speech. As an additional source of word substitutions, we make use of the misspellings dataset of BIBREF25, which is based on query logs of a search engine. To prevent misspellings from dominating the resulting dataset, we only assign misspelling-based substitutes to randomly selected 10% of the words contained in each sentence. Motivated by the results on WNLaMPro-medium, we consider every word that occurs less than 100 times in the WWC and our BooksCorpus replica combined as being rare. Some examples of entries in the resulting datasets can be seen in Table TABREF35.
To measure the effect of adding Bertram to BERT on downstream tasks, we apply the procedure described in Section SECREF4 to a commonly used textual entailment dataset as well as two text classification datasets: MNLI BIBREF21, AG's News BIBREF22 and DBPedia BIBREF23.
Answer: The answers are shown as follows:
* MNLI
* AG's News
* DBPedia
Question: Who is the greatest tennis player of all time?
Answer: This is the most debated question in tennis and it seems to have multiple answers depending on how greatness is defined. While the tennis history gave us many great players, most answers pick one of the so called big three players: Novak Djokovic, Rafael Nadal, or Roger Federer. 

If we look at the number of trophies and records, the most successful player is Novak Djokovic. The dominance achieved by Rafael Nadal on a single surface is unprecedented. Some people however argue that greatness cannot be measured in trophies and records and that Roger Federer’s game is the most elegant they’ve ever seen on court.
Question: From the passage provided, extract the name of the coach of Gujarat Giants
Evidence: Gujarat Giants (formerly known as Gujarat Fortunegiants) is a men's kabaddi team based in Ahmedabad, Gujarat that plays in the Pro Kabaddi League. The team is currently led by Chandran Ranjith and coached by Ram mehar Singh. The team is owned by Adani Wilmar Ltd. The Giants play their home matches at The Arena by TransStadia. They reached the final in both of their attempts in 2017 and 2018, finishing runner's up on both occasions to the Patna Pirates and Bengaluru Bulls respectively.
Answer: Gujarat Giants is coached by Ram mehar Singh
Question: In what way does the team bridge the gap that a semistructured table is not directly comparable to the unstructured generated or reference texts?
Evidence: PARENT evaluates each instance INLINEFORM0 separately, by computing the precision and recall of INLINEFORM1 against both INLINEFORM2 and INLINEFORM3 .
The table is in a semistructured form, and hence not directly comparable to the unstructured generated or reference texts. To bridge this gap, we introduce the notion of entailment probability, which we define as the probability that the presence of an n-gram g in a text is “correct” given the associated table.
Answer: To bridge this gap, they introduce the notion of entailment probability, which they define as the probability that the presence of an n-gram g in a text is “correct” given the associated table.
Question: Where is the Birthplace of 'Silicon Valley
Evidence: The original incarnation of the Hewlett-Packard Company, commonly shortened to Hewlett-Packard (/ˈhjuːlɪt ˈpækərd/ HYEW-lit PAK-ərd) or HP, was an American multinational information technology company headquartered in Palo Alto, California. HP developed and provided a wide variety of hardware components, as well as software and related services to consumers, small and medium-sized businesses (SMBs), and large enterprises, including customers in the government, health, and education sectors. The company was founded in a one-car garage in Palo Alto by Bill Hewlett and David Packard in 1939, and initially produced a line of electronic test and measurement equipment. The HP Garage at 367 Addison Avenue is now designated an official California Historical Landmark, and is marked with a plaque calling it the "Birthplace of 'Silicon Valley'".
Answer: The HP Garage at 367 Addison Avenue
Question: Who sang Big Big Deal
Evidence: "Big Big Deal" is a song by the British singer-songwriter Steve Harley, which was released as his debut, non-album solo single in 1974. The song, which was written and produced by Harley, would be his last release before scoring the UK number one hit "Make Me Smile (Come Up and See Me)" in 1975. "Big Big Deal" also preceded the formation of the second line-up of Harley's band Cockney Rebel.
Answer: Big Big Deal is a song by the British singer-songwriter Steve Harley, which was released as his debut, non-album solo single in 1974.
Question: Does the dataset only involve one topic?
Evidence: In recent years many datasets have been created for the task of automated stance detection, advancing natural language understanding systems for political science, opinion research and other application areas. Typically, such benchmarks BIBREF0 are composed of short pieces of text commenting on politicians or public issues and are manually annotated with their stance towards a target entity (e.g. Climate Change, or Trump). However, they are limited in scope on multiple levels BIBREF1.
First of all, it is questionable how well current stance detection methods perform in a cross-lingual setting, as the multilingual datasets available today are relatively small, and specific to a single target BIBREF2, BIBREF3. Furthermore, specific models tend to be developed for each single target or pair of targets BIBREF4. Concerns have been raised that cross-target performance is often considerably lower than fully supervised performance BIBREF1.
In this paper we propose a much larger dataset that combines multilinguality and a multitude of topics and targets. x-stance comprises more than 150 questions concerning Swiss politics and more than 67k answers given in the last decade by candidates running for political office in Switzerland.
Questions are available in four languages: English, Swiss Standard German, French, and Italian. The language of a comment depends on the candidate's region of origin.
We have extracted the data from the voting advice application Smartvote. On that platform, candidates respond to questions mainly in categorical form (yes / rather yes / rather no / no). They can also submit a free-text comment in order to justify, explain or differentiate their categorical answer. An example is given in Figure FIGREF1.
We transform the dataset into a stance detection task by interpreting the question as a natural-language representation of the target, and the commentary as the input to be classified.
The dataset is split into a multilingual training set and into multiple test sets to evaluate zero-shot cross-lingual and cross-target transfer. To provide a baseline, we fine-tune a multilingual Bert model BIBREF5 on x-stance. We show that the baseline accuracy is comparable to previous stance detection benchmarks while leaving ample room for improvement. In addition, multilingual Bert can generalize to a degree both cross-lingually and in a cross-target setting.
We have made the dataset and the code for reproducing the baseline model publicly available.
In this paper we propose a much larger dataset that combines multilinguality and a multitude of topics and targets.
Answer: No.
Question: What are the types of space objects ?
Answer: The International Astronomical Union (IAU) classifies objects in the Solar system into 3 categories :
1. Planet : a celestial body orbiting the Sun, with enough mass to assume a (nearly round) hydrostatic equilibrium shape, and that has "cleared the neighbourhood" around its orbit,
2. Dwarf planet : a celestial body orbiting the Sun, with enough mass to assume a (nearly round) hydrostatic equilibrium shape, but that has not "cleared the neighbourhood" around its orbit and is not a satellite,
3. Small Solar System Bodies : all other objects orbiting the sun that are not satellites.

Small Solar System Bodies of the Solar system include :
- Moons : celestial bodies that orbit a planet or a dwarf planet,
- Asteroids : rocky non-spherical left-over objects from the early formation of the Solar system. They travel around the sun and most of them are located within the main asteroid belt between Mars and Jupiter,
- Comets : icy bodies that can also contain minerals, rocks and dust left over from the formation of the solar system. They travel around the sun and can be found mostly beyond Neptune in the Kuiper Belt, as well as far beyond Pluto in the Oort cloud,
- Meteoroids : objects in space that often come from other larger bodies such as comets, asteroids or even moons and planets. Meteoroids can be rocky, others can be metallic, or combinations of both. Their sizes range from dust grains to small asteroids.
Question: What 6 language pairs is experimented on?
Evidence: Table TABREF13 shows the inconsistency rates of back translation between Adv-C and our method on MUSE. Compared with Adv-C, our model significantly reduces the inconsistency rates on all language pairs, which explains the overall improvement in Table TABREF12. Table TABREF14 gives several word translation examples. In the first three cases, our regularizer successfully fixes back translation errors. In the fourth case, ensuring cycle consistency does not lead to the correct translation, which explains some errors by our system. In the fifth case, our model finds a related word but not the same word in the back translation, due to the use of cosine similarity for regularization.
FLOAT SELECTED: Table 1: Accuracy on MUSE and Vecmap.
Compared with Adv-C, our model significantly reduces the inconsistency rates on all language pairs, which explains the overall improvement in Table TABREF12.
FLOAT SELECTED: Table 1: Accuracy on MUSE and Vecmap.
Answer: EN<->ES
EN<->DE
EN<->IT
EN<->EO
EN<->MS
EN<->FI
Question: Is Divine Child High School a private or public school?
Evidence: Divine Child High School, commonly known as Divine Child (DC), is a private, Roman Catholic, college-preparatory, parish high school in Dearborn, Michigan, United States. Divine Child is a highly ranked private high school in Michigan, scoring in the top 15 percent of private schools in the State. Notably, it is the seventh-largest private high school in Michigan and the largest co-educational Catholic high school in the State.
Answer: private school
Question: In the series A Song of Ice and Fire, who is the founder of House Casterly?
Answer: Corlos, son of Caster
Question: What is the motivation behind the work? Why question generation is an important task?
Evidence: Existing question generating systems reported in the literature involve human-generated templates, including cloze type BIBREF0, rule-based BIBREF1, BIBREF2, or semi-automatic questions BIBREF3, BIBREF4, BIBREF5. On the other hand, machine learned models developed recently have used recurrent neural networks (RNNs) to perform sequence transduction, i.e. sequence-to-sequence BIBREF6, BIBREF7. In this work, we investigated an automatic question generation system based on a machine learning model that uses transformers instead of RNNs BIBREF8, BIBREF9. Our goal was to generate questions without templates and with minimal human involvement using machine learning transformers that have been demonstrated to train faster and better than RNNs. Such a system would benefit educators by saving time to generate quizzes and tests.
Our goal was to generate questions without templates and with minimal human involvement using machine learning transformers that have been demonstrated to train faster and better than RNNs. Such a system would benefit educators by saving time to generate quizzes and tests.
Answer: The answers are shown as follows:
* Such a system would benefit educators by saving time to generate quizzes and tests.
Question: Tell me whether each of these positions is in soccer or baseball: shortstop, outfielder, midfielder, forward, goalie, catcher
Answer: Shortstop: Baseball
Outfielder: Baseball
Midfielder: Soccer
Forward: Soccer
Goalie: Soccer
Catcher: Baseball
Question: Brainstorm sea shanty lyrics
Answer: There once was a ship that went to sea, and they named her the O'Malley.  It bobbed in the waves and tipped you see!  The sea was rough and the weather went sour.  Then the crew heaved every hour! Oh no, down she goes!  Oh no, down she goes;  Down O'Malley into the sea!
Question: Given this paragraph about KITS, tell me what station it's on and where its radio tower is located.
Evidence: KITS (105.3 FM, "105.3 Dave FM") is a commercial radio station in San Francisco, California. Owned by Audacy, Inc., it broadcasts an adult hits radio format. The studios and offices are co-located with formerly co-owned KPIX-TV on Battery Street in the North Beach district of San Francisco.

KITS's transmitter is located on Radio Road, at San Bruno Mountain in Daly City. It has an effective radiated power (ERP) of 15,000 watts, from a tower at 366 meters (1,201 feet) in height above average terrain. KITS broadcasts in the HD Radio format. Its HD2 subchannel formerly aired a gold-based modern rock format as Classic Live 105, inspired by its previous long-time tenure in the format as Live 105. The HD3 subchannel formerly carried "Radio Zindagi," a South Asian radio service. These subchannels have since been turned off.
Answer: KITS plays on 105.3FM in San Francisco, California. Its radio tower is located on Radio Road at San Bruno Mountain in Daly City.
Question: What has been done to test whether horizontal or vertical consistency is more helpful?
Evidence: The default setup preserves horizontal language consistency in that the language of the questions always matches the language of the comments. For example, the Italian test instances are combined with the Italian version of the questions, even though during training the model has only ever seen the German and French versions of the questions.
An alternative concept is vertical language consistency, whereby the questions are consistently presented in one language, regardless of the comment. To test whether horizontal or vertical consistency is more helpful, we train and evaluate M-Bert on a dataset variant where all questions are in their English version. We chose English as a lingua franca because it had the largest share of data during the pretraining of M-Bert.
The results are shown in Table TABREF39. While the effect is negligible in most settings, the cross-lingual performance clearly increases when all questions are given in English.
To test whether horizontal or vertical consistency is more helpful, we train and evaluate M-BERT on a dataset variant where all questions are in their English version.
Answer: The authors train and evaluate M-BERT on a dataset variant where all questions are in their English version.
Question: Given this reference paragraph, who coined the term dead man zone?
Evidence: The term dead man zone was coined by members of the CSIRO research team in Australia who were investigating the spread of bushfires (Project Vesta). The dead man zone is approximately 8 kilometres (5.0 mi) around the fire that is likely to burn. This dead man zone can be a safety net for firefighters if the fire is moving at a high speed. The dead man zone can also be prepared in a way that slows down the fire. Firefighters can get outside the dead man zone safely without seeing what the fire is going to do and develop a plan of attack or defense upon the fire's next move. If the dead man zone is not used, wind change can sneak up on the firefighters and resulting in potentially unsafe distances where they are not able to defend themselves. This can be a life or death situation if not approached properly, and there have been examples of firefighters that became trapped and ultimately killed in Australia. Project Vesta, headed by scientist Phil Cheney, found that when the wind changes direction, the line of fire will move out at its maximum rate of spread almost immediately, and that the spread speed was nearly three times what was previously thought. Project Vesta's research into bushfire behavior makes up the majority of what is known about bushfires today.
Answer: The team was created by the Australian CSIRO research team
Question: Who is Richard Wayne Bogosian?
Evidence: Richard Wayne Bogosian (born July 18, 1937) is an American diplomat.
Answer: Richard Wayne Bogosian (born July 18, 1937) is an American diplomat. He was the United States Ambassador to Chad from 1990 to 1993 and Niger from 1985 to 1988.
He graduated from Tufts College with an A.B. in 1959 and the University of Chicago with a J.D. in 1962. He joined the U.S. Foreign Service in 1962. He oversaw many diplomatic posts, including at the Foreign Service Institute from 1962 to 1963, in the Bureau of Near Eastern and South Asian Affairs at the Department of State in 1963, at the U.S. Embassy in Baghdad, Iraq from 1963 to 1965, at the Foreign Service Institute again in 1965, as vice consul for the U.S. Embassy in Paris, France, from 1966 to 1968, in the Bureau of Near East and South Asian Affairs at the Department of State from 1968 to 1969, in the Bureau for Intelligence and Research at the Department of State from 1969 to 1971, and in the Foreign Service Institute for Economics in 1972.

He was chief of the economic section at the U.S. Embassy in Kuwait from 1972 to 1976, and Deputy Chief of Mission at the U.S. Embassy in Khartoum, Sudan, from 1976 to 1979. From 1976 to 1979, Bogosian was the chief of the Aviation Negotiations Division. He was the Director of East African Affairs from 1982 to 1985, and was nominated to be the United States Ambassador to Niger on August 1, 1985, by President Ronald Reagan.
From 1988 to 1990 he was the Director of the Office of Monetary Affairs.In 1990, Bogosian became the United States Ambassador to Chad from 1990 to 1993. From 1993 to 1994, he was the Coordinator at the U.S. Liaison Office in Mogadishu, Somalia, and Coordinator for Rwanda and Burundi from 1996 to 1997, after the Rwandan genocide.

Bogosian speaks Arabic and French. He is married to Claire Marie Bogosian, and has three children.
Question: What does “that’s sus” mean?
Answer: “Sus” is slang for “suspicious” or “suspect” and the phrase is used by young people to express skepticism, as in “that new kid who started a fight is sus.”
Question: What does the author's study do?
Evidence: In this paper, we showed how we can effectively leverage a prodigious blog dataset. Not only does the dataset bring out the extensive linguistic content reflected in the blog posts, but also includes location information and rich metadata. These data allow for the generation of maps that reflect the demographics of the population, variations in language use, and differences in psycholinguistic and semantic categories. These mappings can be valuable to both psychologists and linguists, as well as lexicographers. A prototype demo has been made available together with the code used to collect our dataset.
A prototype demo has been made available together with the code used to collect our dataset
Answer: A prototype demo has been made available together with the code used to collect their dataset
Question: What is considered as DACH region?
Answer: The DACH region comprises the three countries in the central Europe: Germany, Austria, and Switzerland. The abbreviation comes a mix of first letters of countries as per international country codes, i.e. Germany (DE), Austria (AT), and Switzerland (CH).