DataHammer/scimrc huggingface dataset:
Question: What are the three separate multi-head attention mechanisms in the Transformer?
Evidence: In NMT, the Transformer BIBREF0 is a sequence-to-sequence (seq2seq) model which maps an input sequence to an output sequence through hierarchical multi-head attention mechanisms, yielding a dynamic, context-dependent strategy for propagating information within and across sentences. It contrasts with previous seq2seq models, which usually rely either on costly gated recurrent operations BIBREF15, BIBREF16 or static convolutions BIBREF17.
Given $n$ query contexts and $m$ sequence items under consideration, attention mechanisms compute, for each query, a weighted representation of the items. The particular attention mechanism used in BIBREF0 is called scaled dot-product attention, and it is computed in the following way:
where $\mathbf {Q} \in \mathbb {R}^{n \times d}$ contains representations of the queries, $\mathbf {K}, \mathbf {V} \in \mathbb {R}^{m \times d}$ are the keys and values of the items attended over, and $d$ is the dimensionality of these representations. The $\mathbf {\pi }$ mapping normalizes row-wise using softmax, $\mathbf {\pi }(\mathbf {Z})_{ij} = \operatornamewithlimits{\mathsf {softmax}}(\mathbf {z}_i)_j$, where
In words, the keys are used to compute a relevance score between each item and query. Then, normalized attention weights are computed using softmax, and these are used to weight the values of each item at each query context.
However, for complex tasks, different parts of a sequence may be relevant in different ways, motivating multi-head attention in Transformers. This is simply the application of Equation DISPLAY_FORM7 in parallel $H$ times, each with a different, learned linear transformation that allows specialization:
In the Transformer, there are three separate multi-head attention mechanisms for distinct purposes:
Encoder self-attention: builds rich, layered representations of each input word, by attending on the entire input sentence.
Context attention: selects a representative weighted average of the encodings of the input words, at each time step of the decoder.
Decoder self-attention: attends over the partial output sentence fragment produced so far.
Together, these mechanisms enable the contextualized flow of information between the input sentence and the sequential decoder.
In the Transformer, there are three separate multihead attention mechanisms for distinct purposes: • Encoder self-attention: builds rich, layered representations of each input word, by attending on the entire input sentence. • Context attention: selects a representative weighted average of the encodings of the input words, at each time step of the decoder. • Decoder self-attention: attends over the partial output sentence fragment produced so far.
Answer: Encoder self-attention, Context attention, and Decoder self-attention.
Question: What is the difference between a goose and a geese?
Answer: A goose and a geese refer to the same type of bird. These bird(s) are of the Anatidae biological family of water birds. Anatidate includes ducks, geese, seabirds, storks, and waterfowls. Goose refers to a single bird where geese refers to multiple birds.
Question: Which of the following two-dimensional shapes are made using only curved lines? And which use only straight lines? oval, rectangle, square, pentagon, circle and rhombus.
Answer: The oval and circle are made using only curved lines.
The square, pentagon, rectangle and rhombus are made using only straight lines.
Question: Identify which instrument is string or percussion: Thappu, Tati
Answer: Tati is string, Thappu is percussion.
Question: What are the largest hotel chains in the United States?
Answer: Marriott
Radisson
Best Western
Hilton
Hyatt
Question: Give me a bulleted list of major soccer leagues in Europe.
Answer: * English Premier League (EPL): This league is the home of some of the famous foot ball clubs like Manchester United, Liverpool FC, Chelsea, Arsenal and many more.
* German Football League (Bundesliga): Some of the major teams in this league are Bayern Munich, and Borussia Dortmund.
* Spanish Football League (La Liga): It is the host of the world two most famous football clubs i.e. FC Barcelona and Real Madrid
* Football Premier League (Ligue 1): In recent years, this is also gaining popularity thanks to the club, Paris Saint Germain, that have some world class footballers.
* Italian League (Serie A): Another premier league that is home of some of the oldest clubs like Juventus Turin, Napoli, AC Milan, Inter Milan, and many more.
Question: What is the objective of this paper?
Evidence: Student reflections are comments provided by students in response to a set of instructor prompts. The prompts are directed towards gathering students' feedback on course material. Student reflections are collected directly following each of a set of classroom lectures over a semester. In this paper, the set of reflections for each prompt in each lecture is considered a student reflection document. The objective of our work is to provide a comprehensive and meaningful abstractive summary of each student reflection document. Our dataset consists of documents and summaries from four course instantiations: ENGR (Introduction to Materials Science and Engineering), Stat2015 and Stat2016 (Statistics for Industrial Engineers, taught in 2015 and 2016, respectively), and CS (Data Structures in Computer Science). All reflections were collected in response to two pedagogically-motivated prompts BIBREF16: “Point of Interest (POI): Describe what you found most interesting in today's class” and “Muddiest Point (MP): Describe what was confusing or needed more detail.”
For each reflection document, at least one human (either a TA or domain expert) created summaries. Table TABREF4 shows example reference summary produced by one annotator for the CS course. Table TABREF5 summarizes the dataset in terms of number of lectures, number of prompts per lecture, average number of reflections per prompt, and number of abstractive reference summaries for each set of reflections.
The objective of our work is to provide a comprehensive and meaningful abstractive summary of each student reflection document.
Answer: The objective of this work is to provide a comprehensive and meaningful abstractive summary of each student reflection document.
Question: What are the tree target languages studied in the paper?
Evidence: To this goal, we select a diverse set of eight source languages from different language families – Basque, French, German, Hungarian, Italian, Navajo, Turkish, and Quechua – and three target languages – English, Spanish and Zulu. We pretrain a neural sequence-to-sequence architecture on each of the source languages and then fine-tune the resulting models on small datasets in each of the target languages. Analyzing the errors made by the systems, we find that (i) source and target language being closely related simplifies the successful learning of inflection in the target language, (ii) the task is harder to learn in a prefixing language if the source language is suffixing – as well as the other way around, and (iii) a source language which exhibits an agglutinative morphology simplifies learning of a second language's inflectional morphology.
To this goal, we select a diverse set of eight source languages from different language families – Basque, French, German, Hungarian, Italian, Navajo, Turkish, and Quechua – and three target languages – English, Spanish and Zulu. 
Answer: The answers are shown as follows:
* English, Spanish and Zulu
Question: Why is pickleball so popular?
Answer: Pickleball combines elements of both tennis and ping pong.   Tennis can take up to 2 years to learn the skills in order to play a proper tennis match while pickleball can be learned very quickly.   The motor skills involved with the fast play of ping pong are necessary with pickleball however there is more exercise involved with pickleball.   When you combine the easy to learn nature of pickleball with the fun exercise involved this is why pickleball is becoming the fastest growing sport in the United States.
Question: How is the proficiency score calculated?
Evidence: Tables and report some statistics extracted from both the written and spoken data collected so far in all the campaigns. Each written or spoken item received a total score by human experts, computed by summing up the scores related to 6 indicators in 2017/2018 (from 3 to 6 in the 2016 campaign, according to the proficiency levels and the type of test). Each indicator can assume a value 0, 1, 2, corresponding to bad, medium, good, respectively.
The list of the indicators used by the experts to score written sentences and spoken utterances in the evaluations, grouped by similarity, is reported in Table . Since every utterance was scored by only one expert, it was not possible to evaluate any kind of agreement among experts. For future evaluations, more experts are expected to provide independent scoring on same data sets, so this kind of evaluation will be possible.
FLOAT SELECTED: Table 4: List of the indicators used by human experts to evaluate specific linguistic competences.
Each written or spoken item received a total score by human experts, computed by summing up the scores related to 6 indicators in 2017/2018 (from 3 to 6 in the 2016 campaign, according to the proficiency levels and the type of test). Each indicator can assume a value 0, 1, 2, corresponding to bad, medium, good, respectively.
The list of the indicators used by the experts to score written sentences and spoken utterances in the evaluations, grouped by similarity, is reported in Table 
FLOAT SELECTED: Table 4: List of the indicators used by human experts to evaluate specific linguistic competences.
Answer: They used 6 indicators for proficiency (same for written and spoken) each marked by bad, medium or good by one expert.
Question: Which of these items are clothing brands and which are banks: Levi’s, Gap, Goldman Sachs,  Banana Republic, JP Morgan Chase, Seven, and Guess.
Answer: The banks are JP Morgan Chase and Goldman Sachs, and the clothing brands are Guess, Gap, Banana Republic, Levi’s, and Seven.
Question: What is the role inventory for each predicate divided into?
Evidence: We use the Bayesian model of garg2012unsupervised as our base monolingual model. The semantic roles are predicate-specific. To model the role ordering and repetition preferences, the role inventory for each predicate is divided into Primary and Secondary roles as follows:

For example, the complete role sequence in a frame could be: INLINEFORM0 INLINEFORM1 , INLINEFORM2 , INLINEFORM3 , INLINEFORM4 , INLINEFORM5 , INLINEFORM6 , INLINEFORM7 , INLINEFORM8 INLINEFORM9 . The ordering is defined as the sequence of PRs, INLINEFORM10 INLINEFORM11 , INLINEFORM12 , INLINEFORM13 , INLINEFORM14 , INLINEFORM15 INLINEFORM16 . Each pair of consecutive PRs in an ordering is called an interval. Thus, INLINEFORM17 is an interval that contains two SRs, INLINEFORM18 and INLINEFORM19 . An interval could also be empty, for instance INLINEFORM20 contains no SRs. When we evaluate, these roles get mapped to gold roles. For instance, the PR INLINEFORM21 could get mapped to a core role like INLINEFORM22 , INLINEFORM23 , etc. or to a modifier role like INLINEFORM24 , INLINEFORM25 , etc. garg2012unsupervised reported that, in practice, PRs mostly get mapped to core roles and SRs to modifier roles, which conforms to the linguistic motivations for this distinction.
Figure FIGREF16 illustrates two copies of the monolingual model, on either side of the crosslingual latent variables. The generative process is as follows:

All the multinomial and binomial distributions have symmetric Dirichlet and beta priors respectively. Figure FIGREF7 gives the probability equations for the monolingual model. This formulation models the global role ordering and repetition preferences using PRs, and limited context for SRs using intervals. Ordering and repetition information was found to be helpful in supervised SRL as well BIBREF9 , BIBREF8 , BIBREF10 . More details, including the motivations behind this model, are in BIBREF3 .
To model the role ordering and repetition preferences, the role inventory for each predicate is divided into Primary and Secondary roles as follows
Answer: Primary and Secondary roles.
Question: What's the aim of introducing RUN?
Evidence: The task of interpreting and following natural language (NL) navigation instructions involves interleaving different signals, at the very least the linguistic utterance and the representation of the world. For example, in turn right on the first intersection, the instruction needs to be interpreted, and a specific object in the world (the intersection) needs to be located in order to execute the instruction. In NL navigation studies, the representation of the world may be provided via visual sensors BIBREF0, BIBREF1, BIBREF2, BIBREF3, BIBREF4 or as a symbolic world representation. This work focuses on navigation based on a symbolic world representation (referred to as a map).
Previous datasets for NL navigation based on a symbolic world representation, HCRC BIBREF5, BIBREF6, BIBREF7 and SAIL BIBREF8, BIBREF9, BIBREF10, BIBREF11, BIBREF12, BIBREF13, BIBREF14, BIBREF15 present relatively simple worlds, with a small fixed set of entities known to the navigator in advance. Such representations bypass the great complexity of real urban navigation, which consists of long paths and an abundance of previously unseen entities of different types.
In this work we introduce Realistic Urban Navigation (RUN), where we aim to interpret navigation instructions relative to a rich symbolic representation of the world, given by a real dense urban map. To address RUN, we designed and collected a new dataset based on OpenStreetMap, in which we align NL instructions to their corresponding routes. Using Amazon Mechanical Turk, we collected 2515 instructions over 3 regions of Manhattan, all specified (and verified) by (respective) sets of humans workers. This task raises several challenges. First of all, we assume a large world, providing long routes, vulnerable to error propagation; secondly, we assume a rich environment, with entities of various different types, most of which are unseen during training and are not known in advance; finally, we evaluate on the full route intended, rather than on last-position only.
We then propose a strong neural baseline for RUN where we augment a standard encoder-decoder architecture with an entity abstraction layer, attention over words and worlds, and a constantly updating world-state. Our experimental results and ablation study show that this architecture is indeed better-equipped to treat grounding in realistic urban settings than standard sequence-to-sequence architectures. Given this RUN benchmark, empirical results, and evaluation procedure, we hope to encourage further investigation into the topic of interpreting NL instructions in realistic and previously unseen urban domains.
[1]The task defined by BIBREF6 is of moving between entities only.
In this work we introduce Realistic Urban Navigation (RUN), where we aim to interpret navigation instructions relative to a rich symbolic representation of the world, given by a real dense urban map.
Answer: To interpret navigation instructions relative to a rich symbolic representation of the world, given by a real dense urban map.
Question: What is the sample size of people used to measure user satisfaction?
Evidence: From January 5, 2019 to March 5, 2019, we collected conversational data for Gunrock. During this time, no other code updates occurred. We analyzed conversations for Gunrock with at least 3 user turns to avoid conversations triggered by accident. Overall, this resulted in a total of 34,432 user conversations. Together, these users gave Gunrock an average rating of 3.65 (median: 4.0), which was elicited at the end of the conversation (“On a scale from 1 to 5 stars, how do you feel about talking to this socialbot again?"). Users engaged with Gunrock for an average of 20.92 overall turns (median 13.0), with an average of 6.98 words per utterance, and had an average conversation time of 7.33 minutes (median: 2.87 min.). We conducted three principal analyses: users' response depth (wordcount), backstory queries (backstorypersona), and interleaving of personal and factual responses (pets).
 Overall, this resulted in a total of 34,432 user conversations. Together, these users gave Gunrock an average rating of 3.65 (median: 4.0), which was elicited at the end of the conversation (“On a scale from 1 to 5 stars, how do you feel about talking to this socialbot again?").
Answer: The answers are shown as follows:
* 34,432 user conversations
Question: What based system does the author use?
Evidence: BioASQ is a biomedical document classification, document retrieval, and question answering competition, currently in its seventh year. We provide an overview of our submissions to semantic question answering task (7b, Phase B) of BioASQ 7 (except for 'ideal answer' test, in which we did not participate this year). In this task systems are provided with biomedical questions and are required to submit ideal and exact answers to those questions. We have used BioBERT BIBREF0 based system , see also Bidirectional Encoder Representations from Transformers(BERT) BIBREF1, and we fine tuned it for the biomedical question answering task. Our system scored near the top for factoid questions for all the batches of the challenge. More specifially, in the third test batch set, our system achieved highest ‘MRR’ score for Factoid Question Answering task. Also, for List-type question answering task our system achieved highest recall score in the fourth test batch set. Along with our detailed approach, we present the results for our submissions and also highlight identified downsides for our current approach and ways to improve them in our future experiments. In last test batch results we placed 4th for List-type questions and 3rd for Factoid-type questions.)
The QA task is organized in two phases. Phase A deals with retrieval of the relevant document, snippets, concepts, and RDF triples, and phase B deals with exact and ideal answer generations (which is a paragraph size summary of snippets). Exact answer generation is required for factoid, list, and yes/no type question.
BioASQ organizers provide the training and testing data. The training data consists of questions, gold standard documents, snippets, concepts, and ideal answers (which we did not use in this paper, but we used last year BIBREF2). The test data is split between phases A and B. The Phase A dataset consists of the questions, unique ids, question types. The Phase B dataset consists of the questions, golden standard documents, snippets, unique ids and question types. Exact answers for factoid type questions are evaluated using strict accuracy (the top answer), lenient accuracy (the top 5 answers), and MRR (Mean Reciprocal Rank) which takes into account the ranks of returned answers. Answers for the list type question are evaluated using precision, recall, and F-measure.
We have used BioBERT based system , see also Bidirectional Encoder Representations from Transformers(BERT), and we fine tuned it for the biomedical question answering task.
Answer: BioBERT based system
Question: Summarize the following Wikipedia entry in three sentences.
Evidence: Seinfeld (/ˈsaɪnfɛld/ SYNE-feld) is an American television sitcom created by Larry David and Jerry Seinfeld. It aired on NBC from July 5, 1989, to May 14, 1998, over nine seasons and 180 episodes. It stars Seinfeld as a fictionalized version of himself and focuses on his personal life with three of his friends: best friend George Costanza (Jason Alexander), former girlfriend Elaine Benes (Julia Louis-Dreyfus) and his neighbor from across the hall, Cosmo Kramer (Michael Richards). It is set mostly in an apartment building in Manhattan's Upper West Side in New York City. It has been described as "a show about nothing", often focusing on the minutiae of daily life. Interspersed in earlier episodes are moments of stand-up comedy from the fictional Jerry Seinfeld, frequently using the episode's events for material.

As a rising comedian in the late 1980s, Jerry Seinfeld was presented with an opportunity to create a show with NBC. He asked Larry David, a fellow comedian and friend, to help create a premise for a sitcom. The series was produced by West-Shapiro Productions and Castle Rock Entertainment and distributed by Columbia Pictures Television.[nb 1] It was largely written by David and Seinfeld, with script writers who included Larry Charles, Peter Mehlman, Gregg Kavet, Carol Leifer, David Mandel, Jeff Schaffer, Steve Koren, Jennifer Crittenden, Tom Gammill, Max Pross, Dan O'Keefe, Charlie Rubin, Marjorie Gross, Alec Berg, Elaine Pope and Spike Feresten. A favorite among critics, the series led the Nielsen ratings in Seasons 6 and 9 and finished among the top two (with NBC's ER) every year from 1994 to 1998. Only two other shows – I Love Lucy and The Andy Griffith Show – have finished their runs at the top of the ratings.

Seinfeld is widely regarded as one of the greatest and most influential sitcoms of all time. It has been ranked among television's best shows in publications such as Entertainment Weekly, Rolling Stone and TV Guide. Its most renowned episodes include "The Chinese Restaurant", "The Soup Nazi", "The Parking Garage", "The Marine Biologist" and "The Contest". In 2013, the Writers Guild of America voted it the No. 2 Best-Written TV Series of All Time (second to The Sopranos). E! named it the "Number 1 reason the '90s ruled", and quotes from numerous episodes have become catchphrases in popular culture.
Answer: Seinfeld was an American television sitcom created by Jerry Seinfeld and Larry David, which ran for 9 seasons on NBC from July 5, 1989 - May 14, 1998. Seinfeld is widely regarded as one of the greatest and most influential sitcoms of all times and is one of only two other shows -- I love Lucy an The Andy Griffith Show -- to have finished their runs at the top of the ratings. It has often been described as "a show about nothing" that focuses on teh minituae of daily life.
Question: What was the first Porsche 911 with AWD?
Evidence: The 993 Turbo coupé was introduced in 1995. It featured a new twin-turbocharged engine displacing 3.6 liters and generating a maximum power output of 300 kW (408 PS; 402 hp). Air-to-air intercoolers, electronic engine management, redesigned cylinder heads, and other modified engine internals completed the new engine. The 993 Turbo was the first 911 Turbo with all-wheel drive, taken from the 959 flagship model. The Turbo's bodywork differs from the Carrera by widened rear wheel arches (about 6 cm), redesigned front and rear bumper moldings, and a fixed "whale tail" rear wing housing the intercoolers. New 18-inch (460 mm) alloy wheels with hollow spokes were standard.
Answer: The 993 had a Turbo variant that started in 1995. It was the first 911 Turbo with AWD. This AWD system was taken from the 959 flagship model.