DataHammer/scimrc huggingface dataset:
Question: How do they correlate user backstory queries to user satisfaction?
Evidence: We assessed the degree of conversational depth by measuring users' mean word count. Prior work has found that an increase in word count has been linked to improved user engagement (e.g., in a social dialog system BIBREF13). For each user conversation, we extracted the overall rating, the number of turns of the interaction, and the user's per-utterance word count (averaged across all utterances). We modeled the relationship between word count and the two metrics of user engagement (overall rating, mean number of turns) in separate linear regressions.
Results showed that users who, on average, produced utterances with more words gave significantly higher ratings ($\beta $=0.01, SE=0.002, t=4.79, p$<$0.001)(see Figure 2) and engaged with Gunrock for significantly greater number of turns ($\beta $=1.85, SE=0.05, t=35.58, p$<$0.001) (see Figure 2). These results can be interpreted as evidence for Gunrock's ability to handle complex sentences, where users are not constrained to simple responses to be understood and feel engaged in the conversation – and evidence that individuals are more satisfied with the conversation when they take a more active role, rather than the system dominating the dialog. On the other hand, another interpretation is that users who are more talkative may enjoy talking to the bot in general, and thus give higher ratings in tandem with higher average word counts.
We modeled the relationship between word count and the two metrics of user engagement (overall rating, mean number of turns) in separate linear regressions.

Results showed that users who, on average, produced utterances with more words gave significantly higher ratings ($\beta $=0.01, SE=0.002, t=4.79, p$<$0.001)(see Figure 2) and engaged with Gunrock for significantly greater number of turns ($\beta $=1.85, SE=0.05, t=35.58, p$<$0.001) (see Figure 2). These results can be interpreted as evidence for Gunrock's ability to handle complex sentences, where users are not constrained to simple responses to be understood and feel engaged in the conversation – and evidence that individuals are more satisfied with the conversation when they take a more active role, rather than the system dominating the dialog. On the other hand, another interpretation is that users who are more talkative may enjoy talking to the bot in general, and thus give higher ratings in tandem with higher average word counts.
Answer: The answers are shown as follows:
* modeled the relationship between word count and the two metrics of user engagement (overall rating, mean number of turns) in separate linear regressions
Question: Tell me which of these US Presidents was Republican or Democrat: Dwight Eisenhower, John Kennedy, Lyndon Johnson, Richard Nixon, Gerald Ford, Jimmy Carter
Answer: Dwight Eisenhower - Republican
John Kennedy - Democrat
Lyndon Johnson - Democrat
Richard Nixon - Republican
Gerald Ford - Republican
Jimmy Carter - Democrat
Question: Which Italian tractor maker tried making cars in 1960s
Answer: Ferruchio Lamborghini
Question: What is it like being alone in the world after an apocolypse?
Answer: I had always imagined the world would be in chaos or destruction. However, it's the same, just that there are no people. It's eerily quiet. Books are now the only source of "human connection" I have. I've never been much of a reader, but I anticipate much of my "free time" will be spent reading. I'm also quickly learning how much I relied on others. I knew I relied on others to do many things like farming, electricity, etc., but it goes much deeper and further than that.
Question: How many sentences does the dataset use in the emotion feature extraction model?
Evidence: We use the CNN structure as described in Section SECREF5 for emotional feature extraction. As a dataset for extracting emotion-related features, we use the corpus developed by BIBREF35 . This dataset consists of blog posts labeled by their corresponding emotion categories. As emotion taxonomy, the authors used six basic emotions, i.e., Anger, Disgust, Surprise, Sadness, Joy and Fear. In particular, the blog posts were split into sentences and each sentence was labeled. The dataset contains 5,205 sentences labeled by one of the emotion labels. After employing this model on the sarcasm dataset, we obtained a 150-dimensional feature vector from the fully-connected layer. As the aim of training is to classify each sentence into one of the six emotion classes, we used six neurons in the softmax layer.
The dataset contains 5,205 sentences labeled by one of the emotion labels.
Answer: 5,205 sentences.
Question: What conclusions are drawn from these experiments?
Evidence: The analysis of results from Tables TABREF17 , TABREF18 and TABREF19 show that 12 of 15 best results were obtained using new word embeddings. The evaluation results presented in Table TABREF20 (the chosen best embeddings models from Table TABREF19 ) prove that the best group of word embeddings is EC. The highest type F1-score was obtained for EC1 model, built using binary FastText Skip-gram method utilising subword information, with vector dimension equal to 300 and negative sampling equal to 10. The ability of the model to provide vector representation for the unknown words seems to be the most important. Also, previous models built using KGR10 (EP) are probably less accurate due to an incorrect tokenisation of the corpus. We used WCRFT tagger BIBREF22 , which utilises Toki BIBREF21 to tokenise the input text before the creation of the embeddings model. The comparison of EC1 with previous results obtained using only CRF BIBREF38 show the significant improvement across all the tested metrics: 3.6pp increase in strict F1-score, 1.36pp increase in relaxed precision, 5.61pp increase in relaxed recall and 3.51pp increase in relaxed F1-score.
The analysis of results from Tables TABREF17 , TABREF18 and TABREF19 show that 12 of 15 best results were obtained using new word embeddings. The evaluation results presented in Table TABREF20 (the chosen best embeddings models from Table TABREF19 ) prove that the best group of word embeddings is EC. The highest type F1-score was obtained for EC1 model, built using binary FastText Skip-gram method utilising subword information, with vector dimension equal to 300 and negative sampling equal to 10. The ability of the model to provide vector representation for the unknown words seems to be the most important.
Answer: The answers are shown as follows:
* best results were obtained using new word embeddings
* best group of word embeddings is EC
* The highest type F1-score was obtained for EC1 model, built using binary FastText Skip-gram method utilising subword information
* ability of the model to provide vector representation for the unknown words seems to be the most important
Question: What models are included in baseline benchmarking results?
Evidence: FLOAT SELECTED: Table 1: Pretraining costs of baseline models. Hardware and pretraining time are collected from original papers, with which costs are estimated with current TPU price at $8 per hour with 4 core TPU v3 chips and V100 GPU at $3.06 per hour. DistilBERT model is trained upon a pretrained BERT model. Parameter numbers are estimated using the pretrained models implemented in the Transformers (https://github.com/huggingface/ transformers) library (Wolf et al., 2019), shown in million.
FLOAT SELECTED: Table 1: Pretraining costs of baseline models. Hardware and pretraining time are collected from original papers, with which costs are estimated with current TPU price at $8 per hour with 4 core TPU v3 chips and V100 GPU at $3.06 per hour. DistilBERT model is trained upon a pretrained BERT model. Parameter numbers are estimated using the pretrained models implemented in the Transformers (https://github.com/huggingface/ transformers) library (Wolf et al., 2019), shown in million.
Answer: BERT, XLNET RoBERTa, ALBERT, DistilBERT
Question: Name some famous basketball players from the 1960's and 1970's
Answer: Oscar Robertson, George Mikan, John Havlicek, Bill Russell, Wilt Chamberlain, and Bill Bradley are all examples of outstanding basketball players from the early days of the National Basketball Association.
Question: When is an object considered a planet?
Answer: To be considered a planet the object must have an orbit around a sun, have enough mass to form a mostly round shape, and must be the dominant gravitational force in the area with no other similarly sized objects that are not orbiting the object in question.
Question: How do you make Crawfish bread?
Answer: Ingredients
1 pound crawfish tail meat, peeled, cleaned and deveined
1/2 stick butter
1 clove garlic, minced
3/4 cup chopped and drained ripe tomatoes
1 tablespoon Cajun seasoning
3/4 cup Monterey Jack Cheese
1/4 cup grated Parmesan
1/2 cup mayonnaise
3 tablespoons chopped fresh parsley
1 loaf French bread

Directions :
1. Preheat the oven to 350 degrees F. 
2. Rinse the crawfish. In a large skillet, melt the butter over medium heat. Add the garlic and cook for a few minutes. Add the tomatoes, Cajun seasoning and crawfish. Saute for 1 minute. Remove from the heat. 
3. In a bowl, mix together the cheeses, mayonnaise and parsley. Add the tomato and crawfish mixture to the mayo mixture. 
4. Cut the loaf of bread in half and then cut the bread lengthwise. Spoon the mixture on the bread and bake for 10 to 15 minutes. Then broil for an additional 3 to 5 minutes. Watch closely, as not to burn. 
5. Cut the bread into individual portions and serve.
Question: What are some of the languages spoken in Spain?
Answer: Most languages spoken in Spain belong to the Romance language family, of which Spanish is the only language which has the "official" status for the whole country. There are other languages that have recognized status in specific territories such as: Basque in the Basque Country, Catalan spoken in Catalonia, Valencian in the Valencian community and Gallician the adjancent western parts of the principality of Asturias.
Question: What is the architecture of the neural network?
Evidence: We describe the RelNet model in this section. Figure 1 provides a high-level view of the model. The model is sequential in nature, consisting of the following steps: read text, process it into a dynamic relational memory and then attention conditioned on the question generates the answer. We model the dynamic memory in a fashion similar to Recurrent Entity Networks BIBREF17 and then equip it with an additional relational memory.
The model is sequential in nature, consisting of the following steps: read text, process it into a dynamic relational memory and then attention conditioned on the question generates the answer. We model the dynamic memory in a fashion similar to Recurrent Entity Networks BIBREF17 and then equip it with an additional relational memory.
Answer: The answers are shown as follows:
* extends memory-augmented neural networks with a relational memory to reason about relationships between multiple entities present within the text. 
* The model is sequential in nature, consisting of the following steps: read text, process it into a dynamic relational memory and then attention conditioned on the question generates the answer. We model the dynamic memory in a fashion similar to Recurrent Entity Networks BIBREF17 and then equip it with an additional relational memory.
Question: What is an LCR circuit?
Answer: An LCR circuit is a resonant circuit that is also often called an RLC circuit. It consist of an inductor (L), a capacitor (C), and a resistor (R). The three components may be connected in series or in parallel.

Radio and television receivers contain oscillators that are based on the LCR circuit. The circuit is used to tune into transmitting stations.
Question: What are questions you can ask to get to know someone?
Answer: Questions you can ask to get to know someone are the following:  

1. What is your name?
2. What is your age?
3. What are your hobbies?
4. Did you go to college?  What college?
5. What do you do for work?
6. Do you have any brothers or sister?
7. What is most important to you in a relationship?
8. Do you like to read?  What was the last book you read?
9. Have you ever done any traveling?  Where?
10. What is your favorite restaurant?  Where?
Question: In which aspect do generative text models perform better?
Evidence: Language models can be optimized to recognize syntax and semantics with great accuracy BIBREF0. However, the output generated can be repetitive and generic leading to monotonous or uninteresting responses (e.g “I don't know”) regardless of the input BIBREF1. While application of attention BIBREF2, BIBREF3 and advanced decoding mechanisms like beam search and variation sampling BIBREF4 have shown improvements, it does not solve the underlying problem. In creative text generation, the objective is not strongly bound to the ground truth—instead the objective is to generate diverse, unique or original samples. We attempt to do this through a discriminator which can give feedback to the generative model through a cost function that encourages sampling of creative tokens. The contributions of this paper are in the usage of a GAN framework to generate creative pieces of writing. Our experiments suggest that generative text models, while very good at encapsulating semantic, syntactic and domain information, perform better with external feedback from a discriminator for fine-tuning objectiveless decoding tasks like that of creative text. We show this by evaluating our model on three very different creative datasets containing poetry, metaphors and lyrics.
Previous work on handling the shortcomings of MLE include length-normalizing sentence probability BIBREF5, future cost estimation BIBREF6, diversity-boosting objective function BIBREF7, BIBREF1 or penalizing repeating tokens BIBREF8. When it comes to poetry generation using generative text models, Zhang and Lapata BIBREF9, Yi et al. BIBREF10 and Wang et al. BIBREF11 use language modeling to generate Chinese poems. However, none of these methods provide feedback on the quality of the generated sample and hence, do not address the qualitative objective required for creative decoding. For the task of text generation, MaskGAN BIBREF12 uses a Reinforcement Learning signal from the discriminator, FMD-GAN BIBREF13 uses an optimal transport mechanism as an objective function. GumbelGAN BIBREF14 uses Gumbel-Softmax distribution that replaces the non-differentiable sample from a categorical distribution with a differentiable sample to propagate stronger gradients. Li et al. BIBREF1 use a discriminator for a diversity promoting objective. Yu et al. BIBREF15 use SeqGAN to generate poetry and comment on the performance of SeqGAN over MLE in human evaluations, encouraging our study of GANs for creative text generation. However, these studies do not focus solely on creative text.
Our experiments suggest that generative text models, while very good at encapsulating semantic, syntactic and domain information, perform better with external feedback from a discriminator for fine-tuning objectiveless decoding tasks like that of creative text.
Answer: They perform better with external feedback from a discriminator for fine-tuning objectiveless decoding tasks like that of creative text.
Question: From this passage, extract the treatments for ACO in a bulleted list.
Evidence: Treatment of ACO is based on expert opinion as there are no universally accepted clinical guidelines. Treatment is usually based on whether clinical features of asthma or COPD predominate. Inhaled corticosteroids are the primary treatment in those with ACOS. Inhaled corticosteroids (ICS) should be continued in those with asthma who develop decreased airway responsiveness to bronchodilators consistent with ACO. Therapy can be escalated to include a long acting beta-agonist (LABA) and inhaled steroid combination (ICS-LABA) or by adding on a long-acting anti-muscarinic inhaler (LAMA), known as triple therapy, in those with more severe or resistant disease.

Monoclonal antibodies targeting type 2 inflammation (which is predominant in asthma) have been used to treat severe asthma, and may also be used in severe cases of ACO. These monoclonal antibodies include omalizumab (an Anti-IgE antibody), mepolizumab (an anti-IL-5 antibody) and benralizumab (an anti-IL-5 receptor α antibody). People with ACOS and eosinophilia have a better response to ICS; with fewer exacerbations and hospitalizations seen in ACOS treated with long term ICS. Systemic corticosteroids (intravenous or oral steroids) may be used during exacerbations of ACOS.
https://en.wikipedia.org/wiki/Asthma-COPD_overlap
Answer: -Inhaled corticosteroids
-long acting beta-agonist (LABA) and inhaled steroid combination (ICS-LABA)
-triple therapy (in other words, adding on a long-acting anti-muscarinic inhaler (LAMA)
-monoclonal antibodies targeting type 2 inflammation
-systemic corticosteroids (intravenous or oral steroids)
Question: How is the best way of collecting questions decided?
Evidence: As a starting point for our pilots, we made use of texts from the InScript corpus BIBREF10 , which provides stories centered around everyday situations (see Section SECREF7 ). We conducted three different pilot studies to determine the best way of collecting questions that require inference over commonsense knowledge:
The most intuitive way of collecting reading comprehension questions is to show texts to workers and let them formulate questions and answers on the texts, which is what we tried internally in a first pilot. Since our focus is to provide an evaluation framework for inference over commonsense knowledge, we manually assessed the number of questions that indeed require common sense knowledge. We found too many questions and answers collected in this manner to be lexically close to the text.
In a second pilot, we investigated the option to take the questions collected for one text and show them as questions for another text of the same scenario. While this method resulted in a larger number of questions that required inference, we found the majority of questions to not make sense at all when paired with another text. Many questions were specific to a text (and not to a scenario), requiring details that could not be answered from other texts. Since the two previous pilot setups resulted in questions that centered around the texts themselves, we decided for a third pilot to not show workers any specific texts at all. Instead, we asked for questions that centered around a specific script scenario (e.g. eating in a restaurant). We found this mode of collection to result in questions that have the right level of specificity for our purposes: namely, questions that are related to a scenario and that can be answered from different texts (about that scenario), but for which a text does not need to provide the answer explicitly.
The next section will describe the mode of collection chosen for the final dataset, based on the third pilot, in more detail.
As a starting point for our pilots, we made use of texts from the InScript corpus (Modi et al., 2016), which provides stories centered around everyday situations (see Section 2.2.2.). We conducted three different pilot studies to determine the best way of collecting questions that require inference over commonsense knowledge:
Answer: They conducted three different pilot studies to determine the best way of collecting questions that require inference over commonsense knowledge.
Question: Why do I have blonde hair?
Answer: You have blonde hair because of your parent's genes.  If your mom and dad both have blonde hair your hair is blonde through a dominant gene.  If either your mom or your dad has brown or any other color hair then you got blonde hair through a recessive gene.  However, it would be more common for you to have brown hair if both your parents had brown hair.
Question: Why Italy is known for
Answer: Italy is known for island vacation. There are so many island (amalfi, Capri etc ) and their food and wine
Question: Write a short story about a woman named Mary who is a barista making a latte.
Answer: Mary was a barista at the local coffee shop. She has worked at this specific cafe for over 5 years and has learned to love the job, and has grown close to her coworkers and regular customers. While Mary is a specialist in preparing many types of coffee and tea drinks, her favorite of all time was to make the latte. One morning around 8:00 am, a regular customer named Eric came into the cafe. Eric usually orders a latte every morning around the same time and Mary really enjoys preparing his drink. When Mary saw Eric walk in, she started preparing his drink without even asking what he would order. First Mary greeted Eric and engaged in some small talk, conversing about the seasonally warm weather we were having. While simultaneously having this conversation, Mary was in process of making the coffee drink. Mary first ground 18g of espresso roast coffee on a fine grind setting. Once the coffee was ground, she filled the portafilter with the ground coffee. Once the portafilter was full of coffee, Mary tamped the coffee with a tamper, using about 30 pounds of force. When the portafilter was ready, she fixed it upon the espresso machine. Next Mary filled a milk pitcher with 12 ounces of whole milk. She then took the pitcher, held it to the steam wand, and started steaming the milk. For the first 5 seconds, she held the milk just below the surface of the milk to aerate the milk, and then moved the pitcher up so the steam wand was fully emerged. While the milk was steaming, Mary pressed the espresso button on the machine to start the pouring of the espresso into the two shot glasses. Mary was an expert in doing multiple latte making tasks simultaneously. As the milk finished steaming, she removed the pitcher, wiped down the steam wand, and set the milk aside. At that time, the espresso shots were done, and Mary poured the espresso from the shot glasses into the 16 oz paper cup. She then took the steamed milk and poured it over the espresso shots in the cup. She poured the milk in a specific way that would create latte art, this time she made a little heart out of the milk and foam. Once all the milk was poured, she put a cup on the coffee and handed the coffee over to Eric, who she was still having a conversation with. Once he received the coffee, Eric scanned his gift card on the computer which Mary had prepared for payment for the latte. After Eric was done paying, he took a sip of the drink and complimented Mary for another excellently prepared latte. After taking his sip, Eric then left the store, and got into this car, and drove away. It was moments like this every morning that reminded Mary why she loves this job so much, and looked forward to helping more customers for the remainder of her shift that morning.
Question: Given this text about gradient descent, what is gradient descent, and who invented it?
Evidence: In mathematics, gradient descent (also often called steepest descent) is a first-order iterative optimization algorithm for finding a local minimum of a differentiable function. The idea is to take repeated steps in the opposite direction of the gradient (or approximate gradient) of the function at the current point, because this is the direction of steepest descent. Conversely, stepping in the direction of the gradient will lead to a local maximum of that function; the procedure is then known as gradient ascent.

Gradient descent is generally attributed to Augustin-Louis Cauchy, who first suggested it in 1847. Jacques Hadamard independently proposed a similar method in 1907. Its convergence properties for non-linear optimization problems were first studied by Haskell Curry in 1944, with the method becoming increasingly well-studied and used in the following decades.
Answer: In mathematics,  gradient descent (also often called steepest descent) is a first-order iterative optimization algorithm for finding a local minimum of a differentiable function. Gradient descent is generally attributed to Augustin-Louis Cauchy, who first suggested it in 1847