DataHammer/scimrc huggingface dataset:
Question: What are the two extensions developed based on BERT by the team?
Evidence: Bidirectional Encoder Representations from Transformers (BERT) is a novel Transformer BIBREF0 model, which recently achieved state-of-the-art performance in several language understanding tasks, such as question answering, natural language inference, semantic similarity, sentiment analysis, and others BIBREF1. While well-suited to dealing with relatively short sequences, Transformers suffer from a major issue that hinders their applicability in classification of long sequences, i.e. they are able to consume only a limited context of symbols as their input BIBREF2.
There are several natural language (NLP) processing tasks that involve such long sequences. Of particular interest are topic identification of spoken conversations BIBREF3, BIBREF4, BIBREF5 and call center customer satisfaction prediction BIBREF6, BIBREF7, BIBREF8, BIBREF9. Call center conversations, while usually quite short and to the point, often involve agents trying to solve very complex issues that the customers experience, resulting in some calls taking even an hour or more. For speech analytics purposes, these calls are typically transcribed using an automatic speech recognition (ASR) system, and processed in textual representations further down the NLP pipeline. These transcripts sometimes exceed the length of 5000 words. Furthermore, temporal information might play an important role in tasks like CSAT. For example, a customer may be angry at the beginning of the call, but after her issue is resolved, she would be very satisfied with the way it was handled. Therefore, simple bag of words models, or any model that does not include temporal dependencies between the inputs, may not be well-suited to handle this category of tasks. This motivates us to employ model such as BERT in this task.
In this paper, we propose a method that builds upon BERT's architecture. We split the input text sequence into shorter segments in order to obtain a representation for each of them using BERT. Then, we use either a recurrent LSTM BIBREF10 network, or another Transformer, to perform the actual classification. We call these techniques Recurrence over BERT (RoBERT) and Transformer over BERT (ToBERT). Given that these models introduce a hierarchy of representations (segment-wise and document-wise), we refer to them as Hierarchical Transformers. To the best of our knowledge, no attempt has been done before to use the Transformer architecture for classification of such long sequences.
Our novel contributions are:
Two extensions - RoBERT and ToBERT - to the BERT model, which enable its application in classification of long texts by performing segmentation and using another layer on top of the segment representations.
State-of-the-art results on the Fisher topic classification task.
Significant improvement on the CSAT prediction task over the MS-CNN model.
Two extensions - RoBERT and ToBERT - to the BERT model
Answer: RoBERT and ToBERT.
Question: Identify which car manufacturer is Chinese or American: Apex Motors, Chevrolet
Answer: Apex Motors is Chinese, Chevrolet is American
Question: Give me a bulleted list of the current members of Pearl Jam.
Answer: - Jeff Ament
- Stone Gossard
- Mike McCready
- Eddie Vedder
- Matt Cameron
Question: Is the filter based feature selection (FSE) a form of regularization?
Evidence: Feature selection is considered an indispensable task in text classification as it removes redundant and irrelevant features of the corpus BIBREF18. Broadly, feature selection approaches can be divided into three classes namely wrapper, embedded, and filter BIBREF7, BIBREF8. In recent years, researchers have proposed various filter based feature selection methods to raise the performance of document text classification BIBREF19.
To develop the vocabulary of most discriminative features, we remove all punctuation symbols and non-significant words (stop words) as a part of the preprocessing step. Furthermore, in order to rank the terms based on their discriminative power among the classes, we use filter based feature selection method named as Normalized Difference Measure (NDM)BIBREF5. Considering the features contour plot, Rehman et al. BIBREF5 suggested that all those features which exist in top left, and bottom right corners of the contour are extremely significant as compared to those features which exist around diagonals. State-of-the-art filter based feature selection algorithms such as ACC2 treat all those features in the same fashion which exist around the diagonals BIBREF5. For instance, ACC2 assigns same rank to those features which has equal difference ($|t_{pr} -f_{pr}|$) value but different $t_{pr}$ and $f_{pr}$ values. Whereas NDM normalizes the difference ($|t_{pr} -f_{pr}|$) with the minimum of $t_{pr}$ and $f_{pr}$ (min($t_{pr}$, $f_{pr}$)) and assign different rank to those terms which have same difference value. Normalized Difference Measure (NDM) considers those features highly significant which have the following properties:
 In recent years, researchers have proposed various filter based feature selection methods to raise the performance of document text classification BIBREF19.
 Furthermore, in order to rank the terms based on their discriminative power among the classes, we use filter based feature selection method named as Normalized Difference Measure (NDM)BIBREF5. Considering the features contour plot, Rehman et al. BIBREF5 suggested that all those features which exist in top left, and bottom right corners of the contour are extremely significant as compared to those features which exist around diagonals. State-of-the-art filter based feature selection algorithms such as ACC2 treat all those features in the same fashion which exist around the diagonals BIBREF5.
Answer: No.
Question: Does money make you happy?
Answer: The short answer is no. It makes it more convenient but does not make people inherently happier. Also, the more you try to acquire money and focus on it, the less happy you become because there is never enough. On average, people want about 15-20% more money than they currently have. Additionally, folks with less money but who live in close-knit communities are among the happiest and most satisfied with life.
Question: What is the difference between instant oatmeal and regular oatmeal?
Answer: Instant oatmeal is ready to eat immediately after sitting in hot water, whereas regular oatmeal has to be boiled in hot water for a longer period of time (the specific duration depends on the type of oats).
Question: Which evaluation metrics do they use for language modelling?
Evidence: The functional dissimilarity score was computed using sentences from the test set in CoNLL 2017 Universal Dependencies task BIBREF20 for the relevant languages with the provided UPOS sequences. Furthermore, none of the evaluated models, including the proposed method, were trained with CoNLL2017 data.
We computed the nearest neighbours experiment for all languages in the training data for the above models. The results are shown in Table TABREF27. The results show that general purpose language models do capture syntax information, which varies greatly across languages and models.
The functional dissimilarity score was computed using sentences from the test set in CoNLL 2017 Universal Dependencies task BIBREF20 for the relevant languages with the provided UPOS sequences.
We computed the nearest neighbours experiment for all languages in the training data for the above models.
Answer: The answers are shown as follows:
*  functional dissimilarity score
* nearest neighbours experiment
Question: What are some good Indian dishes?
Answer: There are many popular Indian dishes. Tandoori chicken, chicken tikka masala, and biryani are popular main courses. Samosas and naan bread are popular side dishes. Gulab jamun is a popular dessert.
Question: Who was the original singer of Static-X?
Answer: The original singer of metal band, Static-X, was Wayne Static.
Question: Classify each of these breads as either a dog or a cat bread: Siamese, Ragdoll, Yorkshire Terrier, Birman, Cocker Spaniel, Saint Bernard.
Answer: Siamese = Cat
Ragdoll = Cat
Yorkshire Terrier = Dog
Birman = Cat
Cocker Spaniel = Dog
Saint Bernard = Dog
Question: Extract the destination countries of the Madeira diaspora from the text. Separate them with a comma.
Evidence: Madeirans migrated to the United States, Venezuela, Brazil, Guyana, Saint Vincent and the Grenadines, South Africa and Trinidad and Tobago.
Answer: United States, Venezuela, Brazil, Guyana, Saint Vincent and the Grenadines, South Africa, Trinidad and Tobago
Question: What are some types of sorting algorithms?
Answer: 1. Bubble Sort
2. Quicksort
3. Insertion Sort
4. Merge Sort
5. Heap Sort
6. Radix Sort
Question: What is the dimension size set when investigating the effect of the choice of pre-trained embedding on performance while using their approach?
Evidence: In the above experiments we initialize the structured syntax components with the pre-trained Gaussian or discrete baseline, which is shown as a useful technique to help train our deep models. We further study the results with fully random initialization. In the POS tagging experiment, we report the results in Table TABREF48 . While the performance with 4 layers is comparable to the pre-trained Gaussian initialization, deeper projections (8 or 16 layers) result in a dramatic drop in performance. This suggests that the structured syntax model with very deep projections is difficult to train from scratch, and a simpler projection might be a good compromise in the random initialization setting.
Different from the Markov prior in POS tagging experiments, our parsing model seems to be quite sensitive to the initialization. For example, directed accuracy of our approach on sentences of length INLINEFORM0 is below 40.0 with random initialization. This is consistent with previous work that has noted the importance of careful initialization for DMV-based models such as the commonly used harmonic initializer BIBREF2 . However, it is not straightforward to apply the harmonic initializer for DMV directly in our model without using some kind of pre-training since we do not observe gold POS.
We investigate the effect of the choice of pre-trained embedding on performance while using our approach. To this end, we additionally include results using fastText embeddings BIBREF41 – which, in contrast with skip-gram embeddings, include character-level information. We set the context windows size to 1 and the dimension size to 100 as in the skip-gram training, while keeping other parameters set to their defaults. These results are summarized in Table TABREF50 and Table TABREF51 . While fastText embeddings lead to reduced performance with our model, our approach still yields an improvement over the Gaussian baseline with the new observed embeddings space.
We investigate the effect of the choice of pre-trained embedding on performance while using our approach. To this end, we additionally include results using fastText embeddings (Bojanowski et al., 2017) – which, in contrast with skip-gram embeddings, include character-level information. We set the context windows size to 1 and the dimension size to 100 as in the skip-gram training, while keeping other parameters set to their defaults.
Answer: 100.
Question: What is the state flower of California?
Answer: The state flower of California is the Golden Poppy
Question: Can SCRF be used to pretrain the model?
Evidence: One of the major drawbacks of SCRF models is their high computational cost. In our experiments, the CTC model is around 3–4 times faster than the SRNN model that uses the same RNN encoder. The joint model by multitask learning is slightly more expensive than the stand-alone SRNN model. To cut down the computational cost, we investigated if CTC can be used to pretrain the RNN encoder to speed up the training of the joint model. This is analogous to sequence training of HMM acoustic models, where the network is usually pretrained by the frame-level CE criterion. Figure 2 shows the convergence curves of the joint model with and without CTC pretraining, and we see pretraining indeed improves the convergence speed of the joint model.
One of the major drawbacks of SCRF models is their high computational cost. In our experiments, the CTC model is around 3–4 times faster than the SRNN model that uses the same RNN encoder.
To cut down the computational cost, we investigated if CTC can be used to pretrain the RNN encoder to speed up the training of the joint model.
Figure 2 shows the convergence curves of the joint model with and without CTC pretraining, and we see pretraining indeed improves the convergence speed of the joint model.
Answer: No.
Question: Is cricket a popular sports in USA?
Answer: Cricket is not a popular sports in USA. Although, it is getting popularity due to multiple reasons. First, the immigrants from Asia, Australia and UK  have taken initiatives to launch local leagues for their second generation living in USA. Second, the Microsoft has recently invested in an international level cricket ground in Redmond, Washington. The USA also has recently launched a state level cricket league which will represent some known legends of Cricket.
Question: What are 5 really good golf ball brands for amateur players
Answer: Vice, Bridgestone, Taylormade, Srixon, Callaway
Question: Do experiment results show consistent significant improvement of new approach over traditional CNN and RNN models?
Evidence: The experimental results are shown in Table TABREF35. As we mentioned before, all RNNs in these models are bi-directional, because we wonder if our bi-CRU could still give substantial improvements over bi-GRU which could capture both history and future information. As we can see that, all variants of our CRU model could give substantial improvements over the traditional GRU model, where a maximum gain of 2.7%, 1.0%, and 1.9% can be observed in three datasets, respectively. We also found that though we adopt a straightforward classification model, our CRU model could outperform the state-of-the-art systems by 0.6%, 0.7%, and 0.8% gains respectively, which demonstrate its effectiveness. By employing more sophisticated architecture or introducing task-specific features, we think there is still much room for further improvements, which is beyond the scope of this paper.
As we can see that, all variants of our CRU model could give substantial improvements over the traditional GRU model, where a maximum gain of 2.7%, 1.0%, and 1.9% can be observed in three datasets, respectively.
Answer: Yes.
Question: Identify which instrument is string or percussion: Hydraulophone, Mandriola
Answer: Mandriola is string, Hydraulophone is percussion.
Question: Which review dataset do they use?
Evidence: We evaluate our proposed model on the SNAP (Stanford Network Analysis Project) Amazon review datasets BIBREF8, which contain not only reviews and ratings, but also golden summaries. In scenarios where there is no user-written summary for a review, we use pointer-generator network BIBREF9 to generate abstractive summaries. Empirical results show that our model significantly outperforms all strong baselines, including joint modeling, separate encoder and joint encoder methods. In addition, our model achieves new state-of-the-art performance, attaining 2.1% (with generated summary) and 4.8% (with golden summary) absolutely improvements compared to the previous best method on SNAP Amazon review benchmark.
We empirically compare different methods using Amazon SNAP Review Dataset BIBREF20, which is a part of Stanford Network Analysis Project. The raw dataset consists of around 34 millions Amazon reviews in different domains, such as books, games, sports and movies. Each review mainly contains a product ID, a piece of user information, a plain text review, a review summary and an overall sentiment rating which ranges from 1 to 5. The statistics of our adopted dataset is shown in Table TABREF20. For fair comparison with previous work, we adopt the same partitions used by previous work BIBREF6, BIBREF7, which is, for each domain, the first 1000 samples are taken as the development set, the following 1000 samples as the test set, and the rest as the training set.
We evaluate our proposed model on the SNAP (Stanford Network Analysis Project) Amazon review datasets BIBREF8, which contain not only reviews and ratings, but also golden summaries.
We empirically compare different methods using Amazon SNAP Review Dataset BIBREF20, which is a part of Stanford Network Analysis Project.
For fair comparison with previous work, we adopt the same partitions used by previous work BIBREF6, BIBREF7, which is, for each domain, the first 1000 samples are taken as the development set, the following 1000 samples as the test set, and the rest as the training set.
Answer: The answers are shown as follows:
* SNAP (Stanford Network Analysis Project)
Question: How much does the author's F1 score improve compared to  Titov and Klementiev's?
Evidence: We use the same baseline as used by lang2011unsupervised which has been shown to be difficult to outperform. This baseline assigns a semantic role to a constituent based on its syntactic function, i.e. the dependency relation to its head. If there is a total of INLINEFORM0 clusters, INLINEFORM1 most frequent syntactic functions get a cluster each, and the rest are assigned to the INLINEFORM2 th cluster.
However, we get an F1 score of 85.1% for argument identification on CoNLL 2009 EN data as opposed to 80.7% reported by Titov and Klementiev (2012b).
Answer: 0.044.
Question: Who are the main songwriters for the Grateful Dead?
Answer: The main song writers for the Grateful Dead were Jerry Garcia, Bob Weir, Phil Lesh, Robert Hunter, and John Perry Barlow.  

Typically, Jerry Garcia and Robert Hunter worked on songs together, and Bob Weir and John Perry Barlow worked together.

A lot of song credits are credited as Garcia/Hunter, or Weir/Barlow.
Question: Who is Jon Snow's biological parents?
Answer: While the answer has yet to be officially acknowledged in the "A Song of Ice and Fire" series, the "Game of Thrones" television show, which is based on the book series, revealed that Jon Snow is the son of the late Rhaegar Targaryen and Lyanna Stark.
Question: Given this reference paragrah, who directed "The Man in the Saddle?"
Evidence: The Man in the Saddle (German: Der Mann im Sattel) is a 1945 German film directed by and starring Harry Piel.
Answer: Harry Piel
Question: Classify the names of the Presidents based on their country.
Droupadi Murmu, Ram Nath Kovind, A. P. J. Abdul Kalam, Ranil Wickremesinghe
Answer: India - Droupadi Murmu, Ram Nath Kovind, A. P. J. Abdul Kalam
Sri Lanka- Ranil Wickremesinghe
Question: Do the authors mention any possible confounds in their study?
Evidence: On Twitter we can see results that are consistent with the RCV results for the left-to-center political spectrum. The exception, which clearly stands out, is the right-wing groups ENL and EFDD that seem to be the most cohesive ones. This is the direct opposite of what was observed in the RCV data. We speculate that this phenomenon can be attributed to the fact that European right-wing groups, on a European but also on a national level, rely to a large degree on social media to spread their narratives critical of European integration. We observed the same phenomenon recently during the Brexit campaign BIBREF38 . Along our interpretation the Brexit was “won” to some extent due to these social media activities, which are practically non-existent among the pro-EU political groups. The fact that ENL and EFDD are the least cohesive groups in the European Parliament can be attributed to their political focus. It seems more important for the group to agree on its anti-EU stance and to call for independence and sovereignty, and much less important to agree on other issues put forward in the parliament.
On Twitter we can see results that are consistent with the RCV results for the left-to-center political spectrum. The exception, which clearly stands out, is the right-wing groups ENL and EFDD that seem to be the most cohesive ones. This is the direct opposite of what was observed in the RCV data. We speculate that this phenomenon can be attributed to the fact that European right-wing groups, on a European but also on a national level, rely to a large degree on social media to spread their narratives critical of European integration.
Answer: Yes.
Question: Who are the almoravids ?
Evidence: The Almoravid dynasty (Arabic: المرابطون, romanized: Al-Murābiṭūn, lit. 'those from the ribats') was an imperial Berber Muslim dynasty centered in the territory of present-day Morocco. It established an empire in the 11th century that stretched over the western Maghreb and Al-Andalus, starting in the 1050s and lasting until its fall to the Almohads in 1147. The Almoravid capital was Marrakesh, a city founded by the Almoravid leader Abu Bakr ibn Umar c. 1070. The dynasty emerged from a coalition of the Lamtuna, Gudala, and Massufa, nomadic Berber tribes living in what is now Mauritania and the Western Sahara, traversing the territory between the Draa, the Niger, and the Senegal rivers.

.
Answer: The Almoravid was imperial Berber Muslim Dynasty. They rulled in Morocco from 1050 until 1147 and Marrakech were their capital. They emerged from a coalition of the Lamtuna, Massufa and Gudala, nomad berbers l.
Question: How can I make every color of paint?
Answer: To make paint colors you need the three primary colors: red, yellow, and blue, as well as black and white to adjust shade (black mixed with color) and tint (white mixed with color). The primary colors red and yellow mix to make orange, red and blue mix to make purple, and yellow and blue mix to make green. Other common colors such as pink can be obtained by mixing black white to these base colors. For example, pink can be created by mixing a lot of white with a small amount of red.
Question: What is an emote?
Answer: An emote is an in-video game response of a playable character (usually in third person).  This is usually a dance move or pop culture reference.  Often times, emotes are a paid engagement via a micro-transaction.
Question: Why can we only see one side of the Moon?
Answer: On Earth we only see one side of the moon because the Moon and the Earth are tidally locked and the Moon is in a state of synchronous rotation. This means that the Moon's rotational period, the time it takes to make one full rotation, is the same as its orbital period, the time it takes to orbit the Earth. So as the moon orbits the Earth it is also rotating the exact amount to keep one side facing Earth at all times.
Question: Which of the following songs are by the Grateful Dead and which are by The Beatles: Don't Let Me Down, Sugar Magnolia, Hey Jude, Bertha, Throwing Stones, Let It Be, Truckin', Friend of the Devil, Uncle John's Band, and Here Comes the Sun.
Answer: The Grateful Dead songs on this list are Sugar Magnolia, Throwing Stones, Truckin', Friend of the Devil, and Uncle John's Band. The Beatles songs on this list are Don't Let Me Down, Hey Jude, Let It Be, and Here Comes the Sun.