DataHammer/scimrc huggingface dataset:
Question: Do the LSTMs outperform than the more recent models?
Evidence: The scientific process by which the deep learning research community operates is guided by empirical studies that evaluate the relative quality of models. Complicating matters, the measured performance of a model depends not only on its architecture (and data), but it can strongly depend on hyperparameter values that affect learning, regularisation, and capacity. This hyperparameter dependence is an often inadequately controlled source of variation in experiments, which creates a risk that empirically unsound claims will be reported.
In this paper, we use a black-box hyperparameter optimisation technique to control for hyperparameter effects while comparing the relative performance of language modelling architectures based on LSTMs, Recurrent Highway Networks BIBREF0 and NAS BIBREF1 . We specify flexible, parameterised model families with the ability to adjust embedding and recurrent cell sizes for a given parameter budget and with fine grain control over regularisation and learning hyperparameters.
Once hyperparameters have been properly controlled for, we find that LSTMs outperform the more recent models, contra the published claims. Our result is therefore a demonstration that replication failures can happen due to poorly controlled hyperparameter variation, and this paper joins other recent papers in warning of the under-acknowledged existence of replication failure in deep learning BIBREF2 , BIBREF3 . However, we do show that careful controls are possible, albeit at considerable computational cost.
Several remarks can be made in light of these results. First, as (conditional) language models serve as the central building block of many tasks, including machine translation, there is little reason to expect that the problem of unreliable evaluation is unique to the tasks discussed here. However, in machine translation, carefully controlling for hyperparameter effects would be substantially more expensive because standard datasets are much larger. Second, the research community should strive for more consensus about appropriate experimental methodology that balances costs of careful experimentation with the risks associated with false claims. Finally, more attention should be paid to hyperparameter sensitivity. Models that introduce many new hyperparameters or which perform well only in narrow ranges of hyperparameter settings should be identified as such as part of standard publication practice.
Once hyperparameters have been properly controlled for, we find that LSTMs outperform the more recent models, contra the published claims.
Answer: Yes.
Question: Who is the company Nestle
Evidence: Nestlé S.A. (/ˈnɛsleɪ, -li, -əl/ NESS-lay, -⁠lee, -⁠əl, French: [nɛsle], German: [ˈnɛstlə] (listen)) is a Swiss multinational food and drink processing conglomerate corporation headquartered in Vevey, Vaud, Switzerland. It is the largest publicly held food company in the world, measured by revenue and other metrics, since 2014. It ranked No. 64 on the Fortune Global 500 in 2017 and No. 33 in the 2016 edition of the Forbes Global 2000 list of largest public companies.
Answer: Nestle S.A. is a Swiss multinational food and drink processing conglomerate corporation headquartered in Vevey, Vaud, Switzerland. It is the largest publicly held food company in the world, measured by revenue and other metrics, since 2014.
Question: Is there a strong correlation between the rise and fall in stock prices with the public sentiments in tweets?
Evidence: Predicting stock market movements is a well-known problem of interest. Now-a-days social media is perfectly representing the public sentiment and opinion about current events. Especially, twitter has attracted a lot of attention from researchers for studying the public sentiments. Stock market prediction on the basis of public sentiments expressed on twitter has been an intriguing field of research. Previous studies have concluded that the aggregate public mood collected from twitter may well be correlated with Dow Jones Industrial Average Index (DJIA). The thesis of this work is to observe how well the changes in stock prices of a company, the rises and falls, are correlated with the public opinions being expressed in tweets about that company. Understanding author's opinion from a piece of text is the objective of sentiment analysis. The present paper have employed two different textual representations, Word2vec and N-gram, for analyzing the public sentiments in tweets. In this paper, we have applied sentiment analysis and supervised machine learning principles to the tweets extracted from twitter and analyze the correlation between stock market movements of a company and sentiments in tweets. In an elaborate way, positive news and tweets in social media about a company would definitely encourage people to invest in the stocks of that company and as a result the stock price of that company would increase. At the end of the paper, it is shown that a strong correlation exists between the rise and falls in stock prices with the public sentiments in tweets.
At the end of the paper, it is shown that a strong correlation exists between the rise and falls in stock prices with the public sentiments in tweets.
Answer: Yes.
Question: When were the most recent eruptions of Mount Hood?
Evidence: The last three eruptions at Mount Hood occurred within the past 1,800 years from vents high on the southwest flank and produced deposits that were distributed primarily to the south and west along the Sandy and Zigzag rivers. The last eruptive period took place around 220 to 170 years ago, when dacitic lava domes, pyroclastic flows and mudflows were produced without major explosive eruptions. The prominent Crater Rock just below the summit is hypothesized to be the remains of one of these now-eroded domes. This period includes the last major eruption of 1781 to 1782 with a slightly more recent episode ending shortly before the arrival of the explorers Lewis and Clark in 1805. The latest minor eruptive event occurred in August 1907.
Answer: Mount Hood most recently erupted around 220 to 170 years ago. The most recent major Mount Hood eruptions occurred between the years of 1781 to 1782 and again in 1805. The latest minor eruption occurred in August 1907.
Question: Write a rap about work
Answer: Hard work they say it always pays off
But I think that smart work is the way to go
It starts by finding and pursuing your passion
The money will follow you'll have plenty of cash then
This ain't gonna happen overnight
You've gotta hustle with grit to do it right
Surround yourself with the best achievers
Keep your mind right don't stop believing 
You'll hit bumps in the road as you go on your journey 
So you gotta set goals and you gotta keep learning
It's true there's more to life than your career
So keep your eye on the prize keep your vision clear
Because all the early mornings and all the late nights
They will fly by without warning so enjoy the great ride
Question: In The Divergent series by Veronica Roth, what is the name used to refer to people who don't fit into a specific faction?
Answer: Divergents are people who cannot be classified neatly into a specific faction.
Question: What are the names of the four tennis grand slam tournaments?
Answer: The four tennis grand slam tournaments are the Australian Open, French Open, Wimbledon, and the US Open.
Question: How are different network components evaluated?
Evidence: Shared Self-attention Layers As our model providing two outputs from one input, there is a bifurcation setting for how much shared part should be determined. Both constituent and dependency parsers share token representation and 8 self-attention layers at most. Assuming that either parser always takes input information flow through 8 self-attention layers as shown in Figure FIGREF4, then the number of shared self-attention layers varying from 0 to 8 may reflect the shared degree in the model. When the number is set to 0, it indicates only token representation is shared for both parsers trained for the joint loss through each own 8 self-attention layers. When the number is set to less than 8, for example, 6, then it means that both parsers first shared 6 layers from token representation then have individual 2 self-attention layers.
For different numbers of shared layers, the results are in Table TABREF14. We respectively disable the constituent and the dependency parser to obtain a separate learning setting for both parsers in our model. The comparison in Table TABREF14 indicates that even though without any shared self-attention layers, joint training of our model may significantly outperform separate learning mode. At last, the best performance is still obtained from sharing full 8 self-attention layers.
Shared Self-attention Layers As our model providing two outputs from one input, there is a bifurcation setting for how much shared part should be determined. Both constituent and dependency parsers share token representation and 8 self-attention layers at most. Assuming that either parser always takes input information flow through 8 self-attention layers as shown in Figure FIGREF4, then the number of shared self-attention layers varying from 0 to 8 may reflect the shared degree in the model. When the number is set to 0, it indicates only token representation is shared for both parsers trained for the joint loss through each own 8 self-attention layers. When the number is set to less than 8, for example, 6, then it means that both parsers first shared 6 layers from token representation then have individual 2 self-attention layers.
For different numbers of shared layers, the results are in Table TABREF14. We respectively disable the constituent and the dependency parser to obtain a separate learning setting for both parsers in our model. The comparison in Table TABREF14 indicates that even though without any shared self-attention layers, joint training of our model may significantly outperform separate learning mode. At last, the best performance is still obtained from sharing full 8 self-attention layers.
Answer: The answers are shown as follows:
* For different numbers of shared layers, the results are in Table TABREF14. We respectively disable the constituent and the dependency parser to obtain a separate learning setting for both parsers in our model.
Question: Name some reasons it's great to live in Marin County, California.
Answer: Marin County is a great place to live for many reasons, including the beautiful scenery, the mild, pleasant weather, and the abundance of outdoor activities, while at the same time being close to San Francisco. Mountain biking on the fire trails in the hills of Marin, and climbing various hilltops is very enjoyable. There are fantastic views of the city of San Francisco, and of San Francisco Bay from many points in the county. Walking or biking the Tiburon Bike Path is also great fun. From Marin it's easy to take great day trips to nearby Napa and Sonoma counties, to enjoy their beautiful natural settings and to visit the wineries and nice restaurants.
Question: What is the official language in Brazil?
Answer: Many assume that in Brazil the official language is Spanish because it's the official language in many countries in South America, but in Brazil the official language is Portuguese. Brazil is the largest Portuguese-speaking country in the world.
Question: Categorize each of the following as 'Programming Language' or 'Not a Programming Language'
Answer: Python, Java, English, Spanish, Javascript, Italian, Japanese
Question: What limitations are mentioned?
Evidence: We identified some limitations during the process, which we describe in this section.
When deciding publisher partisanship, the number of people from whom we computed the score was small. For example, de Stentor is estimated to reach 275K readers each day on its official website. Deciding the audience leaning from 55 samples was subject to sampling bias. Besides, the scores differ very little between publishers. None of the publishers had an absolute score higher than 1, meaning that even the most partisan publisher was only slightly partisan. Deciding which publishers we consider as partisan and which not is thus not very reliable.
The article-level annotation task was not as well-defined as on a crowdsourcing platform. We included the questions as part of an existing survey and didn't want to create much burden to the annotators. Therefore, we did not provide long descriptive text that explained how a person should annotate an article. We thus run under the risk of annotator bias. This is one of the reasons for a low inter-rater agreement.
We identified some limitations during the process, which we describe in this section.

When deciding publisher partisanship, the number of people from whom we computed the score was small. For example, de Stentor is estimated to reach 275K readers each day on its official website. Deciding the audience leaning from 55 samples was subject to sampling bias. Besides, the scores differ very little between publishers. None of the publishers had an absolute score higher than 1, meaning that even the most partisan publisher was only slightly partisan. Deciding which publishers we consider as partisan and which not is thus not very reliable.

The article-level annotation task was not as well-defined as on a crowdsourcing platform. We included the questions as part of an existing survey and didn't want to create much burden to the annotators. Therefore, we did not provide long descriptive text that explained how a person should annotate an article. We thus run under the risk of annotator bias. This is one of the reasons for a low inter-rater agreement.
Answer: deciding publisher partisanship, risk annotator bias because of short description text provided to annotators
Question: What are the two categories of conversational systems according to the types of intentions?
Evidence: According to the types of intentions, conversational systems can be classified into two categories: a) goal-driven or task oriented, and b) non-goal-driven or end-to-end systems.
In a goal-driven system, the main objective is to interact with the user so that back-end tasks, which are application specific, are executed by a supporting system. As an example of application we can cite technical support systems, for instance air ticket booking systems, where the conversation system must interact with the user until all the required information is known, such as origin, destination, departure date and return date, and the supporting system must book the ticket. The most widely used approaches for developing these systems are Partially-observed Decision Processes (POMDP) BIBREF30 , Hidden Markov Models (HMM) BIBREF29 , and more recently, Memory Networks BIBREF28 . Given that these approaches are data-oriented, a major issue is to collect a large corpora of annotated task-specific dialogs. For this reason, it is not trivial to transfer the knowledge from one to domain to another. In addition, it might be difficult to scale up to larger sets of tasks.
Non-goal-driven systems (also sometimes called reactive systems), on the other hand, generate utterances in accordance to user input, e.g. language learning tools or computer games characters. These systems have become more popular in recent years, mainly owning to the increase of popularity of Neural Networks, which is also a data-oriented approach. The most recent state of the art to develop such systems have employed Recurrent Neural Networs (RNN) BIBREF34 , Dynamic Context-Sensitive Generation BIBREF35 , and Memory Networks BIBREF36 , just to name a few. Nevertheless, probabilistic methods such as Hidden Topic Markov Models (HTMM) BIBREF37 have also been evaluated. Goal-driven approach can create both pro-active and reactive chatbots, while non-goal-driven approach creates reactive chatbots. In addition, they can serve as a tool to goal-driven systems as in BIBREF28 . That is, when trained on corpora of a goal-driven system, non-goal-driven systems can be used to simulate user interaction to then train goal-driven models.
According to the types of intentions, conversational systems can be classified into two categories: a) goal-driven or task oriented, and b) non-goal-driven or end-to-end systems.
Answer: According to the types of intentions, conversational systems can be classified into two categories: a) goal-driven or task oriented, and b) non-goal-driven or end-to-end systems.
Question: Categorize each of the following terms as related to "golf" or "hockey": puck, goal, caddy, flag, green, helmet
Answer: Puck: Hockey
Goal: Hockey
Caddy: Golf
Flag: Golf
Green: Golf
Helmet: Hockey
Question: What is the difference between the Debt Snowball and Debt Avalanche methods of paying off debt?
Answer: The Debt Snowball and Debt Avalanche are two widely known methods of paying off debt. One is a mathematically faster way to pay off the debt; whereas, the alternative is slightly slower but is more psychologically motivating. This motivation is what helps the person reach their debt pay off goals. The following set of debts will be used as the basis for explaining each method:

Debt A has a balance of $10,000 at a 4% interest rate. 
Debt B has a balance of $5,000 at a 10% interest rate.
Debt C has a balance of $8,000 at a 24% interest rate. 

The Debt Avalanche is typically the fastest and most cost effective way to pay off debt. The person makes minimum payments on all their debts and any extra goes towards the account with highest interest rate (Debt C in example above). Once that first balance is paid off, the person takes the payments from that and puts it all towards the second highest interest rate (Debt B). Repeat until all debts are paid.

The Debt Snowball is a method popularized by a personal finance expert, Dave Ramsey, in Step 2 of his 7 Baby Steps to building wealth. This method uses psychology by paying off the smallest balance first (Debt B above). The theory is that the feeling of quickly paying off a debt balance provides the motivation to stick with the program. Similar to the Debt Avalanche, the person takes the payments from Debt B and puts it towards the next smallest balance (Debt C). Repeat until all debts are paid.
Question: What is a stock in finance?
Answer: In finance, stock (also capital stock) consist of all the shares by which ownership of a corporation or company is divided. (Especially in American English, the word "stocks" is also used to refer to shares.) A single share of the stock means fractional ownership of the corporation in proportion to the total number of shares. This typically entitles the shareholder (stockholder) to that fraction of the company's earnings, proceeds from liquidation of assets (after discharge of all senior claims such as secured and unsecured debt), or voting power, often dividing these up in proportion to the amount of money each stockholder has invested. Not all stock is necessarily equal, as certain classes of stock may be issued for example without voting rights, with enhanced voting rights, or with a certain priority to receive profits or liquidation proceeds before or after other classes of shareholders.

Stock can be bought and sold privately or on stock exchanges, and such transactions are typically heavily regulated by governments to prevent fraud, protect investors, and benefit the larger economy. The stocks are deposited with the depositories in the electronic format also known as Demat account. As new shares are issued by a company, the ownership and rights of existing shareholders are diluted in return for cash to sustain or grow the business. Companies can also buy back stock, which often lets investors recoup the initial investment plus capital gains from subsequent rises in stock price. Stock options issued by many companies as part of employee compensation do not represent ownership, but represent the right to buy ownership at a future time at a specified price. This would represent a windfall to the employees if the option is exercised when the market price is higher than the promised price, since if they immediately sold the stock they would keep the difference (minus taxes).
Question: Which classifier is the most efficient in several data science challenges?
Evidence: Automatically predicting the level of English of non-native speakers from their written text is an interesting text mining task. Systems that perform well in the task can be useful components for online, second-language learning platforms as well as for organisations that tutor students for this purpose. In this paper I present the system balikasg that achieved the state-of-the-art performance in the CAp 2018 data science challenge among 14 systems. In order to achieve the best performance in the challenge, I decided to use a variety of features that describe an essay's readability and syntactic complexity as well as its content. For the prediction step, I found Gradient Boosted Trees, whose efficiency is proven in several data science challenges, to be the most efficient across a variety of classifiers.
The rest of the paper is organized as follows: in Section 2 I frame the problem of language level as an ordinal classification problem and describe the available data. Section 3 presents the feature extaction and engineering techniques used. Section 4 describes the machine learning algorithms for prediction as well as the achieved results. Finally, Section 5 concludes with discussion and avenues for future research.
For the prediction step, I found Gradient Boosted Trees, whose efficiency is proven in several data science challenges, to be the most efficient across a variety of classifiers.
Answer: Gradient Boosted Trees
Question: What is the difference in recall score between the systems?
Evidence: FLOAT SELECTED: Table 2. Comparison with Traditional NERs as reference
FLOAT SELECTED: Table 3. Comparison with Wikipedia titles as reference
FLOAT SELECTED: Table 2. Comparison with Traditional NERs as reference
FLOAT SELECTED: Table 3. Comparison with Wikipedia titles as reference
Answer: Between the model and Stanford, Spacy and Flair the differences are 42.91, 25.03, 69.8 with Traditional NERs as reference and  49.88, 43.36, 62.43 with Wikipedia titles as reference.
Question: How did Frank Stagg die?
Evidence: Upon conviction, Stagg was given a ten-year sentence and Fell twelve years. (Stagg would later die on a hunger strike.) Thomas Gerald Rush was given seven years and Anthony Roland Lynch, who was also found guilty of possessing articles with intent to destroy property, namely nitric acid, balloons, wax, and sodium chlorate, was given ten years. Fell never admitted IRA membership
Answer: Frank Stagg was sentenced to ten years and prison. During his sentence, he went on a hunger strike and died as a result.
Question: Where can you find Anostraca?
Evidence: Anostraca is one of the four orders of crustaceans in the class Branchiopoda; its members are referred to as fairy shrimp. They live in vernal pools and hypersaline lakes across the world, and they have even been found in deserts, ice-covered mountain lakes and Antarctic ice. They are usually 6–25 mm (0.24–0.98 in) long (exceptionally up to 170 mm or 6.7 in). Most species have 20 body segments, bearing 11 pairs of leaf-like phyllopodia (swimming legs), and the body lacks a carapace. They swim "upside-down" and feed by filtering organic particles from the water or by scraping algae from surfaces, with the exception of Branchinecta gigas, or "giant fairy shrimp", which is itself a predator of other species of anostracans. They are an important food for many birds and fish, and some are cultured and harvested for use as fish food. There are 300 species spread across 8 families.
Answer: The crustaceans Anostraca can be be found in vernal pools and hypersaline lakes, as well as deserts, ice-covered mountain lakes and Antarctic ice.
Question: Classify the sentiment of this piece of text as either positive or negative: “I am not pretty. I am not beautiful. I am as radiant as the sun.”
Answer: Positive