DSGym/DSGym-Tasks huggingface dataset:
Context: The dataset provided, ak91.csv, contains information on individuals' log wages, years of schooling, year of birth, quarter of birth, and state of birth. The purpose of the analysis is to estimate the effect of education on wage, using linear regression and quarter of birth as an instrumental variable (IV). This idea takes advantage of US compulsory attendance law. Usually, they state that a kid must have turned 6 years by January 1 of the year they enter school. For this reason, kids that are born at the beginning of the year will enter school at an older age. Compulsory attendance law also requires students to be in school until they turn 16, at which point they are legally allowed to drop out. The result is that people born later in the year have, on average, more years of education than those born in the beginning of the year.

Question: What is the average additional percentage wage increase associated with each additional year of education based on the instrumental variable of whether the individual is born in the last quarter q4? Please adjust for all the other variables, and round your answer to the nearest hundredth of a percent.
Answer: 8.53%
Topics: Causality, Observational data, Regression analysis, Average treatment effect, Instrumental variables
Context: A study is conducted to measure the effect of a marketing push on user engagement, specifically in-app purchases. Some customers who were assigned to receive the push are not receiving it, because they probably have an older phone that doesn’t support the kind of push the marketing team designed.
The dataset app_engagement_push.csv contains records for 10,000 random customers. Each record includes whether an in-app purchase was made (in_app_purchase), if a marketing push was assigned to the user (push_assigned), and if the marketing push was successfully delivered (push_delivered).

Question: What is the Local Average Treatment Effect (LATE) of receiving the marketing push on in-app purchases, as estimated using linear regression and instrumental variable, rounded to two decimal places?
Answer: 3.29
Topics: Causality, Observational data, Regression analysis, Local average treatment effect, Instrumental variables

Question: Consider we estimate the Average Treatment Effect of receiving the marketing push on in-app purchases, by conducting linear regression with the formula "in_app_purchase ~ 1 + push_assigned + push_delivered". Will the true impact be A) higher than, B) lower than, C) the same with our estimated impact? Please answer with A, B, or C.
Answer: B
Topics: Causality, Observational data, Regression analysis, Average treatment effect, Instrumental variables
Context: To investigate the effect of a medication on the number of days it takes for a patient to recover from an illness, we have a dataset that includes several confounding variables like severity, sex, and age. The dataset medicine_impact_recovery.csv contains data on patients who have been prescribed medication and those who haven't. The variables include sex (0 or 1), age, the severity of the condition, whether the patient was on medication (0 or 1), and the number of days it took for each patient to recover.

Question: What is the average treatment effect of the medication on the recovery time when controlling for severity, sex, and age using the K nearest neighbour matching? Please scale the features, use the euclidean norm as the matching measurement, and also adjust for the matching bias. Provide your answer rounded to two decimal places. The answer should be positive if the mediation makes the recovery time longer.
Answer: -7.36
Topics: Causality, Observational data, Matching, Average treatment effect
Context: The National Study of Learning Mindsets is a randomised study conducted in U.S. public high schools which aims at finding the impact of a growth mindset. The way it works is that students receive from the school a seminar to instill in them a growth mindset. Then, they follow up the students in their college years to measure how well they’ve performed academically. This measurement was compiled into an achievement score and standardized. The CSV file learning_mindset.csv contains simulated data of this research.
Variable Description
intervention: the intervention of the growth mindset;
achievement_score: the standardized academic achievement score;
schoolid: identifier of the student’s school;
success_expect: self-reported expectations for success in the future, a proxy for prior achievement, measured prior to random assignment;
ethnicity: categorical variable for student race/ethnicity;
gender: categorical variable for student identified gender;
frst_in_family: categorical variable for student first-generation status, i.e. first in family to go to college;
school_urbanicity: school-level categorical variable for urbanicity of the school, i.e. rural, suburban, etc;
school_mindset: school-level mean of students’ fixed mindsets, reported prior to random assignment, standardized;
school_achievement: school achievement level, as measured by test scores and college preparation for the previous 4 cohorts of students, standardized;
school_ethnic_minority: school racial/ethnic minority composition, i.e., percentage of student body that is Black, Latino, or Native American, standardized;
school_poverty: school poverty concentration, i.e., percentage of students who are from families whose incomes fall below the federal poverty line, standardized;
school_size: total number of students in all four grade levels in the school, standardized.

Question: What is the average treatment effect of the growth mindset on the achievement score? Please use propensity score weighting in estimation, use logistic regression to estimate the propensity score, and round the final answer to the nearest hundredth.
Answer: 0.39
Topics: Causality, Observational data, Matching, Propensity score, Average treatment effect

Question: What is the average treatment effect of the growth mindset on the achievement score? Please conduct doubly robust estimation, use logistic regression to estimate the propensity score, and round the final answer to the nearest hundredth.
Answer: 0.39
Topics: Causality, Observational data, Matching, Propensity score, Average treatment effect
Context: The dataset billboard_impact.csv details information from a quasi-experiment assessing the influence of billboards on bank deposits in two cities: Porto Alegre (treatment group) and Florianopolis (control group). The csv file contains records with three variables: deposits (average bank deposits in Brazilian Reais), poa (A dummy indicator for the city of Porto Alegre. When it is zero, it means the samples are from Florianopolis.), and jul (A dummy for the month of July, or for the post intervention period. When it is zero it refers to samples from May, the pre-intervention period).

Question: What was the average increase in bank deposits per customer in Porto Alegre after the billboard intervention, as estimated by the difference-in-differences approach? Please round the final answer to the nearest hundredth.
Answer: 6.52
Topics: Causality, Observational data, Difference in differences
Context: To estimate the impacts of alcohol on death, we could use the fact that legal drinking age imposes a discontinuity on nature. In the US, those just under 21 years don’t drink (or drink much less) while those just older than 21 do drink. The csv file drinking.csv contains mortality data aggregated by age. Each row is the average age of a group of people and the average mortality by all causes (all), by moving vehicle accident (mva) and by suicide (suicide).

Question: How much is the effect of alcohol consumption on death of all causes at 21 years? Please only consider data from people that are no older than 22 years and no younger than 20 years. Please answer with the magnitude of change in the number of deaths and round to the nearest hundredth.
Answer: 0.10
Topics: Causality, Observational data, Difference in differences
Context: To evaluate the effectiveness of email marketing on customer investment decisions, the dataset invest_email_rnd_train.csv is used for training, and invest_email_rnd_train.csv is used for testing. They contain customer data including id, age, income, insurance, invested amount, binary indicators for whether they received three different emails (em1, em2, em3), and a binary outcome variable indicating whether the customer converted (invested vs. didn't invest) after receiving an email. The goal is to personalize email marketing by sending email-1 only to customers predicted to have the highest increase in the probability of conversion.

Question: Please train a boosted tree model to predict the conditional average treatment effect (CATE) of sending email-1 (em1) on the conversion rate, using demographic and financial attributes of the customers. What is the predicted increase in the probability that customer 6958 will purchase the investment product, if they are sent email-1, according to the CATE estimate provided by the model? Please provide the answer as a percentage, rounded to two decimal places.
Answer: 10.57%
Topics: Causality, Observational data, Conditional Average Treatment Effect, Target transformation
Context: To find the effect of price on ice cream sales, the dataset ice_cream_sales.csv is used for training, and ice_cream_sales_rnd.csv is used for testing. Our test set has randomly assigned prices but our training data has only observational prices, which is potentially biased. Each unit is a day. For each day, we know which day of a week it is, what was the cost we had to make the ice cream (you can think of the cost as a proxy for quality) and the average temperature for that day. Then, we have our treatment, price, and our outcome, the number of ice cream sold.

Question: Using the debiased machine learning approach after correcting for confounding biases related to temperature, cost, and weekday effects, what is the estimated average treatment effect (ATE) of the ice cream price on sales, rounded to two decimal places?
Answer: -3.92
Topics: Causality, Observational data, Average treatment effect, Debiased ML
Context: To estimate the effect of cigarette taxation on its consumption, data from cigarette sales were collected and analyzed across 39 states in the United States from the years 1970 to 2000. Proposition 99, a Tobacco Tax and Health Protection Act passed in California in 1988, imposed a 25-cent per pack state excise tax on tobacco cigarettes and implemented additional restrictions, including the ban on cigarette vending machines in public areas accessible by juveniles and a ban on the individual sale of single cigarettes. Revenue generated was allocated for environmental and health care programs along with anti-tobacco advertising. We aim to determine if the imposition of this tax and the subsequent regulations led to a reduction in cigarette sales. The data is in the CSV file smoking2.csv.

Question: By the year 2000, what was the estimated reduction in per-capita cigarette sales (in packs) in California as a result of Proposition 99, based on the synthetic control method? Please round the final answer to the nearest hundredth.
Answer: 24.83
Topics: Causality, Observational data, Synthetic control

Question: What is the Average Treatment Effect on the treated (ATT) of Proposition 99 on cigarette sales, as estimated by the difference-in-differences approach? Please round the final answer to the nearest hundredth. Do not consider variables other than cigsale, california, and after_treatment. The answer is positive if the proposition increases the sale.
Answer: -27.35
Topics: Causality, Observational data, Average treatment effect on the treated, Difference in differences

Question: What is the Average Treatment Effect on the treated (ATT) of Proposition 99 on cigarette sales, as estimated by the synthetic control approach? Please round the final answer to the nearest hundredth. Do not consider variables other than year, state, cigsale, california, and after_treatment. The answer is positive if the proposition increases the sale.
Answer: -19.51
Topics: Causality, Observational data, Average treatment effect on the treated, Synthetic control
Context: We are trying to estimate the effect of a trainee program on earnings. Data in the CSV file trainee_unique_on_age.csv contains trainee status, age, and earnings of each unit. Trainees are much younger than non trainees, which indicates that age is probably a confounder.

Question: What is the Average Treatment Effect on the treated (ATT) of the trainee program on earnings, as estimated by a matching estimator? Please round the final answer to the nearest hundredth. The answer is positive if the trainee program increases the earnings.
Answer: 2457.89
Topics: Causality, Observational data, Average treatment effect on the treated, Matching
Context: Does racial discrimination exist in the labor market? Or, should racial disparities in the unemployment rate be attributed to other factors such as racial gaps in
educational attainment? To answer this question, two social scientists conducted the following experiment.1 In response to newspaper ads, the researchers sent out
résumés of fictitious job candidates to potential employers. They varied only the names of job applicants, while leaving the other information in the résumés unchanged.
For some candidates, stereotypically African-American-sounding names such as Lakisha Washington or Jamal Jones were used, whereas other résumés contained
stereotypically white-sounding names, such as Emily Walsh or Greg Baker. The researchers then compared the callback rates between these two groups and examined
whether applicants with stereotypically black names received fewer callbacks than those with stereotypically white names. The positions to which the applications were
sent were either in sales, administrative support, clerical, or customer services.

Data is in the CSV data file resume.csv.
Variable Description
firstname: first name of the fictitious job applicant
sex: sex of applicant (female or male)
race: race of applicant (black or white)
call: whether a callback was made (1= yes, 0= no)

Question: What is the racial gap in callback rate among white and black female applicants? Please round to the nearest thousandth.
Answer: 0.033
Topics: Social science, Statistics, Probability, Conditional probability

Question: What is the racial gap in callback rate among white and black male applicants? Please round to the nearest thousandth.
Answer: 0.030
Topics: Social science, Statistics, Probability, Conditional probability

Question: What is the callback rate of the first name with the lowest callback rate? Please round to the nearest thousandth.
Answer: 0.022
Topics: Social science, Statistics, Probability, Conditional probability

Question: What is the callback rate of the first name with the highest callback rate? Please round to the nearest thousandth.
Answer: 0.159
Topics: Social science, Statistics, Probability, Conditional probability
Context: After the September 11 attacks, the United States and its allies invaded Afghanistan with the goal of dismantling al-Qaeda, which had been operating there under the protection of the Taliban government. In 2003, the North Atlantic Treaty Organization (NATO) became involved in the conflict, sending in a coalition of international troops organized under the name of the International Security Assistance Force (ISAF). To wage this war against the Taliban insurgency, the ISAF engaged in a “hearts and minds” campaign, combining economic assistance, service delivery, and protection in order to win the support of civilians. To evaluate the success of such a campaign, it is essential to measure and understand civilians’ experiences and sentiments during the war. However, measuring the experiences and opinions of civilians during wartime is a challenging task because of harsh security conditions, posing potential threats to interviewers and respondents. This means that respondents may inaccurately answer survey questions in order to avoid giving socially undesirable responses.
A group of social scientists conducted a public opinion survey in southern Afghanistan, the heartland of the insurgency. The survey was administered to a sample of 2754 respondents between January and February 2011. The researchers note that the participation rate was 89%. That is, they originally contacted 3097 males and 343 of them refused to take the survey. Because local culture prohibited interviewers from talking to female citizens, the respondents were all males.

The CSV file afghan.csv contains this data set.
Variable Description
province: province where the respondent lives
district: district where the respondent lives
village.id: ID of the village where the respondent lives
age: age of the respondent
educ.years: years of education of the respondent
employed: whether the respondent is employed
income: monthly income of the respondent (five levels)
violent.exp.ISAF: whether the respondent experienced violence by ISAF
violent.exp.taliban: whether the respondent experienced violence by the Taliban
list.group: randomly assigned group for the list experiment (control, ISAF, taliban)
list.response: response to the list experiment question (0–4)

Question: What proportion of respondents did not report the monthly income? Please round to the nearest thousandth.
Answer: 0.056
Topics: Social science, Statistics, Probability, Missing value
Context: It has been pointed out that rising income inequality may be responsible for the widening partisan gap. To measure income inequality, we use the Gini coefficient (Gini index), which is best understood graphically. Figure 3.4 illustrates the idea. The horizontal axis represents the cumulative share of people.

The CSV data file, USGini.csv, contains the Gini coefficient from 1947 to 2013. We notice that both political polarization and income inequality have been steadily increasing in the United States.

Variable Description
year: year
gini: US Gini coefficient
polarization: measure of political polarization

Question: What is the correlation between the Gini coefficient and the measure of political polarization? Since each US congressional session lasts two years, we take the Gini coefficient for the second year of each session. Please round to the nearest thousandth.
Answer: 0.942
Topics: Social science, Statistics, Correlation
Context: The United States’s unique Electoral College system makes predicting election outcomes challenging. A candidate is elected to office by winning an absolute majority of electoral votes. Each of the 538 electors casts a single electoral vote. As of 2016, 535 of these votes are allocated among 50 states, corresponding to the 435 members of the House of Representatives and the 100 members of the Senate. The remaining 3 votes are given to the District of Columbia. In most cases, the electors vote for the candidate who won the plurality of votes in the state they represent, leading to a “winner-takeall” system in these states. In fact, some states have criminal penalties for voting for the candidate who did not win the plurality of votes. A winning presidential candidate must obtain at least 270 electoral votes.

The CSV data file pres08.csv contains the election results by state of the 2008 US presidential election. The election date was 11/4/2008.
Variable Description
state: abbreviated name of the state
state.name: unabbreviated name of the state
Obama: Obama’s vote share (percentage)
McCain: McCain’s vote share (percentage)
EV: number of Electoral College votes for the state

In addition, we have the CSV file polls08.csv, which contains many polls within each state leading up to the election.
Variable Description
state: abbreviated name of the state in which the poll was conducted
Obama: predicted support for Obama (percentage)
McCain: predicted support for McCain (percentage)
Pollster: name of the organization conducting the poll
middate: middate of the period when the poll was conducted

Question: What is the average prediction error of the margin between the percentage share of Obama and McCain across states? We only consider the latest polls within each state. If there are several latest polls, we consider the mean of them. Please round to the nearest thousandth.
Answer: 1.062
Topics: Social science, Statistics

Question: What is the root mean squared prediction error of the margin between the percentage share of Obama and McCain across states? We only consider the latest polls within each state. If there are several latest polls, we consider the mean of them. Please round to the nearest thousandth.
Answer: 5.909
Topics: Social science, Statistics

Question: How many states got poll results that were wrong? We only consider the latest polls within each state. If there are several latest polls, we consider the mean of them. Please answer with an integer.
Answer: 3
Topics: Social science, Statistics

Question: What is the gap between the actual and predicted total number of electoral votes won by Obama? We only consider the latest polls within each state. If there are several latest polls, we consider the mean of them. Please answer with an integer.
Answer: 15
Topics: Social science, Statistics
Context: Several psychologists have reported the intriguing result of an experiment showing that facial appearance predicts election outcomes better than chance. In their experiment, the researchers briefly showed student subjects the black-and-white head shots of two candidates from a US congressional election (winner and runner-up). The exposure of subjects to facial pictures lasted less than a second, and the subjects were then asked to evaluate the two candidates in terms of their perceived competence.
The researchers used these competence measures to predict election outcomes. The key hypothesis is whether or not a within-a-second evaluation of facial appearance can predict election outcomes. The CSV data set, face.csv, contains the data from the experiment. Note that we include data only from subjects who did not know the candidates’ political parties, their policies, or even which candidate was the incumbent or challenger. They were simply making snap judgments about which candidate appeared more competent based on their facial expression alone.

Variable Description
congress: session of Congress
year: year of the election
state: state of the election
winner: name of the winner
loser: name of the runner-up
w.party: party of the winner
l.party: party of the loser
d.votes: number of votes for the Democratic candidate
r.votes: number of votes for the Republican candidate
d.comp: competence measure for the Democratic candidate
r.comp: competence measure for the Republican candidate

Question: What is the correlation between the perceived competence of the Democratic candidate and the vote share differential of the Democratic candidate minus the Republican candidate? Please round to the nearest thousandth.
Answer: 0.433
Topics: Social science, Statistics, Correlation

Question: Fit a linear regression model using the Democratic margin in the two-party vote share as the response variable and the perceived competence for Democratic candidates as the predictor. What is the estimated slope of the model? Please round to the nearest thousandth.
Answer: 0.660
Topics: Social science, Statistics, Correlation, Linear regression

Question: Fit a linear regression model using the Democratic margin in the two-party vote share as the response variable and the perceived competence for Democratic candidates as the predictor. What is the root-mean-squared error (RMSE) of the model estimation? Please round to the nearest thousandth.
Answer: 0.264
Topics: Social science, Statistics, Correlation, Linear regression, Root-mean-squared error