DSGym/DSGym-Tasks huggingface dataset:
Context: The CSV file stent365.csv summarizes an experiment that studies effectiveness of stents in treating patients at risk of stroke with some unexpected results. These data represent the results 365 days after stroke.

Question: Compute the proportion of patients in the treatment group who had a stroke by the end of their first year. Please round to the nearest hundredth.
Answer: 0.20
Topics: Statistics, Probability
Context: A migraine is a particularly painful type of headache, which patients sometimes wish to treat with acupuncture. To determine whether acupuncture relieves migraine pain, researchers conducted a randomized controlled study where 89 females diagnosed with migraine headaches were randomly assigned to one of two groups: treatment or control. 43 patients in the treatment group received acupuncture that is specifically designed to treat migraines. 46 patients in the control group received placebo acupuncture (needle insertion at non-acupoint locations). 24 hours after patients received acupuncture, they were asked if they were pain free. The research data is in the CSV file migraine.csv.

Question: What proportion of patients in the treatment group were pain free 24 hours after receiving acupuncture? Please round to the nearest hundredth.
Answer: 0.23
Topics: Statistics, Probability
Context: Researchers studying the effect of antibiotic treatment for acute sinusitis compared to symptomatic treatments randomly assigned 166 adults diagnosed with acute sinusitis to one of two groups: treatment or control. Study participants received either a 10-day course of amoxicillin (an antibiotic) or a placebo similar in appearance and taste. The placebo consisted of
symptomatic treatments such as acetaminophen, nasal decongestants, etc. At the end of the 10-day period, patients were asked if they experienced improvement in symptoms. The research data is in the CSV file sinusitis.csv.

Question: What proportion of patients in the treatment group experienced improvement in symptoms? Please round to the nearest hundredth.
Answer: 0.78
Topics: Statistics, Probability

Question: What proportion of patients in the control group experienced improvement in symptoms? Please round to the nearest hundredth.
Answer: 0.80
Topics: Statistics, Probability

Question: In which group did a higher percentage of patients experience improvement in symptoms? Please answer with "treatment group" or "control group".
Answer: control group
Topics: Statistics, Probability
Context: A survey was conducted to study the smoking habits of UK residents. The data is in the CSV file smoking.csv.

Question: How many participants were included in the survey?
Answer: 1691
Topics: Statistics, Data analysis
Context: Daily air quality is measured by the air quality index (AQI) reported by the Environmental Protection Agency. This index reports the pollution level and what associated health effects might be a concern. The index is calculated for five major air pollutants regulated by the Clean Air Act and takes values from 0 to 300, where a higher value indicates lower air quality. AQI was reported for a sample of 91 days in 2011 in Durham, NC. The data is in the CSV file pm25_2011_durham.csv.

Question: Estimate the median AQI value of this sample. Please round to the nearest integer.
Answer: 30
Topics: Statistics, Median

Question: Estimate the interquartile range of AQI in this sample. Please round to the nearest hundredth.
Answer: 19.50
Topics: Statistics, Interquartile range
Context: 910 randomly sampled registered voters from Tampa, FL were asked if they thought workers who have illegally entered the US should be (i) allowed to keep their jobs and apply for US citizenship, (ii) allowed to keep their jobs as temporary guest workers but not allowed to apply for US citizenship, or (iii) lose their jobs and have to leave the country. The results of the survey by political ideology are in the CSV file immigration.csv.

Question: What proportion of these Tampa, FL voters identify themselves as conservatives? Please round to the nearest hundredth.
Answer: 0.41
Topics: Statistics, Probability

Question: What proportion of these Tampa, FL voters identify themselves as conservatives and are in favor of the citizenship option? Please round to the nearest hundredth.
Answer: 0.06
Topics: Statistics, Probability

Question: What proportion of these Tampa, FL voters who identify themselves as conservatives are also in favor of the citizenship option? Please round to the nearest hundredth.
Answer: 0.15
Topics: Statistics, Probability
Context: Rosiglitazone is the active ingredient in the controversial type 2 diabetes medicine Avandia and has been linked to an increased risk of serious cardiovascular problems such as stroke, heart failure, and death. A common alternative treatment is pioglitazone, the active ingredient in a diabetes medicine called Actos. In a nationwide retrospective observational study of 227,571 Medicare beneficiaries aged 65 years or older, it was found that 2,593 of the 67,593 patients using rosiglitazone and 5,386 of the 159,978 using pioglitazone had serious cardiovascular problems. The data is in the CSV file avandia.csv.

Question: Determine if the following statement is true or false. Be careful: The reasoning may be wrong even if the statement's conclusion is correct. In such cases, the statement should be considered false. Answer with "true" or "false".
Statement: The data suggest that diabetic patients who are taking rosiglitazone are more likely to have cardiovascular problems since the rate of incidence was (2,593 / 67,593 = 0.038) 3.8% for patients on this treatment, while it was only (5,386 / 159,978 = 0.034) 3.4% for patients on pioglitazone.
Answer: true
Topics: Statistics, Data analysis

Question: What proportion of all patients had cardiovascular problems? Please round to the nearest thousandth.
Answer: 0.035
Topics: Statistics, Data analysis

Question: If the type of treatment and having cardiovascular problems were independent, about how many patients in the rosiglitazone group would we expect to have had cardiovascular problems? Please round to the nearest integer.
Answer: 2370
Topics: Statistics, Data analysis
Context: This CSV file infmortrate.csv gives the number of deaths of infants under one year old in 2012 per 1,000 live births in the same year. This rate is often used as an indicator of the level of health in a country.

Question: Estimate the first quartile (Q1) of the infant death rate. Please round to the nearest hundredth.
Answer: 6.51
Topics: Statistics, Quartile

Question: Estimate the third quartile (Q3) of the infant death rate. Please round to the nearest hundredth.
Answer: 42.14
Topics: Statistics, Quartile
Context: This CSV file loans_full_schema.csv represents thousands of loans made through the Lending Club platform, which is a platform that allows individuals to lend to other individuals. The homeownership variable describes whether the borrower rents, has a mortgage, or owns her property.

Question: Compute the probability a randomly selected loan from the data set is for someone who has a mortgage or owns her home. Please round to the nearest hundredth.
Answer: 0.61
Topics: Statistics, Probability

Question: Determine the probability a randomly drawn loan from the data set is from a joint application where the couple had a mortgage. Please round to the nearest thousandth.
Answer: 0.095
Topics: Statistics, Probability
Context: Volunteer patients were randomized into one of two experiment groups where they would receive an experimental vaccine or a placebo. They were subsequently exposed to a drug-sensitive strain of malaria and observed to see whether they came down with an infection. The vaccine trial data is described in the CSV file malaria.csv.

Question: What is the difference in infection rates between the placebo and vaccine groups? Please round to the nearest thousandth.
Answer: 0.643
Topics: Statistics, Probability
Context: The CSV file smallpox.csv represents a sample of 6,224 individuals from the year 1721 who were exposed to smallpox in Boston. Some of them had received a vaccine (inoculated) while others had not. Doctors at the time believed that inoculation, which involves exposing a person to the disease in a controlled form, could reduce the likelihood of death.

Question: What is the probability a randomly selected person who was not inoculated died from smallpox? Please round to the nearest thousandth.
Answer: 0.141
Topics: Statistics, Probability

Question: Determine the probability that an inoculated person died from smallpox. Please round to the nearest thousandth.
Answer: 0.025
Topics: Statistics, Probability
Context: A Pew Research poll asked 1,306 Americans "From what you've read and heard, is there solid evidence that the average temperature on earth has been getting warmer over the past few decades, or not?". The data is in the CSV file global_warming_pew.csv

Question: What is the probability that a randomly chosen respondent believes the earth is warming or is a liberal Democrat? Please round to the nearest hundredth.
Answer: 0.62
Topics: Statistics, Probability

Question: What is the probability that a randomly chosen respondent believes the earth is warming given that he is a liberal Democrat? Please round to the nearest hundredth.
Answer: 0.90
Topics: Statistics, Probability

Question: What is the probability that a randomly chosen respondent believes the earth is warming given that he is a conservative Republican? Please round to the nearest hundredth.
Answer: 0.33
Topics: Statistics, Probability

Question: What is the probability that a randomly chosen respondent is a moderate/liberal Republican given that he does not believe that the earth is warming? Please round to the nearest hundredth.
Answer: 0.18
Topics: Statistics, Probability
Context: A 2010 SurveyUSA poll asked 500 Los Angeles residents, "What is the best hamburger place in Southern California? Five Guys Burgers? In-N-Out Burger? Fat Burger? Tommy's Hamburgers? Umami Burger? Or somewhere else?". The data is in the CSV file burger.csv

Question: What is the probability that a randomly chosen male likes In-N-Out the best? Please round to the nearest hundredth.
Answer: 0.65
Topics: Statistics, Probability

Question: What is the probability that a randomly chosen female likes In-N-Out the best? Please round to the nearest hundredth.
Answer: 0.72
Topics: Statistics, Probability

Question: What is the probability that a randomly chosen person likes Umami best or that person is female? Please round to the nearest hundredth.
Answer: 0.51
Topics: Statistics, Probability
Context: Assortative mating is a nonrandom mating pattern where individuals with similar genotypes and/or phenotypes mate with one another more frequently than what would be expected under a random mating pattern. Researchers studying this topic collected data on eye colors of 204 Scandinavian men and their female partners. The data is in the CSV file assortative_mating.csv

Question: What is the probability that a randomly chosen male respondent or his partner has blue eyes? Please round to the nearest hundredth.
Answer: 0.71
Topics: Statistics, Probability

Question: What is the probability that a randomly chosen male respondent with blue eyes has a partner with blue eyes? Please round to the nearest hundredth.
Answer: 0.68
Topics: Statistics, Probability

Question: What is the probability of a randomly chosen male respondent with green eyes having a partner with blue eyes? Please round to the nearest hundredth.
Answer: 0.31
Topics: Statistics, Probability
Context: The CSV file fcid.csv contains a sample of heights from a survey in the USDA Food Commodity Intake Database. It approximates the height distribution in the US adult population.

Question: What proportion of the sample is between 180 cm and 185 cm tall? Please round to the nearest thousandth.
Answer: 0.117
Topics: Statistics, Probability

Question: Three US adults are randomly selected. What is the probability that all three are between 180 and 185 cm tall? Please round to the nearest thousandth.
Answer: 0.002
Topics: Statistics, Probability

Question: Three US adults are randomly selected. What is the probability that none are between 180 and 185 cm tall? Please round to the nearest hundredth.
Answer: 0.69
Topics: Statistics, Probability
Context: The CSV file ami_occurrences.csv contains occurrences of acute myocardial infarction (AMI) on 365 days in New York City.

Question: Use a Poisson distribution to approximate the data. What is the event rate of the Poisson distribution? Please round to the nearest tenth.
Answer: 4.4
Topics: Statistics, Poisson distribution
Context: The CSV file pew_energy_2018.csv contains data of a US-based survey on support for expanding six different sources of energy, including solar, wind, offshore drilling, hydrolic fracturing ("fracking"), coal, and nuclear.

Question: Calculate the lower bound of a 99% confidence interval for the level of American support for expanding the use of wind turbines for power generation. Please round to the nearest thousandth.
Answer: 0.819
Topics: Statistics, Sampling
Context: In New York City on October 23rd, 2014, a doctor who had recently been treating Ebola patients in Guinea went to the hospital with a slight fever and was subsequently diagnosed with Ebola. Soon thereafter, an NBC 4 New York/The Wall Street Journal/Marist Poll asked New Yorkers whether they favored a "mandatory 21-day quarantine for anyone who has come in contact with an Ebola patient". This poll included responses of 1,042 New York adults between October 26th and 28th, 2014.

Question: What is the point estimate in this case? Please round to the nearest hundredth.
Answer: 0.82
Topics: Statistics, Sampling

Question: Estimate the standard error of the point estimate from the Ebola survey. Please round to the nearest thousandth.
Answer: 0.012
Topics: Statistics, Sampling

Question: Construct a 95% confidence interval for p, the proportion of New York adults who supported a quarantine for anyone who has come into contact with an Ebola patient. Please output the lower bound of the confidence interval and round to the nearest thousandth.
Answer: 0.796
Topics: Statistics, Sampling

Question: Construct a 95% confidence interval for p, the proportion of New York adults who supported a quarantine for anyone who has come into contact with an Ebola patient. Please output the upper bound of the confidence interval and round to the nearest thousandth.
Answer: 0.844
Topics: Statistics, Sampling
Context: Public health has improved and evolved, but has the public's knowledge changed with it? This data set rosling_responses.csv contains sample responses for two survey questions posed by Hans Rosling during lectures to a wide array of college-educated audiences.

Question: Compute the upper bound of a 95% confidence interval for the fraction of college-educated adults who answered the children_with_1_or_more_vaccination question correctly. Please round to the nearest thousandth.
Answer: 0.358
Topics: Statistics, Sampling
Context: We consider an experiment for patients who underwent cardiopulmonary resuscitation (CPR) for a heart attack and were subsequently admitted to a hospital. These patients were randomly divided into a treatment group where they received a blood thinner or the control group where they did not receive a blood thinner. The outcome variable of interest was whether the patients survived for at least 24 hours. The data is shown in cpr.csv.

Question: Calculate the lower bound of the 90% confidence interval of the difference for the survival rates in the CPR study. Please round to the nearest thousandth.
Answer: -0.026
Topics: Statistics, Confidence interval
Context: The results in fish_oil_18.csv summarize each of the health outcomes for an experiment where 12,933 subjects received a 1g fish oil supplement daily and 12,938 received a placebo daily. The experiment's duration was 5-years. The first row represents the treatment group and the second row represents the placebo group.

Variables
major_cardio_event - Major cardiovascular event. (Primary end point.)
cardio_event_expanded - Cardiovascular event in expanded composite endpoint.
myocardioal_infarction - Total myocardial infarction. (Heart attack.)
stroke - Total stroke.
cardio_death - Death from cardiovascular causes.
PCI - Percutaneous coronary intervention.
CABG - Coronary artery bypass graft.
total_coronary_heart_disease - Total coronary heart disease.
ischemic_stroke - Ischemic stroke.
hemorrhagic_stroke - Hemorrhagic stroke.
chd_death - Death from coronary heart disease.
myocardial_infarction_death - Death from myocardial infarction.
stroke_death - Death from stroke.
invasive_cancer - Invasive cancer of any type. (Primary end point.)
breast_cancer - Breast cancer.
prostate_cancer - Prostate cancer.
colorectal_cancer - Colorectal cancer.
cancer_death - Death from cancer.
death - Death from any cause.
major_cardio_event_after_2y - Major cardiovascular event, excluding the first 2 years of follow-up.
myocardial_infarction_after_2y - Total myocardial infarction, excluding the first 2 years of follow-up.
invasive_cancer_after_2y - Invasive cancer of any type, excluding the first 2 years of follow-up.
cancer_death_after_2y - Death from cancer, excluding the first 2 years of follow-up.
death_after_2y - Death from any cause, excluding the first 2 years of follow-up.

Question: Calculate the length of the 95% confidence interval for the effect of fish oils on heart attacks for patients who are well-represented by those in the study. Please round to the nearest thousandth.
Answer: 0.006
Topics: Statistics, Confidence interval

Question: Calculate the lower bound of the 95% confidence interval for the effect of fish oils on heart attacks for patients who are well-represented by those in the study. Please round to the nearest thousandth.
Answer: -0.007
Topics: Statistics, Confidence interval
Context: A mammogram is an X-ray procedure used to check for breast cancer. Whether mammograms should be used is part of a controversial discussion, and it's the topic of our next example where we learn about 2-proportion hypothesis tests when H0 is p1 = p2. A 30-year study was conducted with nearly 90,000 female participants. During a 5-year screening period, each woman was randomized to one of two groups: in the first group, women received regular mammograms to screen for breast cancer, and in the second group, women received regular non-mammogram breast cancer exams. No intervention was made during the following 25 years of the study, and we'll consider death resulting from breast cancer over the full 30-year period. Results from the study are in mammogram.csv.

If mammograms are much more effective than non-mammogram breast cancer exams, then we would expect to see additional deaths from breast cancer in the control group. On the other hand, if mammograms are not as effective as regular breast cancer exams, we would expect to see an increase in breast cancer deaths in the mammogram group.

Question: Is this study an experiment or an observational study? Please answer with "experiment" or "observational".
Answer: experiment
Topics: Statistics, Categorial data

Question: Compute the absolute value of the point estimate of the difference in breast cancer death rates in the two groups. Please round to 5 decimal places.
Answer: 0.00012
Topics: Statistics, Categorial data

Question: Compute the standard error of the point estimate of the difference in breast cancer death rates in the two groups. Please round to 5 decimal places.
Answer: 0.00070
Topics: Statistics, Categorial data

Question: Calculate a p-value for the hypothesis that there was no difference in breast cancer deaths in the mammogram and control groups. Please round to 4 decimal places.
Answer: 0.8650
Topics: Statistics, Categorial data
Context: A quadcopter company is considering a new manufacturer for rotor blades. The new manufacturer would be more expensive, but they claim their higher-quality blades are more reliable, with 3% more blades passing inspection than their competitor.
The quality control engineer collects a sample of blades, examining 1000 blades from each company, and the CSV file drone_blades.csv represents the quality control data set for quadcopter drone blades.

Question: Given the hypothesis that the higher-quality blades will pass inspection 3% more frequently than the standard-quality blades, calculate the p-value for the hypothesis. Please round to 3 decimal places.
Answer: 0.012
Topics: Statistics, Categorial data

Question: Given the hypothesis that the higher-quality blades will pass inspection 3% more frequently than the standard-quality blades, will you accept or reject the hypothesis with a significance level of 5%? Please answer with "accept" or "reject".
Answer: reject
Topics: Statistics, Categorial data
Context: The Stanford University Heart Transplant Study was conducted to determine whether an experimental heart transplant program increased lifespan. Each patient entering the program was designated an official heart transplant candidate, meaning that he was gravely ill and would most likely benefit from a new heart. Some patients got a transplant and some did not. The variable transplant indicates which group the patients were in; patients in the treatment group got a transplant and those in the control group did not. Of the 34 patients in the control group, 30 died. Of the 69 people in the treatment group, 45 died. Another variable called survived was used to indicate whether or not the patient was alive at the end of the study. The data is in the CSV file heart_transplant.csv.

Question: How much higher is the proportion of deaths in the control group than in the treatment group? Please round to the nearest hundredth.
Answer: 0.23
Topics: Statistics, Probability

Question: Suppose we are interested in estimating the difference in survival rate between the control and treatment groups using a confidence interval. Can we construct the interval using the normal approximation? Please answer with "yes" or "no".
Answer: no
Topics: Statistics, Categorial data
Context: The National Sleep Foundation conducted a survey on the sleep habits of randomly sampled transportation workers and a control sample of non-transportation workers. The results of the survey are shown in sleep_deprivation.csv.

Question: Given the hypothesis that there is no difference between the proportions of truck drivers and non-transportation workers (the control group) who get less than 6 hours of sleep per day, calculate the p-value for the hypothesis. Please round to 4 decimal places.
Answer: 0.0989
Topics: Statistics, Categorial data
Context: The data file jury.csv contains a random sample of 275 jurors in a small county. Jurors identified their racial group, and we would like to determine if these jurors are racially representative of the population. In the population, 72% are white, 7% are black, 12% are hispanic, and 9% are others.

Question: Given the null hypothesis that the jurors are a random sample, calculate the chi-square (X2) statistic. Please round to the nearest hundredth.
Answer: 5.89
Topics: Statistics, Categorial data, Chi-square