nils-herrmann/clean_scirepeval huggingface dataset:
Improving on Probability Weighting for Household Size

In survey sampling, inverse-probability weights are used to correct for unequal selection probabilities, and poststratification weights are used to correct for known or expected discrepancies between the sample and the population (see, e.g., Kish 1992). In this research note, we consider the effects of these adjustments for household size in telephone polling. In a survey in which households are sampled at random, and then a single individual is sampled from each sampled household, individuals in larger households have a smaller probability of being selected. The probability of an individual being included in the survey is inversely proportional to the size of the household in a simple random sample of households if individuals within a household are selected with equal probability and there is no nonresponse. However, composition of the sample is also affected by nonresponse. One source of nonresponse is nonavailabilityno one answers the phone, or no one receives the message on the answering machine. It seems reasonable to suppose that in a larger household it is more likely that someone will be home to receive the phone call. Another source of nonresponse is refusal to participate in the survey.
Index of authors, volume 60, 1998

Abramovich, F., Sapatinas, T. and Silverman, B. W. Wavelet thresholding via a Bayesian approach 725 Aitkin, M. Simpson's paradox and the Bayes factor 269 Anderson, C. W. See under Nadarajah, S. Bennett, K.  Estimation in multitype epidemics 663 Brown, P. J., Vannucci, M. and Fearn, T. Multivariate Bayesian variable selection and prediction 627 Chakraborty, B. and Chaudhuri, P. On an adaptive transformation±retransformation estimate of multivariate location 145 Chambers, R. L., Dorfman, A. H. and Wang, S. Limited information likelihood analysis of survey data 397 Chaudhuri, P. See under Chakraborty, B. Cheng, C.-L. and Schneeweiss, H. Polynomial regression with errors in the variables 189 Cheng, M.-Y. and Hall, P. Calibrating the excess mass and dip tests of modality 579 Clayton, D., Spiegelhalter, D., Dunn, G. and Pickles, A. Analysis of longitudinal binary data from multiphase sampling 71 Copas,  A. F. M. Automatic Bayesian curve ®tting 333 Dette, H. and Munk, A. Testing heteroscedasticity in nonparametric regression 693 Dette, H., Munk, A. and Wagner, T. Estimating the variance in nonparametric regressionÐ what is a reasonable choice? 751 Dorfman, A. H. See under Chambers, R. L. Dunn, G. See under Clayton, D. van Dyk, D. See under Meng, X.-L. Eguchi, S. and Copas, J. A class of local likelihood methods and near-parametric asymptotics 709 Fan, J., Farmen, M. and Gijbels, I. Local maximum likelihood estimation and inference 591 Farmen, M. See under Fan, J. Fearn, T. See under Brown, P. J. Firth, D. and Bennett, K. E. Robust models in probability sampling 3 Forster,  Hall, P. and Presnell, B. Nonparametric estimation of the mode of a distribution of random curves 681 Gijbels, I. See under Fan, J. Goldstein, H. See under Pfeermann, D. Goutis, C. Second-derivative functional regression with applications to near infra-red spectroscopy 103 Green, P.  Hall, P. See under Cheng, M.-Y. ÐSee under Gasser, Th. Haslett, J. and Hayes, K. Residuals for the linear model with general covariance structure 201 Hayes, K. See under Haslett, J. He, X., Ng, P. and Portnoy, S. Bivariate quantile smoothing splines 537 Heernan, P. M. Acknowledgement of priority: Unbiased estimation of central moments by using U-statistics 813
Generalizing the probability matrix decomposition model: An example of Bayesian model checking and model expansion

Probability matrix decomposition (PMD) models can be used to explain observed associations between two sets of elements. More speciically, observed associations are modeled as a deterministic function of B latent Bernoulli variables that are realized for each element. To estimate the parameters of this model, a sample of the posterior distribution is computed with a data augmentation algorithm. The obtained posterior sample can also be used to assess the t of the model with the technique of posterior predictive checks. In this paper a PMD model is applied to data on psychiatric diagnosis. In checking the model for this analysis, we focus on the appropriateness of the prior distribution for a set of latent parameters. Based on the posterior distribution for the values of the parameters corresponding to the observed data, we conclude that a relatively at prior distribution is inappropriate. In order to solve this problem, a mixture prior density with two beta distributed components is used to expand the model in a meaningful way.
Simulating Normalizing Constants: From Importance Sampling to Bridge Sampling to Path Sampling

Computing (ratios of) normalizing constants of probability models is a fundamental computational problem for many statistical and scientific studies. Monte Carlo simulation is an effective technique, es- pecially with complex and high-dimensional models. This paper aims to bring to the attention of general statistical audiences of some effective methods originating from theoretical physics and at the same time to ex- plore these methods from a more statistical perspective, through estab- lishing theoretical connections and illustrating their uses with statistical problems. We show that the acceptance ratio method and thermodynamic integration are natural generalizations of importance sampling, which is most familiar to statistical audiences. The former generalizes importance sampling through the use of a single "bridge" density and is thus a case of bridge sampling in the sense of Meng and Wong. Thermodynamic integration, which is also known in the numerical analysis literature as Ogata's method for high-dimensional integration, corresponds to the use of infinitely many and continuously connected bridges (and thus a "path"). Our path sampling formulation offers more flexibility and thus potential efficiency to thermodynamic integration, and the search of op- timal paths turns out to have close connections with the Jeffreys prior density and the Rao and Hellinger distances between two densities. We provide an informative theoretical example as well as two empirical ex- amples (involving 17- to 70-dimensional integrations) to illustrate the potential and implementation of path sampling. We also discuss some open problems.
General methods for monitoring convergence of iterative simulations

We generalize the method proposed by Gelman and Rubin (1992a) for monitoring the convergence of iterative simulations by comparing between and within variances of multiple chains, in order to obtain a family of tests for convergence. We review methods of inference from simulations in order to develop convergence-monitoring summaries that are relevant for the purposes for which the simulations are used. We recommend applying a battery of tests for mixing based on the comparison of inferences from individual sequences and from the mixture of sequences. Finally, we discuss multivariate analogues, for assessing convergence of several parameters simultaneously.
Student projects on statistical literacy and the media

An important theme in an introductory statistics course is the connection between statistics and the outside world. This article describes some assignments that have been useful in getting students to learn how to gather and process information presented in the newspaper articles and scientific reports they read. We discuss two related assignments. For the first kind of assignment, students work through prepared instructional packets. Each packet contains a newspaper article that reports on a scientific study or statistical analysis, the original report on which the article was based, a worksheet with guidelines for summarizing the reported study, and a series of questions. In the second kind of assignment, each student is required to find a newspaper article themselves, track down the original report, summarize the study using our guidelines, and write a critique of the article. Here, we describe the guidelines we developed to help the student in reading the newspaper article and original source...
Not Asked and Not Answered: Multiple Imputation for Multiple Surveys

We present a method of analyzing a series of independent cross-sectional surveys in which some questions are not answered in some surveys and some respondents do not answer some of the questions posed. The method is also applicable to a single survey in which different questions are asked or different sampling methods are used in different strata or clusters. Our method involves multiply imputing the missing items and questions by adding to existing methods of imputation designed for single surveys a hierarchical regression model that allows covariates at the individual and survey levels. Information from survey weights is exploited by including in the analysis the variables on which the weights were based, and then reweighting individual responses (observed and imputed) to estimate population quantities. We also develop diagnostics for checking the fit of the imputation model based on comparing imputed data to nonimputed data. We illustrate with the example that motivated this project: a study o...
Estimating the Probability of Events that Have Never Occured: When is Your Vote Decisive?

Researchers sometimes argue that statisticians have little to contribute when few realizations of the process being estimated are observed. We show that this argument is incorrect even in the extreme situation of estimating the probabilities of events so rare that they have never occurred. We show how statistical forecasting models allow us to use empirical data to improve inferences about the probabilities of these events. Our application is estimating the probability that your vote will be decisive in a U.S. presidential election, a problem that has been studied by political scientists for more than two decades. The exact value of this probability is of only minor interest, but the number has important implications for understanding the optimal allocation of campaign resources, whether states and voter groups receive their fair share of attention from prospective presidents, and how formal “rational choice” models of voter behavior might be able to explain why people vote at all. We show how th...
Poststrati cation Into Many Categories Using HierarchicalLogistic Regression

A standard method for correcting for unequal sampling probabilities and nonresponse in sample surveys is poststratiication: that is, dividing the population into several categories, estimating the distribution of responses in each category, and then counting each category in proportion to its size in the population. We consider poststratiication as a general framework that includes many weighting schemes used in survey analysis (see Little, 1993). We construct a hierarchical logistic regression model for the mean of a binary response variable conditional on poststratiication cells. The hierarchical model allows us to t many more cells than is possible using classical methods, and thus to include much more population-level information, while at the same time including all the information used in standard survey sampling inferences. We are thus combining the modeling approach often used in small-area estimation with the population information used in poststratiication. We apply the method to a set of U.S. pre-election polls, poststratiied by state as well as the usual demographic variables. We evaluate the models graphically by comparing to state-level election outcomes.
Weak convergence and optimal scaling of random walk Metropolis algorithms

This paper considers the problem of scaling the proposal distribution of a multidimensional random walk Metropolis algorithm in order to maximize the efficiency of the algorithm. The main result is a weak convergence result as the dimension of a sequence of target densities, n, converges to `. When the proposal variance is appropriately scaled according to n, the sequence of stochastic processes formed by the first component of each Markov chain converges to the appropriate limiting Langevin diffusion process. The limiting diffusion approximation admits a straightforward efficiency maximization problem, and the resulting asymptotically optimal policy is related to the asymptotic acceptance rate of proposed moves for the algorithm. The asymptotically optimal acceptance rate is 0.234 under quite general conditions. The main result is proved in the case where the target density has a symmetric product form. Extensions of the result are discussed.
Using Exams for Teaching Concepts in Probability and Statistics. Teacher's Corner.

We include a question at the end of the first midterm asking the student to guess his or her total score on the other questions of the exam. As an incentive, the student receives 5 points extra credit if the guess is within 10 points of the actual score (which is on a scale of 0 to 125). When the students complete their exams, we keep track of the order in which they are handed in, so that we can later check to see if students who finish the exam early are more or less accurate in their self-assessments than the students who take the full hour. When grading the exams, we do not look at the guessed score until all the other questions are graded. We then record the guessed grade, actual grade, and order of finish for each student. We have three reasons for including the self-evaluation question. First, it forces the students to check their work before turning in the exam. Second, it teaches them that subjective predictions can have systematic bias (in this case, students tend to be overconfident about their scores). And third, the students' guesses provide us with data for a class discussion, as described below.
Population toxicokinetics of tetrachloroethylene

In assessing the distribution and metabolism of toxic compounds in the body, measurements are not always feasible for ethical or technical reasons. Computer modeling offers a reasonable alternative, but the variability and complexity of biological systems pose unique challenges in model building and adjustment. Recent tools from population pharmacokinetics, Bayesian statistical inference, and physiological modeling can be brought together to solve these problems. As an example, we modeled the distribution and metabolism of tetrachloroethylene (PERC) in humans. We derive statistical distributions for the parameters of a physiological model of PERC, on the basis of data from Monster et al. (1979). The model adequately fits both prior physiological information and experimental data. An estimate of the relationship between PERC exposure and fraction metabolized is obtained. Our median population estimate for the fraction of inhaled tetrachloroethylene that is metabolized, at exposure levels exceeding current occupational standards, is 1.5% [95% confidence interval (0.52%, 4.1%)]. At levels approaching ambient inhalation exposure (0.001 ppm), the median estimate of the fraction metabolized is much higher, at 36% [95% confidence interval (15%, 58%)]. This disproportionality should be taken into account when deriving safe exposure limits for tetrachloroethylene and deserves to be verified by further experiments.
Physiological Pharmacokinetic Analysis Using Population Modeling and Informative Prior Distributions

We describe a general approach using Bayesian analysis for the estimation of parameters in physiological pharmacokinetic models. The chief statistical difficulty in estimation with these models is that any physiological model that is even approximately realistic will have a large number of parameters, often comparable to the number of observations in a typical pharmacokinetic experiment (e.g., 28 measurements and 15 parameters for each subject). In addition, the parameters are generally poorly identified, akin to the well-known ill-conditioned problem of estimating a mixture of declining exponentials. Our modeling includes (a) hierarchical population modeling, which allows partial pooling of information among different experimental subjects; (b) a pharmacokinetic model including compartments for well-perfused tissues, poorly perfused tissues, fat, and the liver; and (c) informative prior distributions for population parameters, which is possible because the parameters represent real physiological...
POSTERIOR PREDICTIVE ASSESSMENT OF MODEL FITNESS VIA REALIZED DISCREPANCIES

This paper considers Bayesian counterparts of the classical tests for good- ness of fit and their use in judging the fit of a single Bayesian model to the observed data. We focus on posterior predictive assessment, in a framework that also includes conditioning on auxiliary statistics. The Bayesian formulation facilitates the con- struction and calculation of a meaningful reference distribution not only for any (classical) statistic, but also for any parameter-dependent "statistic" or discrep- ancy. The latter allows us to propose the realized discrepancy assessment of model fitness, which directly measures the true discrepancy between data and the posited model, for any aspect of the model which we want to explore. The computation required for the realized discrepancy assessment is a straightforward byproduct of the posterior simulation used for the original Bayesian analysis. We illustrate with three applied examples. The first example, which serves mainly to motivate the work, illustrates the difficulty of classical tests in assessing the fitness of a Poisson model to a positron emission tomography image that is constrained to be nonnegative. The second and third examples illustrate the details of the posterior predictive approach in two problems: estimation in a model with inequality constraints on the parameters, and estimation in a mixture model. In all three examples, standard test statistics (either a χ 2 or a likelihood ratio) are not pivotal: the difficulty is not just how to compute the reference distribution for the test, but that in the classical framework no such distribution exists, independent of the unknown model parameters.
Bayesian prediction of mean indoor radon concentrations for Minnesota counties.

Past efforts to identify areas with higher than average indoor radon concentrations by examining the statistical relationship between local mean concentrations and physical parameters such as the soil radium concentration have been hampered by the variation in local means caused by the small number of homes monitored in most areas. In this paper, indoor radon data from a survey in Minnesota are analyzed to minimize the effect of finite sample size within counties, to determine the true county-to-county variation of indoor radon concentrations in the state, and to find the extent to which this variation is explained by the variation in surficial radium concentration among counties. The analysis uses hierarchical modeling, in which some parameters of interest (such as county geometric mean radon concentrations) are assumed to be drawn from a single population, for which the distributional parameters are estimated from the data. Extensions of this technique, known as random effects regression and mixed effects regression, are used to determine the relationship between predictive variables and indoor radon concentrations; the results are used to refine the predictions of each county's radon levels, resulting in a great decrease in uncertainty. The true county-to-county variation of geometric mean radon levels is found to be substantially less than the county-to-county variation of the observed geometric means, much of which is due to the small sample size in each county. The variation in the logarithm of surficial radium content is shown to explain approximately 80% of the variation of the logarithm of geometric mean radon concentration among counties. The influences of housing and measurement factors, such as whether the monitored home has a basement and whether the measurement was made in a basement, are also discussed. The statistical method can be used to predict mean radon concentrations, or applied to other geographically distributed environmental parameters.
Advantages of Conflictual Redistricting

This article describes the results of an analysis we did of state legislative elections in the United States, where each state is required to redraw the boundaries of its state legislative districts every ten years. In the United States, redistrictings are sometimes controlled by the Democrats, sometimes by Republicans, and sometimes by bipartisan committees, but never by neutral boundary commissions. Our goal was to study the consequences of redistricting; at the conclusion of this article, we discuss how our findings might be relevant to British elections. From George Washington's first presidential veto to the present day, redistricting issues have been extremely controversial at every level of government. Most redistrictings are contested in state and federal court cases heard so late that there is insufficient time to follow the usual rules of discovery, evidence, or due process. In total, legislative redistricting is one of the most conflictual forms of regular politics in the United States short of violence. While partisan and bipartisan redistricting plans can protect incumbents, they only protect some of those who survive the redistricting process, and many do not survive. Indeed, most incumbent politicians would give an awful lot to avoid redistricting altogether. After all, they are fighting over the fundamental rules of the game (fights that might well 1 This Chapter is adapted from Gelman and King 1994. Figures, tables, and more details appear in the original article.
Markov chain Monte Carlo methods in biostatistics

Appropriate models in biostatistics are often quite complicated. Such models are typically most easily fit using Bayesian methods, which can often be implemented using simulation techniques. Markov chain Monte Carlo (MCMC) methods are an important set of tools for such simulations. We give an overview and references of this rapidly emerging technology along with a relatively simple example. MCMC techniques can be viewed as extensions of iterative maximization techniques, but with random jumps rather than maximizations at each step. Special care is needed when implementing iterative maximization procedures rather than closed-form methods, and even more care is needed with iterative simulation procedures: it is substantially more difficult to monitor convergence to a distribution than to a point. The most reliable implementations of MCMC build upon results from simpler models fit using combinations of maximization algorithms and noniterative simulations, so that the user has a rough idea of the location and scale of the posterior distribution of the quantities of interest under the more complicated model. These concerns with implementation, however, should not deter the biostatistician from using MCMC methods, but rather help to ensure wise use of these powerful techniques.