bayesian analysis of dichotomous quantal response models abstract qualitative response models qrm s are analyzed from the bayesian point of view using diffuse and informative prior distributions exact finite sample bayesian and large sample bayesian and non bayesian estimation results are compared in addition the paper provides 1 plots and discussion of the properties of likelihood functions for qrm s 2 posterior distributions for logit models derivatives and elasticities 3 bayesian prediction procedures for qrm s 4 new estimates for the median and other fractiles of the logistic distribution 5 posterior odds ratios for model selection problems and 6 comparisons of two alternative monte carlo numerical integration procedures it is concluded that asymptotic approximations are not accurate for small to moderate sized samples even when only a single input variable is used and that operational bayesian methods are available for providing both exact small sample and large sample approximate inferences for drm s

an exact likelihood analysis of the multinomial probit model abstract we develop new methods for conducting a finite sample likelihood based analysis of the multinomial probit model using a variant of the gibbs sampler an algorithm is developed to draw from the exact posterior of the multinomial probit model with correlated errors this approach avoids direct evaluation of the likelihood and thus avoids the problems associated with calculating choice probabilities which affect both the standard likelihood and method of simulated moments approaches both simulated and actual consumer panel data are used to fit six dimensional choice models we also develop methods for analyzing random coefficient and multiperiod probit models

bayesian solutions to graduate admissions and related selection problems a graduate program has to decide how many offers of admission to make to achieve a targeted number of acceptances for its entering class given uncertainty about how many will accept such offers we first structure and solve selection problems under the assumption that probabilities associated with uncertain outcomes have assumed values using symmetric and asymmetric loss functions and show sensitivity of solutions to errors in these assumed values then we show how these problems can be solved when these probabilities are heterogeneous and have to be estimated from sample data

bayesian analysis of logit models using natural conjugate priors abstract this study provides a bayesian investigation of conditional logit and multinomial logit models based on a conjugate prior which has many appealing computational properties a one parameter neutral prior is developed which concentrates prior mass over the region corresponding to equiprobable alternatives this prior is particularly attractive in sensitivity analysis an empirical example of occupational choice illustrates the proposed techniques

computing bayes factors using a generalization of the savage dickey density ratio abstract we present a simple method for computing bayes factors the method derives from observing that in general a bayes factor can be written as the product of a quantity called the savage dickey density ratio and a correction factor both terms are easily estimated from posterior simulation in some cases it is possible to do these computations without ever evaluating the likelihood

maximum entropy and bayesian approaches to the ratio problem abstract maximum entropy and bayesian approaches provide superior estimates of a ratio of parameters as this paper illustrates using the classic nerlove model of agricultural supply providing extra information in the supports for the underlying parameters for generalized maximum entropy gme estimators or as an analytically derived prior distribution in zellner s minimum expected loss melo estimators and bayesian method of moments bmom estimators helps substantially simulations illustrate that gme melo and bmom estimators with conservative priors have much smaller mean square errors and average biases than do standard ordinary least squares or melo and bmom estimators with uninformative priors in addition a new estimator of the structural agricultural supply model provides estimates of parameters that cannot be obtained directly using traditional reduced form approaches

markov chains for exploring posterior distributions several markov chain methods are available for sampling from a posterior distribution two important examples are the gibbs sampler and the metropolis algorithm in addition several strategies are available for constructing hybrid algorithms this paper outlines some of the basic methods and strategies and discusses some related theoretical and practical issues on the theoretical side results from the theory of general state space markov chains can be used to obtain convergence rates laws of large numbers and central limit theorems for estimates obtained from markov chain methods these theoretical results can be used to guide the construction of more efficient algorithms for the practical use of markov chain methods standard simulation methodology provides several variance reduction techniques and also give guidance on the choice of sample size and allocation

posterior computations for censored regression data abstract this article describes the computation of and sampling from the posterior density for censored regression problems with normal and generalized log gamma errors the data augmentation algorithm tanner and wong 1987 is facilitated in the normal error case because of the form of the augmented posterior in the generalized log gamma context this simplicity is absent the work of sweeting 1981 is used as a motivation to develop an importance sampling scheme to sample from an augmented posterior it is shown how the predictive distribution for a new observation may be computed and sampled from the methodology is illustrated with two examples

using historical controls to adjust for covariates in trend tests for binary data abstract historical data often play an important role in helping interpret the results of a current study this article is motivated primarily by one specific application the analysis of data from rodent carcinogenicity studies by proposing a suitable informative prior distribution on the relationship between control outcome data and covariates we derive modified trend test statistics that incorporate historical control information to adjust for covariate effects frequentist and fully bayesian methods are presented and novel computational techniques are developed to compute the test statistics several attractive theoretical and computational properties of the proposed priors are derived in addition a semiautomatic elicitation scheme for the priors is developed our approach is used to modify a widely used prevalence test for carcinogenicity studies the proposed methodology is applied to data from a national toxicology program carcinogenicity experiment and is shown to provide helpful insight on t

calculation of maximum entropy distributions and approximation of marginalposterior distributions abstract it is shown how members of the broad class of cobb koppstein chen probability density functions can be generated by maximizing shannon s entropy subject to appropriate side conditions a particular member of the class the quartic exponential distribution is obtained by maximizing shannon s entropy subject to the conditions that the probability density function integrate to one and have given first four moments an algorithm for computing the quartic exponential distribution is provided and applied to approximate marginal posterior probability densities these approximations are compared with those provided by the pearson density function approximation procedure and exact densities obtained by numerical integration it is found that the maximum entropy method performs very well

combining historical and randomized controls for assessing trends in proportions abstract a statistical method for incorporating historical control data in the analysis of proportions is proposed and illustrated the method has as its extremes logistic regressions completely pooling and completely ignoring historical controls the degree of pooling used is determined by the variability from experiment to experiment in the control incidences the fit of historical control groups to an assumed normal logistic model is assessed using probability plotting techniques monte carlo studies evaluate the adequacy of the asymptotic approximation used sensitivity analyses show that results are insensitive to alternative priors the method is applied to several sets of tumor data from animal experiments

comparison of alternative functional forms in production abstract in this paper a bayesian approach to choosing between two non nested multivariate regression systems is developed the bayesian approach involves the calculation of the posterior probabilities of alternative hypotheses and formation of a posterior odds ratio specific results are obtained for the comparison of a multivariate regression and seemingly unrelated regression model these odds ratio results are applied to the choice between translog and fourier flexible form cost share equation systems when calculated for aggregate u s manufacturing cost data the odds ratios favor the fourier flexible form over the translog form even in small samples the fourier series expansion appears to be an effective approximation technique

econometric illustrations of novel numerical integration strategies for bayesian inference abstract recent progress with numerical techniques for implementing bayesian methods for irregular multi parameter likelihoods is reviewed and illustrated with two econometric examples the numerical integration strategies involve novel iterative adaptive uses of cartesian product and spherical rule quadrature formulae together with importance sampling techniques

a maximum entropy approach to recovering information from multinomial response data abstract the classical maximum entropy me approach to estimating the unknown parameters of a multinomial discrete choice problem which is equivalent to the maximum likelihood multinomial logit ml estimator is generalized the generalized maximum entropy gme model includes noise terms in the multinomial information constraints each noise term is modeled as the mean of a finite set of a priori known points in the interval 1 1 with unknown probabilities where no parametric assumptions about the error distribution are made a gme model for the multinomial probabilities and for the distributions associated with the noise terms is derived by maximizing the joint entropy of multinomial and noise distributions under the assumption of independence the gme formulation reduces to the me in the limit as the sample grows large or when no noise is included in the entropy maximization further even though the gme and the logit estimators are conceptually different the dual gme model is related to a gener

a bayesian multidimensional scaling procedure for the spatial analysis of revealed choice data we present a new bayesian formulation of a vector multidimensional scaling procedure for the spatial analysis of binary choice data the gibbs sampler is gainfully employed to estimate the posterior distribution of the specified scalar products bilinear model parameters the computational procedure allows for the explicit estimation of a covariance matrix which can accommodate violations of iia due to context effects in addition posterior standard errors can be estimated which reflect differential degrees of consumer choice uncertainty and or brand position instability a marketing application concerning the analysis of consumers consideration sets for luxury automobiles is provided to illustrate the use of the proposed methodology

the matrix logarithmic covariance model abstract a flexible method is introduced to model the structure of a covariance matrix c and study the dependence of the covariances on explanatory variables by observing that for any real symmetric matrix a the matrix exponential transformation c exp a is a positive definite matrix because there is no constraint on the possible values of the upper triangular elements on a any possible structure of interest can be imposed on them the method presented here is not intended to replace the existing special models available for a covariance matrix but rather to provide a broad range of further structures that supplements existing methodology maximum likelihood estimation procedures are used to estimate the parameters and the large sample asymptotic properties are obtained a simulation study and two real life examples are given to illustrate the method introduced

using multiple imputation with gee with non monotone missing longitudinal binary outcomes this paper considers multiple imputation mi approaches for handling non monotone missing longitudinal binary responses when estimating parameters of a marginal model using generalized estimating equations gee gee has been shown to yield consistent estimates of the regression parameters for a marginal model when data are missing completely at random mcar however when data are missing at random mar the gee estimates may not be consistent the mi approaches proposed in this paper minimize bias under mar the first mi approach proposed is based on a multivariate normal distribution but with the addition of pairwise products among the binary outcomes to the multivariate normal vector even though the multivariate normal does not impute 0 or 1 values for the missing binary responses as discussed by horton et al am stat 57 229 232 2003 we suggest not rounding when filling in the missing binary data because it could increase bias the second mi approach considered is the fully conditional specification fcs approach in this approach we specify a logistic regression model for each outcome given the outcomes at other time points and the covariates typically one would only include main effects of the outcome at the other times as predictors in the fcs approach but we explore if bias can be reduced by also including pairwise interactions of the outcomes at other time point in the fcs in a study of asymptotic bias with non monotone missing data the proposed mi approaches are also compared to gee without imputation finally the proposed methods are illustrated using data from a longitudinal clinical trial comparing four psychosocial treatments from the national institute on drug abuse collaborative cocaine treatment study where patients cocaine use is collected monthly for 6 months during treatment

three families of monomial functions with three valued walsh spectrum let bbb f p be a finite field with p elements where p is a prime let n ge 2 be an integer and d be the least positive integer satisfying p d equiv 1 pmod n let q p 2sd for some integers s in some special cases we obtain the explicit evaluation of the following exponential sums s a b sum x in bbb f q zeta p mathrm tr q p ax q 1 n bx as applications walsh spectrums of the monomial functions mathrm tr q p x q 1 n in three cases are investigated our results show that walsh spectrums of the monomial functions have at most four five or seven distinct values furthermore three families of the monomial functions with three valued walsh spectrums are presented see corollaries 12 21 31 and 32 consequently certain previously known results by li and yue and moisio are extended

a new perspective on priors for generalized linear models abstract this article deals with specifications of informative prior distributions for generalized linear models our emphasis is on specifying distributions for selected points on the regression surface the prior distribution on regression coefficients is induced from this specification we believe that it is inherently easier to think about conditional means of observables given the regression variables than it is to think about model dependent regression coefficients previous use of conditional means priors seems to be restricted to logistic regression with one predictor variable and to normal theory regression we expand on the idea of conditional means priors and extend these to arbitrary generalized linear models we also consider data augmentation priors where the prior is of the same form as the likelihood we show that data augmentation priors are special cases of conditional means priors with current monte carlo methodology such as importance sampling and gibbs sampling our priors result in

bayesian analysis of binary and polychotomous response data abstract a vast literature in statistics biometrics and econometrics is concerned with the analysis of binary and polychotomous response data the classical approach fits a categorical response regression model using maximum likelihood and inferences about the model are based on the associated asymptotic theory the accuracy of classical confidence statements is questionable for small sample sizes in this article exact bayesian methods for modeling categorical response data are developed using the idea of data augmentation the general approach can be summarized as follows the probit regression model for binary outcomes is seen to have an underlying normal regression structure on latent continuous data values of the latent data can be simulated from suitable truncated normal distributions if the latent data are known then the posterior distribution of the parameters can be computed using standard results for normal linear models draws from this posterior are used to sample new latent data and t

low rank separated representation surrogates of high dimensional stochastic functions application in bayesian inference this study introduces a non intrusive approach in the context of low rank separated representation to construct a surrogate of high dimensional stochastic functions e g pdes odes in order to decrease the computational cost of markov chain monte carlo simulations in bayesian inference the surrogate model is constructed via a regularized alternative least square regression with tikhonov regularization using a roughening matrix computing the gradient of the solution in conjunction with a perturbation based error indicator to detect optimal model complexities the model approximates a vector of a continuous solution at discrete values of a physical variable the required number of random realizations to achieve a successful approximation linearly depends on the function dimensionality the computational cost of the model construction is quadratic in the number of random inputs which potentially tackles the curse of dimensionality in high dimensional stochastic functions furthermore this vector valued separated representation based model in comparison to the available scalar valued case leads to a significant reduction in the cost of approximation by an order of magnitude equal to the vector size the performance of the method is studied through its application to three numerical examples including a 41 dimensional elliptic pde and a 21 dimensional cavity flow

bayesian marginal inference abstract a method is proposed for approximating the marginal posterior density of a continuous function of several unknown parameters thus permitting inferences about any parameter of interest for nonlinear models when the sample size is finite possibly tedious numerical integrations are replaced by conditional maximizations which are shown to be quite accurate in a number of special cases there are similarities with the profile likelihood ideas originated by kalbfleisch and sprott 1970 and the method is contrasted with a laplacian approximation recommended by kass tierney and kadane 1988 in press referred to here as the ktk procedure the methods are used to approximate the marginal posterior densities of the log linear interaction effects and an overall measure of association in a two way contingency table snee s 1974 hair eye color data are reanalyzed and adjustments are proposed to goodman s 1964 analysis for the full rank interaction model another application concerns marginaliz

on the existence of maximum likelihood estimators for the binomial response models summary necessary and sufficient conditions are given for the existence of maximum likelihood estimators of the linear regression parameter in binomial response this includes logit and probit models the question of existence of maximum likelihood estimators mle for logit models arose in an analysis of the relationship of psychiatric caseness to scores on a psychiatric screening questionnaire tennant 1977 administered the ghq general health questionnaire goldberg 1972 to 120 patients attending a general practitioner s surgery and also gave each one a standardized psychiatric interview from the interview patients were classified as psychiatric case non case in a secondary analysis of tennant s data duncan jones and henderson 1978 fitted a logit regression of caseness on ghq score and obtained a good fit with the model

on bayesian analysis of generalized linear models using jeffreys s prior abstract generalized linear models glm s have proved suitable for modeling various kinds of data consisting of exponential family response variables with covariates bayesian analysis of such data requires specification of a prior for the regression parameters in the model used uniform priors are very commonly used as conventional noninformative priors we show however that uniform priors for glm s can lead to improper posterior distributions thus making them undesirable alternative reference priors may be constructed from jeffreys s rule in this article we give two theorems that support the use of jeffreys s priors for glm s with intrinsically fixed or known scale parameters these theorems provide i sufficient and ii necessary and sufficient conditions for the propriety of the i posterior and ii prior distributions as well as for the existence of moments implications of these theorems for some commonly used glm s are discussed finally an illustrative example is given for the binomial

prior elicitation variable selection and bayesian computation for logistic regression models summary bayesian selection of variables is often difficult to carry out because of the challenge in specifying prior distributions for the regression parameters for all possible models specifying a prior distribution on the model space and computations we address these three issues for the logistic regression model for the first we propose an informative prior distribution for variable selection several theoretical and computational properties of the prior are derived and illustrated with several examples for the second we propose a method for specifying an informative prior on the model space and for the third we propose novel methods for computing the marginal distribution of the data the new computational algorithms only require gibbs samples from the full model to facilitate the computation of the prior and posterior model probabilities for all possible models several properties of the algorithms are also derived the prior specification for the first challenge focuses on the observables in that the elicitation is based on a prior prediction yo for the response vector and a quantity ao quantifying the uncertainty in yo then yo and ao are used to specify a prior for the regression coefficients semi automatically examples using real data are given to demonstrate the methodology

assessment and propagation of model uncertainty in most examples of inference and prediction the expression of uncertainty about unknown quantities y on the basis of known quantities x is based on a model m that formalizes assumptions about how x and y are related m will typically have two parts structural assumptions s such as the form of the link function and the choice of error distribution in a generalized linear model and parameters whose meaning is specific to a given choice of s it is common in statistical theory and practice to acknowledge parametric uncertainty about given a particular assumed structure s it is less common to acknowledge structural uncertainty about s itself a widely used approach involves enlisting the aid of x to specify a plausible single best choice s for s and then proceeding as if s were known to be correct in general this approach fails to assess and propagate structural uncertainty fully and may lead to miscalibrated uncertainty assessments about y given x when miscalibration occurs it will often result in understatement of inferential or predictive uncertainty about y leading to inaccurate scientific summaries and overconfident decisions that do not incorporate sufficient hedging against uncertainty in this paper i discuss a bayesian approach to solving this problem that has long been available in principle but is only now becoming routinely feasible by virtue of recent computational advances and examine its implementation in examples that involve forecasting the price of oil and estimating the chance of catastrophic failure of the us space shuttle

bayesian inference in econometric models using monte carlo integration methods for the systematic application of monte carlo integration with importance sampling to bayesian inference in econometric models are developed conditions under which the numerical approximation of a posterior moment converges almost surely to the true value as the number of monte carlo replications increases and the numerical accuracy of this approximation may be assessed reliably are set forth methods for the analytical verification of these conditions are discussed

bayesian methods for variance component models abstract an exact bayesian analysis can be performed for normal theory variance components models using importance sampling but remarkably accurate approximations are available using the laplacian t approximation introduced by leonard hsu and ritter instead of maximizing the joint posterior density conditional upon the parameter of interest a device described by o hagan is used first the bayesian estimators are compared to the lindley stein shrinkage estimators and the lindley smith joint modal estimators it is confirmed that joint modes can overcollapse toward prior hypotheses when compared with more sensible bayesian procedures this is referred to as a collapsing phenomenon a numerical example from the one way random effects model is considered and the risks of the different estimators are simulated under a variety of loss functions it is concluded that although the lindley stein estimator performs well a full hierarchical bayesian analysis performs at least equally well while permitti

bayes factors for nonlinear hypotheses and likelihood distributions summary new methods are proposed which allow bayes factors to be computed for hypotheses which impose nonlinear restrictions on the parameters projection methods are used to induce the prior distribution over the restricted parameter space which is required for computation of the bayes factor various distance metrics are introduced to define the projection including a utility based metric which gives kullback leibler divergence as a special case draws from the restricted and unrestricted prior distributions are used to construct marginal distributions of the likelihood which is shown to have additional diagnostic value over and above the bayes factor these methods are applied to hypotheses statisticians are frequently asked to entertain hypotheses which are nonlinear in the model parameters nonlinear hypotheses arise both from subject matter theory as well as from the process of model selection to address nonlinear hypotheses researchers use frequentist methods which are usually implemented using asymptotic approximations berger 1987 zellner 1971 1978 1984 and others have discussed the weaknesses of the classical significance testing approach and suggest bayesian alternatives such as odds ratios the odds ratio is the ratio of posterior probabilities of two competing hypotheses and is the product of the prior odds ratio and the bayes factor the odds ratio has the conceptual advantages of providing a natural probability metric for assessing the strength of data and prior beliefs because of the difficulties of assessing priors under the nonlinear restrictions and the computation of the required integrals few bayes factors have been computed for nonlinear hypotheses see lindley 1988 for an exception in the present paper we outline a general method for computing bayes factors which is flexible conceptually appealing and computationally tractable the literature on bayes factors starts with the work of jeffreys 1961 other references include berger delampady 1987 dickey 1971 good 1985 kass vaidyanathan 1992 smith spiegelhalter 1980 and zellner 1971 1978 1984 these papers deal with the testing of linear restrictions on the parameter space often in the context of linear models even the widely used schwarz 1978 model selection criterion considers only linear restrictions in models of the exponential family form the key difficulty which has kept researchers from developing bayes factors for nonlinear hypotheses is the

bayes inference in the tobit censored regression model we consider the bayes estimation of the tobit censored regression model with normally distributed errors a simple condition for the existence of posterior moments is provided suitable versions of monte carlo procedures based on symmetric multivariate t distributions and laplacian approximations in a certain parametrization are developed and illustrated ideas involving data augmentation and gibbs sampling cf tanner and wong 1987 and gelfand and smith 1990 are also developed the methods are compared in two examples with diffuse priors and various combinations of sample sizes and degrees of censoring

monte carlo evaluation of multivariate normal probabilities abstract this paper extends research on the simulation of multivariate normal probabilities of high order dimension by developing a new family of simulators such simulators are useful for models with limited dependent variables including multinomial probit in panel studies spatial analysis and time series analysis the simulators are derived from a cholesky decomposition of the covariance matrix combined with a suitable choice of an importance sampling distribution the paper studies among others the impact of antithetical sampling the insights gained in this paper are of use in bayesian analysis as well in the evaluation of posterior densities

jeffreys prior for logit models abstract this paper investigates jeffreys prior for logit models with covariates in conditional logit models it is not recommended and in multinomial logit models it has properties similar to a neutral natural conjugate prior but none of the computational advantages

bounds for the covariance matrices of zellner s estimator in the sur model and the 2sae in a heteroscedastic model abstract this article gives a bound for the covariance matrix of zellner s estimator with the unrestricted sample covariance matrix in the two equations seemingly unrelated regression sur model and a bound for the covariance matrix of the two stage aitken estimator 2sae in a heteroscedastic model with two distinct variances some efficiencies of these estimators relative to the ordinary least squares estimator olse are also considered

the calculation of posterior distributions by data augmentation abstract the idea of data augmentation arises naturally in missing value problems as exemplified by the standard ways of filling in missing cells in balanced two way tables thus data augmentation refers to a scheme of augmenting the observed data so as to make it more easy to analyze this device is used to great advantage by the em algorithm dempster laird and rubin 1977 in solving maximum likelihood problems in situations when the likelihood cannot be approximated closely by the normal likelihood maximum likelihood estimates and the associated standard errors cannot be relied upon to make valid inferential statements from the bayesian point of view one must now calculate the posterior distribution of parameters of interest if data augmentation can be used in the calculation of the maximum likelihood estimate then in the same cases one ought to be able to use it in the computation of the posterior distribution it is the purpose of this article to explain how this can be done the basic idea