bayesian analysis of dichotomous quantal response models abstract qualitative response models qrm s are analyzed from the bayesian point of view using diffuse and informative prior distributions exact finite sample bayesian and large sample bayesian and non bayesian estimation results are compared in addition the paper provides 1 plots and discussion of the properties of likelihood functions for qrm s 2 posterior distributions for logit models derivatives and elasticities 3 bayesian prediction procedures for qrm s 4 new estimates for the median and other fractiles of the logistic distribution 5 posterior odds ratios for model selection problems and 6 comparisons of two alternative monte carlo numerical integration procedures it is concluded that asymptotic approximations are not accurate for small to moderate sized samples even when only a single input variable is used and that operational bayesian methods are available for providing both exact small sample and large sample approximate inferences for drm s

some aspects of polya tree distributions for statistical modelling the definition and elementary properties of polya tree distributions are reviewed two theorems are presented showing that polya trees can be constructed to concentrate arbitrarily closely about any desired pdf and that polya tree priors can put positive mass in every relative entropy neighborhood of every positive density with finite entropy thereby satisfying a consistency condition such theorems are false for dirichlet processes models are constructed combining partially specified polya trees with other information such as monotonicity or unimodality it is shown how to compute bounds on posterior expectations over the class of all priors with the given specifications a numerical example is given a theorem of diaconis and freedman about dirichlet processes is generalized to polya trees allowing polya trees to be the models for errors in regression problems finally empirical bayes models using dirichlet processes are generalized to polya trees an example from berry and christensen is reanalyzed with a polya tree model

bayesian computation via the gibbs sampler and related markov chain monte carlo methods the use of the gibbs sampler for bayesian computation is reviewed and illustrated in the context of some canonical examples other markov chain monte carlo simulation methods are also briefly described and comments are made on the advantages of sample based approaches for bayesian inference summaries

computing bayes factors using a generalization of the savage dickey density ratio abstract we present a simple method for computing bayes factors the method derives from observing that in general a bayes factor can be written as the product of a quantity called the savage dickey density ratio and a correction factor both terms are easily estimated from posterior simulation in some cases it is possible to do these computations without ever evaluating the likelihood

globally optimal linear precoders for finite alphabet signals over complex vector gaussian channels we study the design optimization of linear precoders for maximizing the mutual information between finite alphabet input and the corresponding output over complex valued vector channels this mutual information is a nonlinear and non concave function of the precoder parameters posing a major obstacle to precoder design optimization our work presents three main contributions first we prove that the mutual information is a concave function of a matrix which itself is a quadratic function of the precoder matrix second we propose a parameterized iterative algorithm for finding optimal linear precoders to achieve the global maximum of the mutual information the proposed iterative algorithm is numerically robust computationally efficient and globally convergent third we demonstrate that maximizing the mutual information between a discrete constellation input and the corresponding output of a vector channel not only provides the highest practically achievable rate but also serves as an excellent criterion for minimizing the coded bit error rate our numerical examples show that the proposed algorithm achieves mutual information very close to the channel capacity for channel coding rate under 0 75 and also exhibits a large gain over existing linear precoding and or power allocation algorithms moreover our examples show that certain existing methods are susceptible to being trapped at locally optimal precoders

on the consistency of bayes estimates on etudie les proprietes de frequence des regles de bayes avec une attention particuliere a la consistence

capacity of the discrete time gaussian channel with intersymbol interference the discrete time gaussian channel with intersymbol interference isi where the inputs are subject to a per symbol average energy constraint is considered the capacity of this channel is derived by means of a hypothetical channel model called the n circular gaussian channel ncgc whose capacity is readily derived using the theory of the discrete fourier transform the results obtained for the ncgc are used further to prove that in the limit of increasing block length n the capacity of the discrete time gaussian channel dtgc with isi using a per block average energy input constraint n block dtgc is indeed also the capacity when using the per symbol average energy constraint

multivariate adaptive regression splines a new method is presented for flexible regression modeling of high dimensional data the model takes the form of an expansion in product spline basis functions where the number of basis functions as well as the parameters associated with each one product degree and knot locations are automatically determined by the data this procedure is motivated by the recursive partitioning approach to regression and shares its attractive properties unlike recursive partitioning however this method produces continuous models with continuous derivatives it has more power and flexibility to model relationships that are nearly additive or involve interactions in at most a few variables in addition the model can be represented in a form that separately identifies the additive contributions and those associated with the different multivariable interactions

easily computed lower bounds on the information rate of intersymbol interference channels provable lower bounds are presented for the information rate i x x s n where x is the symbol drawn independently and uniformly from a finite size alphabet s is a discrete valued random variable rv and n is a gaussian rv it is well known that with s representing the precursor intersymbol interference isi at the decision feedback equalizer dfe output i x x s n serves as a tight lower bound for the symmetric information rate sir as well as capacity of the isi channel corrupted by gaussian noise when evaluated on a number of well known finite isi channels these new bounds provide a very similar level of tightness against the sir to the conjectured lower bound by shamai and laroia at all signal to noise ratio snr ranges while being actually tighter when viewed closed up at high snrs the new lower bounds are obtained in two steps first a mismatched mutual information function is introduced which can be proved as a lower bound to i x x s n secondly this function is further bounded from below by an expression that can be computed easily via a few single dimensional integrations with a small computational load

dimension free quasiconformal distortion in n space most distortion theorems for k quasiconformal mappings in rn n 2 depend on both n and k in an essential way with bounds that become infinite as n tends to oo the present authors obtain dimension free versions of four well known distortion theorems for quasiconformal mappings namely bounds for the linear dilatation the schwarz lemma the e distortion theorem and the r1 quasisymmetry property of these mappings they show that the upper estimates they have obtained in each of these four main results remain bounded as n tends to oo with k fixed the proofs are based on a dimensioncancellation property of the function t h r 1 r t k t 0 k 0 where r t is the capacity of a teichmiiller extremal ring in rn the authors also prove a dimension free distortion theorem for the absolute cross ratio under k quasiconformal mappings of n from which several other distortion theorems follow as special cases

mmse decision feedback equalizers and coding ii coding results for pt i see ibid vol 43 no 10 p 2582 1995 the minimum mean squared error decision feedback equalizer mmse dfe has properties that suggest that it is a canonical equalization structure in systems that combine equalization with coded modulation with a given symbol rate 1 t and transmit spectrum the output signal to noise ratio snr sub mmse dfe u of a mmse dfe with an unbiased decision rule is a single parameter that characterizes the channel for coding purposes indeed the transmit spectrum that maximizes snr sub mmse dfe u is the capacity achieving water pouring spectrum and the capacity c t in bits per two dimensions is given by c t log sub 2 1 snr sub mmse dfe u regardless of the channel characteristics the performance of a coded system with a mmse dfe equalization structure may be accurately estimated using the gain of the coding scheme at a given pr e this performance is shown to be approximately the same as that of a multicarrier system using the same transmit spectrum and similar coding such systems are known to be able to approach capacity arbitrarily closely the mmse dfe can perform significantly better than a zero forcing decision feedback equalizer particularly at moderate to low snr s and on severe isi channels simulation results indicate that performance of the mmse dfe is surprisingly insensitive to transmit spectral shaping as long as the transmit spectrum exceeds the capacity achieving band but that there is an optimal symbol rate that should approximately be used

limited information estimators and exogeneity tests for simultaneous probit models a two step maximum likelihood procedure is proposed for estimating simultaneous probit models and is compared to alternative limited information estimators conditions under which each estimator attains the cramer rao lower bound are obtained simple tests for exogeneity based on the new two step estimator are proposed and are shown to be asymptotically equivalent to one another and to have the same local asymptotic power as classical tests based on the limited information maximum likelihood estimator finite sample comparisons between the new and alternative estimators are presented based on some monte carlo evidence the performance of the proposed tests for exogeneity is also assessed

simulation based computation of information rates for channels with memory the information rate of finite state source channel models can be accurately estimated by sampling both a long channel input sequence and the corresponding channel output sequence followed by a forward sum product recursion on the joint source channel trellis this method is extended to compute upper and lower bounds on the information rate of very general channels with memory by means of finite state approximations further upper and lower bounds can be computed by reduced state methods

inverse groundwater modeling for hydraulic conductivity estimation using bayesian model averaging and variance window 1 this study proposes a bayesian model averaging bma method to address parameter estimation uncertainty arising from nonuniqueness in parameterization methods bma is able to incorporate multiple parameterization methods for prediction through the law of total probability and to obtain an ensemble average of hydraulic conductivity estimates two major issues in applying bma to hydraulic conductivity estimation are discussed the first problem is using occam s window in usual bma applications to measure approximated posterior model probabilities occam s window only accepts models in a very narrow range tending to single out the best method and discard other good methods we propose a variance window to replace occam s window to cope with this problem the second problem is the kashyap information criterion kic in the approximated posterior model probabilities which tends to prefer highly uncertain parameterization methods by considering the fisher information matrix with sufficient amounts of observation data the bayesian information criterion bic is a good approximation and is able to avoid controversial results from using kic this study adopts multiple generalized parameterization gp methods such as the bma models to estimate spatially correlated hydraulic conductivity numerical examples illustrate the issues of using kic and occam s window and show the advantages of using bic and the variance window in bma application finally we apply bma to the hydraulic conductivity estimation of the 1500 foot sand in east baton rouge parish louisiana

sampling based approaches to calculating marginal densities abstract stochastic substitution the gibbs sampler and the sampling importance resampling algorithm can be viewed as three alternative sampling or monte carlo based approaches to the calculation of numerical estimates of marginal probability distributions the three approaches will be reviewed compared and contrasted in relation to various joint probability structures frequently encountered in applications in particular the relevance of the approaches to calculating bayesian posterior densities for a variety of structured models will be discussed and illustrated

estimation in gaussian noise properties of the minimum mean square error consider the minimum mean square error mmse of estimating an arbitrary random variable from its observation contaminated by gaussian noise the mmse can be regarded as a function of the signal to noise ratio snr as well as a functional of the input distribution of the random variable to be estimated it is shown that the mmse is concave in the input distribution at any given snr for a given input distribution the mmse is found to be infinitely differentiable at all positive snr and in fact a real analytic function in snr under mild conditions the key to these regularity results is that the posterior distribution conditioned on the observation through gaussian channels always decays at least as quickly as some gaussian density furthermore simple expressions for the first three derivatives of the mmse with respect to the snr are obtained it is also shown that as functions of the snr the curves for the mmse of a gaussian input and that of a non gaussian input cross at most once over all snrs these properties lead to simple proofs of the facts that gaussian inputs achieve both the secrecy capacity of scalar gaussian wiretap channels and the capacity of scalar gaussian broadcast channels as well as a simple proof of the entropy power inequality in the special case where one of the variables is gaussian

stochastic complexity and modeling on demontre un theoreme fondamental qui donne une borne inferieure pour la longueur de code et donc pour les erreurs de prediction on definit les notions d information a priori et d information utile dans les donnees

the calculation of posterior distributions by data augmentation abstract the idea of data augmentation arises naturally in missing value problems as exemplified by the standard ways of filling in missing cells in balanced two way tables thus data augmentation refers to a scheme of augmenting the observed data so as to make it more easy to analyze this device is used to great advantage by the em algorithm dempster laird and rubin 1977 in solving maximum likelihood problems in situations when the likelihood cannot be approximated closely by the normal likelihood maximum likelihood estimates and the associated standard errors cannot be relied upon to make valid inferential statements from the bayesian point of view one must now calculate the posterior distribution of parameters of interest if data augmentation can be used in the calculation of the maximum likelihood estimate then in the same cases one ought to be able to use it in the computation of the posterior distribution it is the purpose of this article to explain how this can be done the basic idea

bayesian inference in a simultaneous equation model with limited dependent variables abstract this paper develops a bayesian approach for inference in a simultaneous equation model with limited dependent variables sldv by employing a combination of gibbs sampling and data augmentation we can draw from the exact posterior of this sldv model and avoid direct evaluation of the non trivial likelihood function a by product from our posterior simulation is the savage dickey density ratio which is used for computing the bayes factor the practicality and efficiency of the proposed method are illustrated through an example in corporate finance

the hierarchical logistic regression model for multilevel analysis abstract a hierarchical logistic regression model is proposed for studying data with group structure and a binary response variable the group structure is defined by the presence of micro observations embedded within contexts macro observations and the specification is at both of these levels at the first micro level the usual logistic regression model is defined for each context the same regressors are used in each context but the micro regression coefficients are free to vary over contexts at the second level the micro coefficients are treated as functions of macro regressors an empirical bayes estimation procedure is proposed for estimating the micro and macro coefficients explicit formulas are provided that are computationally feasible for large scale data analyses these include an algorithm for finding the maximum likelihood estimates of the covariance components representing within and between macro equation error variability the methodology is applied to world fertility survey data

bayesian inference in a correlated random coefficients model modeling causal effect heterogeneity with an application to heterogeneous returns to schooling abstract we consider the problem of causal effect heterogeneity from a bayesian point of view this is accomplished by introducing a three equation system similar in spirit to the work of heckman and vytlacil 1998 describing the joint determination of a scalar outcome an endogenous treatment variable and an individual specific causal return to that treatment we describe a bayesian posterior simulator for fitting this model which recovers far more than the average causal effect in the population the object which has been the focus of most previous work parameter identification and generalized methods for flexibly modeling the outcome and return heterogeneity distributions are also discussed combining data sets from high school and beyond hsb and the 1980 census we illustrate our methods in practice and investigate heterogeneity in returns to education our analysis decomposes the impact of key hsb covariates on log wages into three parts a direct effect and two separate indirect effects through educational attainment and returns to education our results strongly suggest that the quantity of schooling attained is determined at least in part by the individual s own return to education specifically a one percentage point increase in the return to schooling parameter is associated with the receipt of approximately 0 14 more years of schooling furthermore when we control for variation in returns to education across individuals we find no difference in predicted schooling levels for men and women however women are predicted to attain approximately 1 4 of a year more schooling than men on average as a result of higher rates of return to investments in education

spectral analysis of fractionally spaced mmse equalizers and stability of the lms algorithm in the communication systems that use a linear modulation scheme for transmission the fractionally spaced fs samples of the received signal constitute a wide sense cyclostationary time series hence the standard fourier transform techniques cannot be used to study the spectral characteristics of the received fs samples or to derive the transfer function tf of the corresponding digital minimum mean square error mmse receiver in this paper an analytical expression for the tf of the fs mmse equalizer is derived which includes the effects of the continuous time to discrete time c d converter used at the receiver front end using this tf the sources of instability of the fs least mean square lms algorithm and the effects of the equalizer length and sampling phase on convergence of the lms algorithm are explained for stabilization of the fs lms algorithm conditions on the front end c d converter are provided such that when satisfied the lms algorithm becomes more stable and the learning characteristics of the modified receiver are better than the leaky fs lms algorithm theoretical results are corroborated by simulations

bayesian inference in a sample selection model this paper develops methods of bayesian inference in a sample selection model the main feature of this model is that the outcome variable is only partially observed we first present a gibbs sampling algorithm for a model in which the selection and outcome errors are normally distributed the algorithm is then extended to analyze models that are characterized by nonnormality specifically we use a dirichlet process prior and model the distribution of the unobservables as a mixture of normal distributions with a random number of components the posterior distribution in this model can simultaneously detect the presence of selection effects and departures from normality our methods are illustrated using some simulated data and an abstract from the rand health insurance experiment

ber minimized ofdm systems with channel independent precoders we consider the minimization of uncoded bit error rate ber for the orthogonal frequency division multiplexing ofdm system with an orthogonal precoder we analyze the ber performance of precoded ofdm systems with zero forcing and minimum mean squared error mmse receivers in the case of mmse receivers we show that for quadrature phase shift keying qpsk there exists a class of optimal precoders that are channel independent examples of this class include the discrete fourier transform dft matrix and the hadamard matrix when the precoder is the dft matrix the resulting optimal transceiver becomes the single carrier system with cyclic prefix sc cp system we also show that the worst solution corresponds to the conventional ofdm system the conventional ofdm system has the largest ber in the case of zero forcing receivers the design of optimal transceiver depends on the signal to noise ratio snr for higher snr solutions of optimal precoders are the same as those of mmse receivers

bayesian analysis of binary and polychotomous response data abstract a vast literature in statistics biometrics and econometrics is concerned with the analysis of binary and polychotomous response data the classical approach fits a categorical response regression model using maximum likelihood and inferences about the model are based on the associated asymptotic theory the accuracy of classical confidence statements is questionable for small sample sizes in this article exact bayesian methods for modeling categorical response data are developed using the idea of data augmentation the general approach can be summarized as follows the probit regression model for binary outcomes is seen to have an underlying normal regression structure on latent continuous data values of the latent data can be simulated from suitable truncated normal distributions if the latent data are known then the posterior distribution of the parameters can be computed using standard results for normal linear models draws from this posterior are used to sample new latent data and t

the intersymbol interference channel lower bounds on capacity and channel precoding loss the discrete time additive gaussian intersymbol interference isi channel with i i d not necessarily gaussian input signals is considered several new and old lower bounds on the capacity are derived in a unified manner by assuming different front end receiver filters in particular the sampled whitened matched filter swmf and the minimum mean squared error decision feedback equalizer mmse dfe filter the features of the bounds are demonstrated and compared in several examples with binary and quaternary input signals it is also shown that the effect of an ideal post cursor or tail cancellation in an information preserving context depends primarily on the front end filter while as is well known ideal post cursor cancellation at the output of the swmf decreases the information the opposite trend is seen when an mmse dfe front filter is considered this observation reflects the basic theoretical obstacles in precoding i e ideal post cursor cancellation in the presence of a pre cursor it is used to assess the inherent loss in terms of information rates as compared to the rates achievable with the hypothetical ideal post cursor cancellation associated with any post cursor cancellation technique such as precoding dfe or other variants when operating in synergy with the mmse dfe front end filter the effect of the front end filter on an ideally interleaved precoded coded modulation system is also addressed

calibration based empirical probability probability forecasts for a sequence of uncertain events may be compared with the outcomes of those events by means of a natural criterion of empirical validity calibration it is shown that any two sequences of forecasts which both meet this criterion must be in asymptotic agreement these agreed values can then be considered as correct objective probability forecasts for the particular sequence of outcome results obtained however the objective forecasts vary with the extent of the information taken into account when they are formulated we thus obtain a general theory of empirical probability relative to an information base this theory does not require that such probabilities be interpreted in terms of repeated trials of the same event some implications of this theory are discussed

hessian and concavity of mutual information differential entropy and entropy power in linear vector gaussian channels within the framework of linear vector gaussian channels with arbitrary signaling the jacobian of the minimum mean square error and fisher information matrices with respect to arbitrary parameters of the system are calculated in this paper capitalizing on prior research where the minimum mean square error and fisher information matrices were linked to information theoretic quantities through differentiation the hessian of the mutual information and the entropy are derived these expressions are then used to assess the concavity properties of mutual information and entropy under different channel conditions and also to derive a multivariate version of an entropy power inequality due to costa

multimodel bayesian analysis of groundwater data worth we explore the way in which uncertain descriptions of aquifer heterogeneity and groundwater flow impact one s ability to assess the worth of collecting additional data we do so on the basis of maximum likelihood bayesian model averaging mlbma by accounting jointly for uncertainties in geostatistical and flow model structures and parameter hydraulic conductivity as well as system state hydraulic head estimates given uncertain measurements of one or both variables previous description of our approach was limited to geostatistical models based solely on hydraulic conductivity data here we implement the approach on a synthetic example of steady state flow in a two dimensional random log hydraulic conductivity field with and without recharge by embedding an inverse stochastic moment solution of groundwater flow in mlbma a moment equations based geostatistical inversion method is utilized to circumvent the need for computationally expensive numerical monte carlo simulations the approach is compatible with either deterministic or stochastic flow models and consistent with modern statistical methods of parameter estimation admitting but not requiring prior information about the parameters it allows but does not require approximating lead predictive statistical moments of system states by linearization while updating model posterior probabilities and parameter estimates on the basis of potential new data both before and after such data are actually collected

mutual information and minimum mean square error in gaussian channels this paper deals with arbitrarily distributed finite power input signals observed through an additive gaussian noise channel it shows a new formula that connects the input output mutual information and the minimum mean square error mmse achievable by optimal estimation of the input given the output that is the derivative of the mutual information nats with respect to the signal to noise ratio snr is equal to half the mmse regardless of the input statistics this relationship holds for both scalar and vector signals as well as for discrete time and continuous time noncausal mmse estimation this fundamental information theoretic result has an unexpected consequence in continuous time nonlinear estimation for any input signal with finite power the causal filtering mmse achieved at snr is equal to the average value of the noncausal smoothing mmse achieved with a channel whose snr is chosen uniformly distributed between 0 and snr

marginal likelihood from the gibbs output abstract in the context of bayes estimation via gibbs sampling with or without data augmentation a simple approach is developed for computing the marginal density of the sample data marginal likelihood given parameter draws from the posterior distribution consequently bayes factors for model comparisons can be routinely computed as a by product of the simulation hitherto this calculation has proved extremely challenging our approach exploits the fact that the marginal density can be expressed as the prior times the likelihood function over the posterior density this simple identity holds for any parameter value an estimate of the posterior density is shown to be available if all complete conditional densities used in the gibbs sampler have closed form expressions to improve accuracy the posterior density is estimated at a high density point and the numerical standard error of resulting estimate is derived the ideas are applied to probit regression and finite mixture models

mmse decision feedback equalizers and coding i equalization results the minimum mean squared error decision feedback equalizer mmse dfe has properties that suggest that it is a canonical equalization structure for systems that combine equalization with coded modulation the structure and performance of the mmse dfe are succinctly derived using linear estimation theoretic principles in this first part of this two part paper the front end of the mmse dfe called the mean square whitened matched filter ms wmf is preferable in some ways to a matched filter or a whitened matched filter as a canonical receiver front end in a coded system the feedback filter of the mmse dfe may be implemented in the transmitter using precoding the mmse dfe can perform significantly better than a zero forcing decision feedback equalizer particularly at moderate to low snr s and on severe isi channels the mmse dfe is biased the optimum unbiased mmse dfe is the mmse dfe with the bias removed removing bias improves error probability but reduces the snr to snr sub mmse dfe u snr sub mmse dfe 1 it is shown that this snr relationship is a particular case of a very general result and that snr sub mmse dfe u gives a more realistic estimate of snr the results are extended to partial response equalization and to equalization with correlated inputs in an appendix

assessment and propagation of model uncertainty in most examples of inference and prediction the expression of uncertainty about unknown quantities y on the basis of known quantities x is based on a model m that formalizes assumptions about how x and y are related m will typically have two parts structural assumptions s such as the form of the link function and the choice of error distribution in a generalized linear model and parameters whose meaning is specific to a given choice of s it is common in statistical theory and practice to acknowledge parametric uncertainty about given a particular assumed structure s it is less common to acknowledge structural uncertainty about s itself a widely used approach involves enlisting the aid of x to specify a plausible single best choice s for s and then proceeding as if s were known to be correct in general this approach fails to assess and propagate structural uncertainty fully and may lead to miscalibrated uncertainty assessments about y given x when miscalibration occurs it will often result in understatement of inferential or predictive uncertainty about y leading to inaccurate scientific summaries and overconfident decisions that do not incorporate sufficient hedging against uncertainty in this paper i discuss a bayesian approach to solving this problem that has long been available in principle but is only now becoming routinely feasible by virtue of recent computational advances and examine its implementation in examples that involve forecasting the price of oil and estimating the chance of catastrophic failure of the us space shuttle

the cost of generalizing logistic regression abstract binary response regression models in which the link function is a family defined by one or more unknown shape parameters are considered detailed attention is given to the two single parameter families proposed by aranda ordaz 1981 that incorporate the logistic linear and complementary log log link functions as special cases one model is a symmetric family of alternatives to the logistic shape and the second model is an asymmetric family the increase in variance of quantities of interest due to the addition of an extra parameter is calculated using asymptotic methods this inflation in variance is interpreted as the cost of adding an additional parameter to the model in considering the seriousness of this cost the reduction in bias due to the estimation of the additional parameter should also be taken into account three quantities of interest predicted probabilities for a fixed value of the explanatory variable predicted value of the explanatory variable to obtain a specific probability

bounds on the information rate for sparse channels with long memory and i u d inputs in this paper we propose new bounds on the achievable information rate for discrete time gaussian channels with intersymbol interference isi and independent and uniformly distributed i u d channel input symbols drawn from finite order modulation alphabets specifically we are interested in developing new bounds on the achievable rates for sparse channels with long memory we obtain a lower bound which can be achieved by practical receivers based on mmse channel shortening and suboptimal symbol detection for a reduced state channel an upper bound is given in the form of a semi analytical solution derived using basic information theoretic inequalities by a grouping of the channel taps into several clusters resulting in a newly defined single input multiple output simo channel we show that the so obtained time dispersive simo channel can be represented by an equivalent single input single output siso channel with a significantly shorter channel memory the reduced computational complexity allows the use of the bcjr algorithm for the newly defined channel the proposed bounds are illustrated through several sparse channel examples and i u d input symbols showing that the upper bound significantly outperforms existing bounds performance of our lower bound strongly depends on the channel structure showing best results for minimum phase and maximum phase systems

bayesian inference in econometric models using monte carlo integration methods for the systematic application of monte carlo integration with importance sampling to bayesian inference in econometric models are developed conditions under which the numerical approximation of a posterior moment converges almost surely to the true value as the number of monte carlo replications increases and the numerical accuracy of this approximation may be assessed reliably are set forth methods for the analytical verification of these conditions are discussed

high snr asymptotics of mutual information for discrete constellations with applications to bicm asymptotic expressions of the mutual information between any discrete input and the corresponding output of the scalar additive white gaussian noise channel are presented in the limit as the signal to noise ratio snr tends to infinity asymptotic expressions of the symbol error probability sep and the minimum mean square error mmse achieved by estimating the channel input given the channel output are also developed it is shown that for any input distribution the conditional entropy of the channel input given the output mmse and sep have an asymptotic behavior proportional to the gaussian q function the argument of the q function depends only on the minimum euclidean distance med of the constellation and the snr and the proportionality constants are functions of the med and the probabilities of the pairs of constellation points at med the developed expressions are then generalized to study the high snr behavior of the generalized mutual information gmi for bit interleaved coded modulation bicm by means of these asymptotic expressions the long standing conjecture that gray codes are the binary labelings that maximize the bicm gmi at high snr is proven it is further shown that for any equally spaced constellation whose size is a power of two there always exists an anti gray code giving the lowest bicm gmi at high snr

robust bayes and empirical bayes analysis with _ epsilon contaminated priors on considere deux directions dans l analyse bayesienne robuste la premiere consiste a determiner l etendue des probabilites a posteriori d un ensemble la seconde consiste a selectionner une bonne distribution a priori

optimum power allocation for parallel gaussian channels with arbitrary input distributions the mutual information of independent parallel gaussian noise channels is maximized under an average power constraint by independent gaussian inputs whose power is allocated according to the waterfilling policy in practice discrete signaling constellations with limited peak to average ratios m psk m qam etc are used in lieu of the ideal gaussian signals this paper gives the power allocation policy that maximizes the mutual information over parallel channels with arbitrary input distributions such policy admits a graphical interpretation referred to as mercury waterfilling which generalizes the waterfilling solution and allows retaining some of its intuition the relationship between mutual information of gaussian channels and nonlinear minimum mean square error mmse proves key to solving the power allocation problem

model selection and accounting for model uncertainty in graphical models using occam s window abstract we consider the problem of model selection and accounting for model uncertainty in high dimensional contingency tables motivated by expert system applications the approach most used currently is a stepwise strategy guided by tests based on approximate asymptotic p values leading to the selection of a single model inference is then conditional on the selected model the sampling properties of such a strategy are complex and the failure to take account of model uncertainty leads to underestimation of uncertainty about quantities of interest in principle a panacea is provided by the standard bayesian formalism that averages the posterior distributions of the quantity of interest under each of the models weighted by their posterior model probabilities furthermore this approach is optimal in the sense of maximizing predictive ability but this has not been used in practice because computing the posterior model probabilities is hard and the number of models is very large often greater than 1

bayes inference in the tobit censored regression model we consider the bayes estimation of the tobit censored regression model with normally distributed errors a simple condition for the existence of posterior moments is provided suitable versions of monte carlo procedures based on symmetric multivariate t distributions and laplacian approximations in a certain parametrization are developed and illustrated ideas involving data augmentation and gibbs sampling cf tanner and wong 1987 and gelfand and smith 1990 are also developed the methods are compared in two examples with diffuse priors and various combinations of sample sizes and degrees of censoring

variable selection via gibbs sampling abstract a crucial problem in building a multiple regression model is the selection of predictors to include the main thrust of this article is to propose and develop a procedure that uses probabilistic considerations for selecting promising subsets this procedure entails embedding the regression setup in a hierarchical normal mixture model where latent variables are used to identify subset choices in this framework the promising subsets of predictors can be identified as those with higher posterior probability the computational burden is then alleviated by using the gibbs sampler to indirectly sample from this multinomial posterior distribution on the set of possible subset choices those subsets with higher probability the promising ones can then be identified by their more frequent appearance in the gibbs sample

comparison of the achievable rates in ofdm and single carrier modulation with i i d inputs we compare the maximum achievable rates in single carrier sc and orthogonal frequency division multiplexing ofdm modulation schemes under the practical assumptions of independent identically distributed finite alphabet inputs and linear intersymbol interference with additive gaussian noise we show that the shamai laroia approximation serves as a bridge between the two rates while it is well known that this approximation is often a lower bound on the sc achievable rate it is revealed to also essentially upper bound the ofdm achievable rate we apply information estimation relations in order to rigorously establish this result for both general input distributions and to sharpen it for commonly used pulse amplitude modulation pam and quadratic amplitude modulation constellations to this end novel bounds on minimum mean square error estimation of pam inputs to a scalar gaussian channel are derived which may be of general interest our results show that under reasonable assumptions optimal sc schemes may offer spectral efficiency significantly superior to that of ofdm motivating further research of such systems

lower bounds and approximations for the information rate of the isi channel we consider the discrete time intersymbol interference isi channel model with additive gaussian noise and fixed independent identically distributed inputs in this setting we investigate the expression put forth by shamai and laroia as a conjectured lower bound for the input output mutual information after application of a minimum mean square error decision feedback equalizer receiver a low signal to noise ratio snr expansion is used to prove that the conjectured bound does not hold under general conditions and to characterize inputs for which it is particularly ill suited one such input is used to construct a counterexample indicating that the shamai laroia expression does not always bound even the achievable rate of the channel thus excluding a natural relaxation of the original conjectured bound however this relaxed bound is then shown to hold for any finite entropy input and isi channel when the snr is sufficiently high we derive two conditions under which the relaxed bound holds involving compound channel capacity and quasiconvexity arguments finally new simple bounds for the achievable rate are proven and compared with other known bounds information estimation relations and estimation theoretic bounds play a key role in establishing our results