a comparison of two model averaging techniques with an application to growth empirics parameter estimation under model uncertainty is a difficult and fundamental issue in econometrics this paper compares the performance of various model averaging techniques in particular it contrasts bayesian model averaging bma currently one of the standard methods used in growth empirics with a new method called weighted average least squares wals the new method has two major advantages over bma its computational burden is trivial and it is based on a transparent definition of prior ignorance the theory is applied to and sheds new light on growth empirics where a high degree of model uncertainty is typically present

distributed multi agent optimization with state dependent communication we study distributed algorithms for solving global optimization problems in which the objective function is the sum of local objective functions of agents and the constraint set is given by the intersection of local constraint sets of agents we assume that each agent knows only his own local objective function and constraint set and exchanges information with the other agents over a randomly varying network topology to update his information state we assume a state dependent communication model over this topology communication is markovian with respect to the states of the agents and the probability with which the links are available depends on the states of the agents we study a projected multi agent subgradient algorithm under state dependent communication the state dependence of the communication introduces significant challenges and couples the study of information exchange with the analysis of subgradient steps and projection errors we first show that the multi agent subgradient algorithm when used with a constant stepsize may result in the agent estimates to diverge with probability one under some assumptions on the stepsize sequence we provide convergence rate bounds on a disagreement metric between the agent estimates our bounds are time nonhomogeneous in the sense that they depend on the initial starting time despite this we show that agent estimates reach an almost sure consensus and converge to the same optimal solution of the global optimization problem with probability one under different assumptions on the local constraint sets and the stepsize sequence

fixed point optimization algorithms for distributed optimization in networked systems this paper considers a networked system with a finite number of users and deals with the problem of minimizing the sum of all users objective functions over the intersection of all users constraint sets onto which the projection cannot be easily implemented the main objective of this paper is to devise distributed optimization algorithms which enable each user to find the solution of the problem without using other users objective functions and constraint sets to reach this goal we first introduce easily implementable nonexpansive mappings of which the intersection of the fixed point sets is equal to the constraint set in the problem we formulate the problem as a convex minimization problem over the intersection of the fixed point sets of the nonexpansive mappings we then present an iterative algorithm based on the conventional incremental subgradient methods which use the projection for solving the problem the algorithm can be implemented by using nonexpansive mappings other than the project

convergence of approximate and incremental subgradient methods for convex optimization we present a unified convergence framework for approximate subgradient methods that covers various stepsize rules including both diminishing and nonvanishing stepsizes convergence in objective values and convergence to a neighborhood of the optimal set we discuss ways of ensuring the boundedness of the iterates and give efficiency estimates our results are extended to incremental subgradient methods for minimizing a sum of convex functions which have recently been shown to be promising for various large scale problems including those arising from lagrangian relaxation

least squares model averaging this paper considers the problem of selection of weights for averaging across least squares estimates obtained from a set of models existing model average methods are based on exponential akaike information criterion aic and bayesian information criterion bic weights in distinction this paper proposes selecting the weights by minimizing a mallows criterion the latter an estimate of the average squared error from the model average fit we show that our new mallows model average mma estimator is asymptotically optimal in the sense of achieving the lowest possible squared error in a class of discrete model average estimators in a simulation experiment we show that the mma estimator compares favorably with those based on aic and bic weights the proof of the main result is an application of the work of li 1987

a use of conjugate gradient direction for the convex optimization problem over the fixed point set of a nonexpansive mapping in this paper we discuss the convex optimization problem over the fixed point set of a nonexpansive mapping the main objective of the paper is to accelerate the hybrid steepest descent method for the problem to this goal we present a new iterative scheme that utilizes the conjugate gradient direction its convergence to the solution is guaranteed under certain assumptions in order to demonstrate the effectiveness performance and convergence of our proposed algorithm we present numerical comparisons of the algorithm with the existing algorithm

acceleration method for convex optimization over the fixed point set of a nonexpansive mapping the existing algorithms for solving the convex minimization problem over the fixed point set of a nonexpansive mapping on a hilbert space are based on algorithmic methods such as the steepest descent method and conjugate gradient methods for finding a minimizer of the objective function over the whole space and attach importance to minimizing the objective function as quickly as possible meanwhile it is of practical importance to devise algorithms which converge in the fixed point set quickly because the fixed point set is the set with the constraint conditions that must be satisfied in the problem this paper proposes an algorithm which not only minimizes the objective function quickly but also converges in the fixed point set much faster than the existing algorithms and proves that the algorithm with diminishing step size sequences strongly converges to the solution to the convex minimization problem we also analyze the proposed algorithm with each of the fletcher reeves polak ribiere polyak hestenes stiefel and dai yuan formulas used in the conventional conjugate gradient methods and show that there is an inconvenient possibility that their algorithms may not converge to the solution to the convex minimization problem we numerically compare the proposed algorithm with the existing algorithms and show its effectiveness and fast convergence

frequentist model average estimators the traditional use of model selection methods in practice is to proceed as if the final selected model had been chosen in advance without acknowledging the additional uncertainty introduced by model selection this often means underreporting of variability and too optimistic confidence intervals we build a general large sample likelihood apparatus in which limiting distributions and risk properties of estimators post selection as well as of model average estimators are precisely described also explicitly taking modeling bias into account this allows a drastic reduction in complexity as competing model averaging schemes may be developed discussed and compared inside a statistical prototype experiment where only a few crucial quantities matter in particular we offer a frequentist view on bayesian model averaging methods and give a link to generalized ridge estimators our work also leads to new model selection criteria the methods are illustrated with real data applications

incremental subgradient methods for nondifferentiable optimization we consider a class of subgradient methods for minimizing a convex function that consists of the sum of a large number of component functions this type of minimization arises in a dual context from lagrangian relaxation of the coupling constraints of large scale separable problems the idea is to perform the subgradient iteration incrementally by sequentially taking steps along the subgradients of the component functions with intermediate adjustment of the variables after processing each component function this incremental approach has been very successful in solving large differentiable least squares problems such as those arising in the training of neural networks and it has resulted in a much better practical rate of convergence than the steepest descent method in this paper we establish the convergence properties of a number of variants of incremental subgradient methods including some that are stochastic based on the analysis and computational experiments the methods appear very promising and effective for important classes of large problems a particularly interesting discovery is that by randomizing the order of selection of component functions for iteration the convergence rate is substantially improved

bayesian model averaging for linear regression models abstract we consider the problem of accounting for model uncertainty in linear regression models conditioning on a single selected model ignores model uncertainty and thus leads to the underestimation of uncertainty when making inferences about quantities of interest a bayesian solution to this problem involves averaging over all possible models i e combinations of predictors when making inferences about quantities of interest this approach is often not practical in this article we offer two alternative approaches first we describe an ad hoc procedure occam s window which indicates a small set of models over which a model average can be computed second we describe a markov chain monte carlo approach that directly approximates the exact solution in the presence of model uncertainty both of these model averaging procedures provide better predictive performance than any single model that might reasonably have been selected in the extreme case where there are many candidate predictors but

random algorithms for convex minimization problems this paper deals with iterative gradient and subgradient methods with random feasibility steps for solving constrained convex minimization problems where the constraint set is specified as the intersection of possibly infinitely many constraint sets each constraint set is assumed to be given as a level set of a convex but not necessarily differentiable function the proposed algorithms are applicable to the situation where the whole constraint set of the problem is not known in advance but it is rather learned in time through observations also the algorithms are of interest for constrained optimization problems where the constraints are known but the number of constraints is either large or not finite we analyze the proposed algorithm for the case when the objective function is differentiable with lipschitz gradients and the case when the objective function is not necessarily differentiable the behavior of the algorithm is investigated both for diminishing and non diminishing stepsize values the almost sure convergence to an optimal solution is established for diminishing stepsize for non diminishing stepsize the error bounds are established for the expected distances of the weighted averages of the iterates from the constraint set as well as for the expected sub optimality of the function values along the weighted averages

the viscosity approximation process for quasi nonexpansive mappings in hilbert spaces in this paper we discuss the strong convergence of the viscosity approximation method in hilbert spaces relatively to the computation of fixed points of operators in the wide class of quasi nonexpansive mappings our convergence results improve previously known ones obtained for the class of nonexpansive mappings

on the harm that ignoring pretesting can cause in econometrics the same data set is typically used to select the model and to estimate the parameters in the selected model in applied econometrics practice however one typically acts as if the model had been given a priori thus ignoring the fact that the estimators are in fact pretest estimators hence one assumes incorrectly that the estimator is unbiased and that the reported variance conditional on the selected model is equal to its unconditional variance in this paper we nd the unconditional rst and second moments of the pretest estimator in fact of a more general estimator the wals estimator taking full account of the fact that model selection and estimation are an integrated procedure and show that the error in not reporting the correct moments can be large we also show that this error can vary substantially between di2erent model selection procedures finally we ask how the error increases when the number of auxiliary regressors increases

acceleration method combining broadcast and incremental distributed optimization algorithms this paper considers a networked system consisting of an operator which manages the system and a finite number of subnetworks with all users and studies the problem of minimizing the sum of the operator s and all users objective functions over the intersection of the operator s and all users constraint sets when users in each subnetwork can communicate with each other they can implement an incremental subgradient method that uses the transmitted information from their neighbor users since the operator can communicate with users in the subnetworks it can implement a broadcast distributed algorithm that uses all available information in the subnetworks we present an iterative method combining broadcast and incremental distributed optimization algorithms our method has faster convergence and a wider range of application compared with conventional distributed algorithms we also prove that under certain assumptions our method converges to the solution to the problem in the sense of the strong topol

a randomized incremental subgradient method for distributed optimization in networked systems we present an algorithm that generalizes the randomized incremental subgradient method with fixed stepsize due to nedic and bertsekas siam j optim 12 2001 pp 109 138 our novel algorithm is particularly suitable for distributed implementation and execution and possible applications include distributed optimization e g parameter estimation in networks of tiny wireless sensors the stochastic component in the algorithm is described by a markov chain which can be constructed in a distributed fashion using only local information we provide a detailed convergence analysis of the proposed algorithm and compare it with existing both deterministic and randomized incremental subgradient methods

a convergent incremental gradient method with a constant step size an incremental aggregated gradient method for minimizing a sum of continuously differentiable functions is presented the method requires a single gradient evaluation per iteration and uses a constant step size for the case that the gradient is bounded and lipschitz continuous we show that the method visits infinitely often regions in which the gradient is small under certain unimodality assumptions global convergence is established in the quadratic case a global linear rate of convergence is shown the method is applied to distributed optimization problems arising in wireless sensor networks and numerical experiments compare the new method with other incremental gradient methods

assessment and propagation of model uncertainty in most examples of inference and prediction the expression of uncertainty about unknown quantities y on the basis of known quantities x is based on a model m that formalizes assumptions about how x and y are related m will typically have two parts structural assumptions s such as the form of the link function and the choice of error distribution in a generalized linear model and parameters whose meaning is specific to a given choice of s it is common in statistical theory and practice to acknowledge parametric uncertainty about given a particular assumed structure s it is less common to acknowledge structural uncertainty about s itself a widely used approach involves enlisting the aid of x to specify a plausible single best choice s for s and then proceeding as if s were known to be correct in general this approach fails to assess and propagate structural uncertainty fully and may lead to miscalibrated uncertainty assessments about y given x when miscalibration occurs it will often result in understatement of inferential or predictive uncertainty about y leading to inaccurate scientific summaries and overconfident decisions that do not incorporate sufficient hedging against uncertainty in this paper i discuss a bayesian approach to solving this problem that has long been available in principle but is only now becoming routinely feasible by virtue of recent computational advances and examine its implementation in examples that involve forecasting the price of oil and estimating the chance of catastrophic failure of the us space shuttle

proximal point algorithms for nonsmooth convex optimization with fixed point constraints abstract the problem of minimizing the sum of nonsmooth convex objective functions defined on a real hilbert space over the intersection of fixed point sets of nonexpansive mappings onto which the projections cannot be efficiently computed is considered the use of proximal point algorithms that use the proximity operators of the objective functions and incremental optimization techniques is proposed for solving the problem with the focus on fixed point approximation techniques two algorithms are devised for solving the problem one blends an incremental subgradient method which is a useful algorithm for nonsmooth convex optimization with a halpern type fixed point iteration algorithm the other is based on an incremental subgradient method and the krasnosel ski mann fixed point algorithm it is shown that any weak sequential cluster point of the sequence generated by the halpern type algorithm belongs to the solution set of the problem and that there exists a weak sequential cluster point of the sequence generated by the krasnosel ski mann type algorithm which also belongs to the solution set numerical comparisons of the two proposed algorithms with existing subgradient methods for concrete nonsmooth convex optimization show that the proposed algorithms achieve faster convergence

convergence analysis of iterative methods for nonsmooth convex optimization over fixed point sets of quasi nonexpansive mappings this paper considers a networked system with a finite number of users and supposes that each user tries to minimize its own private objective function over its own private constraint set it is assumed that each user s constraint set can be expressed as a fixed point set of a certain quasi nonexpansive mapping this enables us to consider the case in which the projection onto the constraint set cannot be computed efficiently this paper proposes two methods for solving the problem of minimizing the sum of their nondifferentiable convex objective functions over the intersection of their fixed point sets of quasi nonexpansive mappings in a real hilbert space one method is a parallel subgradient method that can be implemented under the assumption that each user can communicate with other users the other is an incremental subgradient method that can be implemented under the assumption that each user can communicate with its neighbors investigation of the two methods convergence properties for a constant step size reveals that with a small constant step size they approximate a solution to the problem consideration of the case in which the step size sequence is diminishing demonstrates that the sequence generated by each of the two methods strongly converges to the solution to the problem under certain assumptions convergence rate analysis of the two methods under certain situations is provided to illustrate the two methods efficiency this paper also discusses nonsmooth convex optimization over sublevel sets of convex functions and provides numerical comparisons that demonstrate the effectiveness of the proposed methods

distributed algorithms for computing a common fixed point of a group of nonexpansive operators this paper addresses the problem of seeking a common fixed point for a collection of nonexpansive operators over time varying multi agent networks in real hilbert spaces each operator is only privately and approximately known to each individual agent and all agents need to cooperate to solve this problem by local communications over time varying networks to handle this problem inspired by the centralized inexact krasnosel skii mann ikm iteration we propose a distributed algorithm called distributed inexact averaged operator algorithm d iao it is shown that the d iao algorithm can converge weakly to a common fixed point of the family of nonexpansive operators moreover under the linearly regular assumption it is proved that the d iao algorithm converges with a rate o gamma log 16 4k for some constant gamma in 0 1 where k is the iteration number the same converge speed can be ensured by a novel power regularity condition which is strictly weaker than linear regularity to reduce computational complexity another algorithm called d biao is developed and fully analyzed simulation examples are given to support the theoretical res

the focused information criterion a variety of model selection criteria have been developed of general and specific types most of these aim at selecting a single model with good overall properties for example formulated via average prediction quality or shortest estimated overall distance to the true model the akaike the bayesian and the deviance information criteria along with many suitable variations are examples of such methods these methods are not concerned however with the actual use of the selected model which varies with context and application the present article takes the view that the model selector should instead focus on the parameter singled out for interest in particular a model that gives good precision for one estimand may be worse when used for inference for another estimand we develop a method that for a given focus parameter estimates the precision of any submodel based estimator the framework is that of large sample likelihood inference using an unbiased estimate of limiting risk we propose a foc

a weak to strong convergence principle for feje monotone methods in hilbert spaces we consider a wide class of iterative methods arising in numerical mathematics and optimization that are known to converge only weakly exploiting an idea originally proposed by haugazeau we present a simple modification of these methods that makes them strongly convergent without additional assumptions several applications are discussed