entropy and data compression schemes some new ways of defining the entropy of a process by observing a single typical output sequence as well as a new kind of shannon mcmillan breiman theorem are presented this provides a new and conceptually very simple ways of estimating the entropy of an ergodic stationary source as well as new insight into the workings of such well known data compression schemes as the lempel ziv algorithm

optimality functions in stochastic programming optimality functions define stationarity in nonlinear programming semi infinite optimization and optimal control in some sense in this paper we consider optimality functions for stochastic programs with nonlinear possibly nonconvex expected value objective and constraint functions we show that an optimality function directly relates to the difference in function values at a candidate point and a local minimizer we construct confidence intervals for the value of the optimality function at a candidate point and hence provide a quantitative measure of solution quality based on sample average approximations we develop an algorithm for classes of stochastic programs that include cvar problems and utilize optimality functions to select sample sizes

a simulation based approach to two stage stochastic programming with recourse in this paper we consider stochastic programming problems where the objective function is given as an expected value function we discuss monte carlo simulation based approaches to a numerical solution of such problems in particular we discuss in detail and present numerical results for two stage stochastic programming with recourse where the random data have a continuous multivariate normal distribution we think that the novelty of the numerical approach developed in this paper is twofold first various variance reduction techniques are applied in order to enhance the rate of convergence successful application of those techniques is what makes the whole approach numerically feasible second a statistical inference is developed and applied to estimation of the error validation of optimality of a calculated solution and statistically based stopping criteria for an iterative alogrithm 1998 the mathematical programming society inc published by elsevier science b v

efficient sample sizes in stochastic nonlinear programming we consider a class of stochastic nonlinear programs for which an approximation to a locally optimal solution is specified in terms of a fractional reduction of the initial cost error we show that such an approximate solution can be found by approximately solving a sequence of sample average approximations the key issue in this approach is the determination of the required sequence of sample average approximations as well as the number of iterations to be carried out on each sample average approximation in this sequence we show that one can express this requirement as an idealized optimization problem whose cost function is the computing work required to obtain the required error reduction the specification of this idealized optimization problem requires the exact knowledge of a few problems and algorithm parameters since the exact values of these parameters are not known we use estimates which can be updated as the computation progresses we illustrate our approach using two numerical examples from structural engineering design

extreme value laws for fractal intensity functions in dynamical systems minkowski analysis typically in the dynamical theory of extremal events the function that gauges the intensity of a phenomenon is assumed to be convex and maximal or singular at a single or at most a finite collection of points in phase space in this paper we generalize this situation to fractal landscapes i e intensity functions characterized by an uncountable set of singularities located on a cantor set this reveals the dynamical role of classical quantities like the minkowski dimension and content whose definition we extend to account for singular continuous invariant measures we also introduce the concept of extremely rare event quantified by non standard minkowski constants and we study its consequences to extreme value statistics limit laws are derived from formal calculations and are verified by numerical experiments

inexact restoration approach for minimization with inexact evaluation of the objective function a new method is introduced for minimizing a function that can be computed only inexactly with different levels of accuracy the challenge is to evaluate the potentially very expensive objective function with low accuracy as far as this does not interfere with the goal of getting high accuracy minimization at the end for achieving this goal the problem is reformulated in terms of constrained optimization and handled with an inexact restoration technique convergence is proved and numerical experiments motivated by electronic structure calculations are presented which indicate that the new method overcomes current approaches for solving large scale problems

a sequential sampling procedure for stochastic programming we develop a sequential sampling procedure for a class of stochastic programs we assume that a sequence of feasible solutions with an optimal limit point is given as input to our procedure such a sequence can be generated by solving a series of sampling problems with increasing sample size or it can be found by any other viable method our procedure estimates the optimality gap of a candidate solution from this sequence if the point estimate of the optimality gap is sufficiently small according to our termination criterion then we stop otherwise we repeat with the next candidate solution from the sequence under an increased sample size we provide conditions under which this procedure i terminates with probability one and ii terminates with a solution that has a small optimality gap with a prespecified probability

fixed width sequential stopping rules for a class of stochastic programs monte carlo sampling based methods are frequently used in stochastic programming when exact solution is not possible a critical component of monte carlo sampling based methods is determining when to stop sampling to ensure the desired quality of the solutions in this paper we develop stopping rules for sequential sampling procedures that depend on the width of an optimality gap confidence interval estimator the procedures solve a sequence of sampling approximations with increasing sample size to generate solutions and stop when the width of the confidence interval on the current solution s optimality gap plus an inflation factor fall below a prespecified value epsilon we first present a method that takes the schedule of sample sizes as an input and provide guidelines on the growth of sample sizes then we present a method that increases the sample sizes according to the current estimates of the optimality gap the larger the estimates the larger the increases in sample sizes we provide conditi

the global statistics of return times return time dimensions versus generalized measure dimensions we investigate return times in dynamical systems i e the time required by a trajectory to complete a return journey to a neighborhood of the initial position in particular we study the relations holding between the scaling exponents of phase space moments of return times in balls of diminishing radius on the one side and the generalized dimensions of invariant measures on the other because of a heuristic use of kac theorem the former have been used in place of the latter in numerical and experimental investigations to mark the distinction we call them return time dimensions we derive a full set of inequalities linking generalized dimensions of invariant measures and return time dimensions we comment on their optimality with the aid of two maps due to von neumann kakutani and to gaspard wang we conjecture a formula for the return time dimensions in a typical system we only assume that the dynamical system under investigation is ergodic and that motion takes place in a compact finite dimensional space

dynamic fractals in spatial evolutionary games abstract we investigate critical properties of a spatial evolutionary game based on the prisoner s dilemma simulations demonstrate a jump in the component densities accompanied by drastic changes in average sizes of the component clusters we argue that the cluster boundary is a random fractal our simulations are consistent with the fractal dimension of the boundary being equal to 2 and the cluster boundaries are hence asymptotically space filling as the system size increases

sample path optimization of convex stochastic performance functions in this paper we propose a method for optimizing convex performance functions in stochastic systems these functions can include expected performance in static systems and steady state performance in discrete event dynamic systems they may be nonsmooth the method is closely related to retrospective simulation optimization it appears to overcome some limitations of stochastic approximation which is often applied to such problems we explain the method and give computational results for two classes of problems tandem production lines with up to 50 machines and stochastic pert program evaluation and review technique problems with up to 70 nodes and 110 arcs

pointwise dimensions of the lorenz attractor we discuss a connection between two complementary views of the lorenz attractor the first is the accepted view where the attractor has a smooth measure on a fractal support this complex system is alternatively manifest as a self similar curve for the pointwise dimension alpha we describe why the latter approach is accessible for the analysis of an experimental signal

numerical estimates of the generalized dimensions of the henon attractor for negative q the usual fixed size box counting algorithms are inefficient for computing generalized fractal dimensions in the range of q 0 in this letter we describe a new numerical algorithm specifically devised to estimate generalized dimensions for large negative q providing evidence of its better performance we compute the complete spectrum of the henon attractor and interpret our results in terms of a phase transition between different multiplicative laws

the infinite number of generalized dimensions of fractals and strange attractors abstract we show that fractals in general and strange attractors in particular are characterized by an infinite number of generalized dimensions dq q 0 to this aim we develop a rescaling transformation group which yields analytic expressions for all the quantities dq we prove that lim q 0 dq fractal dimension d limq 1dq information dimension and dq 2 correlation exponent v dq with other integer q s correspond to exponents associated with ternary quaternary and higher correlation functions we prove that generally dq dq for any q q for homogeneous fractals dq dq a particularly interesting dimension is dq for two examples feigenbaum attractor generalized baker s transformation we calculate the generalized dimensions and find that d is a non trivial number all the other generalized dimensions are bounded between the fractal dimension and d

a multifractal analysis of equilibrium measures for conformal expanding maps and moran like geometric constructions in this paper we establish the complete multifractal formalism for equilibrium measures for holder continuous conformal expanding maps andexpanding markov moran like geometric constructions examples include markov maps of an interval beta transformations of an interval rational maps with hyperbolic julia sets and conformal toral endomorphisms we also construct a holder continuous homeomorphism of a compact metric space with an ergodic invariant measure of positive entropy for which the dimension spectrum is not convex and hence the multifractal formalism fails

sharp error terms and neccessary conditions for exponential hitting times in mixing processes we prove an upper bound for the error in the exponential approximation of the hitting time law of a rare event in mixing processes with exponential decay mixing processes with a summable function and for general mixing processes with a finite alphabet in the first case the bound is uniform as a function of the measure of the event in the last two cases the bound depends also on the time scale t this allows us to get further statistical properties as the ratio convergence of the expected hitting time and the expected return time a uniform bound is a consequence we present an example that shows that this bound is sharp we also prove that second moments are not necessary for having the exponential law moreover we prove a necessary condition for having the exponential limit law

effective diagonalization strategies for the solution of a class of optimal design problems it is shown that it is possible to obtain optimal diagonalization strategies for the discretization of semiinfinite minimax optimal design problems both exact and approximate methods for the computation of these optimal diagonalization strategies are proposed the algorithms for computing approximate diagonalization strategies yield very good approximations in much less computing time than needed to compute an optimal diagonalization strategy exactly the proposed diagonalization strategies can be implemented by using estimation schemes to obtain approximations to the various quantities which determine an optimal strategy experimental results involving the solution of optimal loop shaping problems for multivariable linear feedback systems show that the use of these implementable strategies leads to considerable savings in computer time over alternative approaches

adaptive importance sampling for optimization under uncertainty problems abstract design under uncertainty problems where a probabilistic performance is adopted as objective function and its estimation is obtained through stochastic simulation are discussed the focus is on reducing the computational burden associated with the stochastic simulation through adaptive implementation of importance sampling is across the iterations of the optimization algorithm the proposed formulation relies only on available information i e function evaluations from the current iteration of the optimization process to improve estimation accuracy in subsequent iterations and therefore corresponds to a is selection with a small additional computational burden kernel density estimation kde is employed to construct the is densities based on samples distributed proportional to the integrand of the probabilistic performance the characteristics of the proposal density are optimally selected to minimize the anticipated coefficient of variation for the objective function if such a proposal density is used as is distribution to avoid numerical problems that can occur when trying to develop is for all uncertain model parameters a prioritization is first performed using a recently proposed global sensitivity analysis to quantify the relative importance of each model parameter therefore the is density is only constructed for the most important parameters with the exact number also a variable that is optimally selected based on the anticipated accuracy to facilitate the overall adaptive scheme efficient guidelines for the sharing of information across iterations of the optimization algorithm are developed the numerical example considered verifies the efficiency of the proposed adaptive is framework

asymptotic theory for solutions in statistical estimation and stochastic programming new techniques of local sensitivity analysis for nonsmooth generalized equations are applied to the study of sequences of statistical estimates and empirical approximations to solutions of stochastic programs consistency is shown to follow from a certain local invertibility property and asymptotic distributions are derived from a generalized implicit function theorem that characterizes asymptotic behavior in situations where estimates are subjected to constraints and estimation functionals are nonsmooth

the hausdorff dimension of the boundary of the mandelbrot set and julia sets it is shown that the boundary of the mandelbrot set m has hausdorff dimension two and that for a generic c e am the julia set of z i z2 c also has hausdorff dimension two the proof is based on the study of the bifurcation of parabolic periodic points

sporadicity between periodic and chaotic dynamical behaviors abstract we define the class of sporadic dynamical systems as the systems where the algorithmic complexity of kolmogorov kolmogorov a n 1983 russ math surv 38 29 40 and chaitin chaitin g j 1987 algorithmic information theory cambridge univ press cambridge u k as well as the logarithm of separation of initially nearby trajectories grow as nv0 log n v1 with 0 v0 1 or v0 1 and v1 0 as time n these systems present a behavior intermediate between the multiperiodic v0 0 v1 1 and the chaotic ones v0 1 v1 0 we show that intermittent systems of manneville manneville p 1980 j phys paris 41 1235 1243 as well as some countable markov chains may be sporadic and furthermore that the dynamical fluctuations of these systems may be of levy s type rather than gaussian

multifractal properties of return time statistics the global statistics of the return times of a dynamical system can be described by a new spectrum of generalized dimensions comparison with the usual multifractal analysis of measures is presented and the difference between the two corresponding sets of dimensions is established theoretical analysis and numerical examples of dynamical systems in the class of iterated functions are presented

some results on the behavior and estimation of the fractal dimensions of distributions on attractors the strong interest in recent years in analyzing chaotic dynamical systems according to their asymptotic behavior has led to various definitions of fractal dimension and corresponding methods of statistical estimation in this paper we first provide a rigorous mathematical framework for the study of dimension focusing on pointwise dimension x and the generalized renyi dimensionsd q and give a rigorous proof of inequalities first derived by grassberger and procaccia and hentschel and procaccia we then specialize to the problem of statistical estimation of the correlation dimension and information dimension it has been recognized for some time that the error estimates accompanying the usual procedures which generally involve least squares methods and nearest neighbor calculations grossly underestimate the true statistical error involved in least squares analyses of and we identify sources of error not previously discussed in the literature and address the problem of obtaining accurate error estimates we then develop an estimation procedure for which corrects for an important bias term the local measure density and provides confidence intervals for the general applicability of this method is illustrated with various numerical examples

statistical description of chaotic attractors the dimension function a method for the investigation of fractal attractors is developed based on statistical properties of the distributionp n of nearest neighbor distances between points on the attractor a continuous infinity of dimensions called dimension function is defined through the moments ofp n in particular for the case of self similar sets we prove that the dimension function df yields in suitable points capacity information dimension and all other renyi dimensions an algorithm to compute df is derived and applied to several attractors as a consequence an estimate of nonuniformity in dynamical systems can be performed either by direct calculation of the uniformity factor or by comparison among various dimensions finally an analytical study of the distributionp n is carried out in some simple meaningful examples

line search methods with variable sample size for unconstrained optimization minimization of unconstrained objective functions in the form of mathematical expectation is considered the sample average approximation saa method transforms the expectation objective function into a real valued deterministic function using a large sample and thus deals with deterministic function minimization the main drawback of this approach is its cost a large sample of the random variable that defines the expectation must be taken in order to get a reasonably good approximation and thus the sample average approximation method requires a very large number of function evaluations we present a line search strategy that uses variable sample size and thus makes the process significantly cheaper two measures of progress lack of precision and a decrease of function value are calculated at each iteration based on these two measures a new sample size is determined the rule we present allows us to increase or decrease the sample size at each iteration until we reach some neighborhood of the solution an additional safeguard check is performed to avoid unproductive sample decrease eventually the maximal sample size is reached so that the variable sample size strategy generates a solution of the same quality as the saa method but with a significantly smaller number of function evaluations the algorithm is tested on a couple of examples including the discrete choice problem

optimal budget allocation for sample average approximation the sample average approximation approach to solving stochastic programs induces a sampling error caused by replacing an expectation by a sample average as well as an optimization error due to approximating the solution of the resulting sample average problem we obtain estimators of an optimal solution and the optimal value of the original stochastic program after executing a finite number of iterations of an optimization algorithm applied to the sample average problem we examine the convergence rate of the estimators as the computing budget tends to infinity and we characterize the allocation policies that maximize the convergence rate in the case of sublinear linear and superlinear convergence regimes for the optimization algorithm

generalized dimensions large deviations and the distribution of rare events abstract generalized dimensions of multifractal measures are usually seen as static objects related to the scaling properties of suitable partition functions or moments of measures of cells when these measures are invariant for the flow of a chaotic dynamical system generalized dimensions take on a dynamical meaning as they provide the rate function for the large deviations of the first hitting time which is the average time required to connect any two different regions in phase space we prove this result rigorously under a set of stringent assumptions as a consequence the statistics of hitting times provides new algorithms for the computation of the spectrum of generalized dimensions numerical examples presented along with the theory suggest that the validity of this technique reaches far beyond the range covered by the theorem we state our result within the framework of extreme value theory this approach reveals that hitting times are also linked to dynamical indicators such as stability of the motion and local dimensions of the invariant measure this suggests that one can use local dynamical indicators from finite time series to gather information on the multifractal spectrum of generalized dimension we show an application of this technique to experimental data from climate dynamics

an improved multifractal formalism and self similar measures abstract to characterize the geometry of a measure its generalized dimensions dq have been introduced recently the mathematically precise definition given by falconer fractal geometry 1990 turns out to be unsatisfactory for reasons of convergence as well as of undesired sensitivity to the particular choice of coordinates in the negative q range a new definition is introduced which is also based on box counting but which carries relevant information about for negative q also in particular the legendre connection between generalized dimensions and the multifractal spectrum is established using large deviations and rigorous proofs are provided for the implicit formula giving the generalized dimensions of selfsimilar measures which was until now known only for positive q

statistics of return times a general framework and new applications in this paper we provide general estimates for the errors between the distribution of the first and more generally the k th return time suitably rescaled and the poisson law for measurable dynamical systems in the case that the system exhibits strong mixing properties these bounds are explicitly expressed in terms of the speed of mixing using these approximations the poisson law is finally proved to hold for a large class of non hyperbolic systems on the interval

characterization of strange attractors a new measure of strange attractors is introduced which offers a practical algorithm to determine their character from the time series of a single observable the relation of this new measure to fractal dimension and information theoretic entropy is discussed

on choosing parameters in retrospective approximation algorithms for stochastic root finding and simulation optimization the stochastic root finding problem is that of finding a zero of a vector valued function known only through a stochastic simulation the simulation optimization problem is that of locating a real valued function s minimum again with only a stochastic simulation that generates function estimates retrospective approximation ra is a sample path technique for solving such problems where the solution to the underlying problem is approached via solutions to a sequence of approximate deterministic problems each of which is generated using a specified sample size and solved to a specified error tolerance our primary focus in this paper is providing guidance on choosing the sequence of sample sizes and error tolerances in ra algorithms we first present an overview of the conditions that guarantee the correct convergence of ra s iterates then we characterize a class of error tolerance and sample size sequences that are superior to others in a certain precisely defined sense we also identify and recommend members of this class and provide a numerical example illustrating the key results

generalised spectra for the dimensions of strange sets the author gives a thermodynamical description of the spectrum of generalised dimensions for invariant sets recently proposed by tel 1986 the author s approach allows a definition of an uncountable set of generalised spectra depending on the equilibrium measure supported by the set and the author indicates that the dimensions are independent of the measurable partition of the set