a second order perceptron algorithm kernel based linear threshold algorithms such as support vector machines and perceptron like algorithms are among the best available techniques for solving pattern classification problems in this paper we describe an extension of the classical perceptron algorithm called second order perceptron and analyze its performance within the mistake bound model of on line learning the bound achieved by our algorithm depends on the sensitivity to second order data information and is the best known mistake bound for efficient kernel based linear threshold classifiers to date this mistake bound which strictly generalizes the well known perceptron bound is expressed in terms of the eigenvalues of the empirical data correlation matrix and depends on a parameter controlling the sensitivity of the algorithm to the distribution of these eigenvalues since the optimal setting of this parameter is not known a priori we also analyze two variants of the second order perceptron algorithm one that adaptively sets the value of the parameter in terms of the number of mistakes made so far and one that is parameterless based on pseudoinverses

multi scale second order computational homogenization of multi phase materials a nested finite element solution strategy this paper presents the detailed implementation and computational aspects of a novel second order computational homogenization procedure which is suitable for a multi scale modelling of macroscopic localization and size effects the second order scheme is an extension of the classical first order computational homogenization framework and is based on a proper incorporation of the gradient of the macroscopic deformation gradient tensor into the kinematical macro micro scale transition from the microstructural analysis the macroscopic stress and higher order stress tensors are obtained thus delivering a microstructurally based constitutive response of the macroscopic second gradient continuum the higher order macroscopic constitutive tangents are derived through static condensation of the microscopic global tangent matrix for the solution of the second gradient equilibrium problem on the macrolevel a mixed finite element formulation is developed as an example the second order computational homogenization approach is applied for the multi scale analysis of simple shear of a constrained heterogeneous strip where a pronounced boundary size effect appears

two scale analysis of heterogeneous elastic plastic materials with asymptotic homogenization and voronoi cell finite element model in this paper a multiple scale finite element model vcfem homo has been developed for elastic plastic analysis of heterogeneous porous and composite materials by combining asymptotic homogenization theory with the voronoi cell finite element model vcfem vcfem for microstructural modeling originates from dirichlet tessellation of representative material elements at sampling points in the structure structural modeling is done by the general purpose finite element code abaqus and interfacing with the microscale vcfem analysis is done through the user subroutine in abaqus for material constitutive relation umat asymptotic homogenization in umat generates macroscopic material parameters for abaqus following the macroscopic analysis a local vcfem analysis is invoked to depict the true evolution of microstructural state variables various numerical examples are executed for validating the effectiveness of vcfem homo and the effect of size shape and distribution of heterogeneities on local and global response is examined

on the optimality of the discrete karhunen loeve expansion a short proof of the optimality of the discrete karhunen loeve expansion as the best linear approximation in the quadratic mean is presented

metrics for power spectra an axiomatic approach we present an axiomatic framework for seeking distances between power spectral density functions the axioms require that the sought metric respects the effects of additive and multiplicative noise in reducing our ability to discriminate spectra as well as they require continuity of statistical quantities with respect to perturbations measured in the metric we then present a particular metric which abides by these requirements the metric is based on the monge kantorovich transportation problem and is contrasted with an earlier riemannian metric based on the minimum variance prediction geometry of the underlying time series it is also being compared with the more traditional itakura saito distance measure as well as the aforementioned prediction metric on two representative examples

maximum entropy and conditional probability it is well known that maximum entropy distributions subject to appropriate moment constraints arise in physics and mathematics in an attempt to find a physical reason for the appearance of maximum entropy distributions the following theorem is offered the conditional distribution of x l given the empirical observation 1 n sum n i l h x i alpha where x 1 x 2 cdots are independent identically distributed random variables with common density g converges to f lambda x e lambda t h x g x suitably normalized where lambda is chosen to satisfy int f lambda x h x dx alpha thus the conditional distribution of a given random variable x is the normalized product of the maximum entropy distribution and the initial distribution this distribution is the maximum entropy distribution when g is uniform the proof of this and related results relies heavily on the work of zabell and lanford

exponentiated gradient versus gradient descent for linear predictors we consider two algorithm for on line prediction based on a linear model the algorithms are the well known gradient descent gd algorithm and a new algorithm which we call eg they both maintain a weight vector using simple updates for the gd algorithm the update is based on subtracting the gradient of the squared error made on a prediction the eg algorithm uses the components of the gradient in the exponents of factors that are used in updating the weight vector multiplicatively we present worst case loss bounds for eg and compare them to previously known bounds for the gd algorithm the bounds suggest that the losses of the algorithms are in general incomparable but eg has a much smaller loss if only a few components of the input are relevant for the predictions we have performed experiments which show that our worst case upper bounds are quite tight already on simple artificial data

a multi level computational model for multi scale damage analysis in composite and porous materials abstract an adaptive multi level methodology is developed in this paper to create a hierarchy of computational sub domains with varying resolution for multiple scale problems it is intended to concurrently predict evolution of variables at the structural and microstructural scales as well as to track the incidence and propagation of microstructural damage in composite and porous materials the microstructural analysis is conducted with the voronoi cell finite element model vcfem while a conventional displacement based fem code executes the macroscopic analysis the model introduces three levels in the computational domain which include macro macro micro and microscopic analysis it differentiates between non critical and critical regions and ranges from macroscopic computations using continuum constitutive relations to zooming in at hotspots for pure microscopic simulations coupling between the scales in regions of periodic microstructure is accomplished through asymptotic homogenization an adaptive process significantly increases the efficiency while retaining appropriate level of accuracy for each region numerical examples are conducted for composite and porous materials with a variety of microscopic architectures to demonstrate the potential of the model

dimensional model reduction in non linear finite element dynamics of solids and structures a general approach to the dimensional reduction of non linear finite element models of solid dynamics is presented for the newmark implicit time discretization the computationally most expensive phase is the repeated solution of the system of linear equations for displacement increments to deal with this it is shown how the problem can be formulated in an approximation ritz basis of much smaller dimension similarly the explicit newmark algorithm can be also written in a reduced dimension basis and the computation time savings in that case follow from an increase in the stable time step length in addition the empirical eigenvectors are proposed as the basis in which to expand the incremental problem this basis achieves approximation optimality by using computational data for the response of the full model in time to construct a reduced basis which reproduces the full system in a statistical sense because of this global time viewpoint the basis need not be updated as with reduced bases computed from a linearization of the full finite element model if the dynamics of a finite element model is expressed in terms of a small number of basis vectors the asymptotic cost of the solution with the reduced model is lowered and optimal scalability of the computational algorithm with the size of the model is achieved at the same time numerical experiments indicate that by using reduced models substantial savings can be achieved even in the pre asymptotic range furthermore the algorithm parallelizes very efficiently the method we present is expected to become a useful tool in applications requiring a large number of repeated non linear solid dynamics simulations such as convergence studies design optimization and design of controllers of mechanical systems

an identity for kernel ridge regression this paper derives an identity connecting the square loss of ridge regression in on line mode with the loss of the retrospectively best regressor some corollaries about the properties of the cumulative loss of on line ridge regression are also obtained

tracking the best linear predictor in most on line learning research the total on line loss of the algorithm is compared to the total loss of the best off line predictor u from a comparison class of predictors we call such bounds static bounds the interesting feature of these bounds is that they hold for an arbitrary sequence of examples recently some work has been done where the predictor ut at each trial t is allowed to change with time and the total on line loss of the algorithm is compared to the sum of the losses of ut at each trial plus the total cost for shifting to successive predictors this is to model situations in which the examples change over time and different predictors from the comparison class are best for different segments of the sequence of examples we call such bounds shifting bounds they hold for arbitrary sequences of examples and arbitrary sequences of predictors naturally shifting bounds are much harder to prove the only known bounds are for the case when the comparison class consists of a sequences of experts or boolean disjunctions in this paper we develop the methodology for lifting known static bounds to the shifting case in particular we obtain bounds when the comparison class consists of linear neurons linear combinations of experts our essential technique is to project the hypothesis of the static algorithm at the end of each trial into a suitably chosen convex region this keeps the hypothesis of the algorithm well behaved and the static bounds can be converted to shifting bounds

prediction in the worst case a predictor is a method of estimating the probability of future events over an infinite data sequence one predictor is as strong as another if for all data sequences the former has at most the mean square error mse of the latter given any countable set mathscr d of predictors we explicitly construct a predictor s that is at least as strong as every element of mathscr d finite sample bounds are also given which hold uniformly on the space of all possible data

a game theory approach to constrained minimax state estimation this paper presents a game theory approach to the constrained state estimation of linear discrete time dynamic systems in the application of state estimators there is often known model or signal information that is either ignored or dealt with heuristically for example constraints on the state values which may be based on physical considerations are often neglected because they do not easily fit into the structure of the state estimator this paper develops a method for incorporating state equality constraints into a minimax state estimator the algorithm is demonstrated on a simple vehicle tracking simulation

analysis of the normalized lms algorithm with gaussian inputs the lms adaptive filter algorithm requires a priori knowledge of the input power level to select the algorithm gain parameter for stability and convergence since the input power level is usually one of the statistical unknowns it is normally estimated from the data prior to beginning the adaptation process it is then assumed that the estimate is perfect in any subsequent analysis of the lms algorithm behavior in this paper the effects of the power level estimate are incorporated in a data dependent that appears explicitly within the algorithm the transient mean and second moment behavior of the modified lms nlms algorithm are evaluated taking into account the explicit statistical dependence of upon the input data the mean behavior of the algorithm is shown to converge to the wiener weight a constant coefficient matrix difference equation is derived for the weight fluctuations about the wiener weight the equation is solved for a white data covariance matrix and for the adaptive line enhancer with a single frequency input in steady state for small expressions for the misadjustment error are also presented it is shown for the white data covariance matrix case that the averaging of about ten data samples causes negligible degradation as compared to the lms algorithm in the ale application the steady state weight fluctuations are shown to be mode dependent being largest at the frequency of the input

weighted last step min max algorithm with improved sub logarithmic regret abstract in online learning the performance of an algorithm is typically compared to the performance of a fixed function from some class with a quantity called regret forster 12 proposed a last step min max algorithm which was somewhat simpler than the algorithm of vovk 26 yet with the same regret in fact the algorithm he analyzed assumed that the choices of the adversary are bounded yielding artificially only the two extreme cases we fix this problem by weighing the examples in such a way that the min max problem will be well defined and provide analysis with logarithmic regret that may have better multiplicative factor than both bounds of forster 12 and vovk 26 we also derive a new bound that may be sub logarithmic as a recent bound of orabona et al 21 but may have better multiplicative factor finally we analyze the algorithm in a weak type of non stationary setting and show a bound that is sublinear if the non stationarity is sub linear as well

a model reduction method for the post buckling analysis of cellular microstructures multiscale finite element models are useful in some applications such as biological tissues or foams to determine the effective properties of the solids in this paper a model reduction method is proposed that is applied in the post buckling regime of cellular microstructures the proposed approach combines a perturbation technique with the proper orthogonal decomposition pod the computation of finite series leads to an efficient continuation method which enables handling limit points and instabilities naturally each term of the series is itself developed using pod the size of the problems to be solved is thus drastically reduced computations involving two and three dimensional microstructures in compression enable to evaluate the potential of the technique in view of applications to multiscale analyses

proper orthogonal decomposition and its applications part i theory abstract in view of the increasing popularity of the application of proper orthogonal decomposition pod methods in engineering fields and the loose description of connections among the pod methods the purpose of this paper is to give a summary of the pod methods and to show the connections among these methods firstly the derivation and the performance of the three pod methods karhunen loeve decomposition kld principal component analysis pca and singular value decomposition svd are summarized then the equivalence problem is discussed via a theoretical comparison among the three methods the equivalence of the matrices for processing the objective functions the optimal basis vectors the mean square errors and the asymptotic connections of the three methods are demonstrated and proved when the methods are used to handle the pod of discrete random vectors

adaptive subgradient methods for online learning and stochastic optimization we present a new family of subgradient methods that dynamically incorporate knowledge of the geometry of the data observed in earlier iterations to perform more informative gradient based learning metaphorically the adaptation allows us to find needles in haystacks in the form of very predictive but rarely seen features our paradigm stems from recent advances in stochastic optimization and online learning which employ proximal functions to control the gradient steps of the algorithm we describe and analyze an apparatus for adaptively modifying the proximal function which significantly simplifies setting a learning rate and results in regret guarantees that are provably as good as the best proximal function that can be chosen in hindsight we give several efficient algorithms for empirical risk minimization problems with common and important regularization functions and domain constraints we experimentally study our theoretical analysis and show that adaptive subgradient methods outperform state of the art yet non adaptive subgradient algorithms

a class of statistical and spectral distance measures based on bose einstein statistics a class of statistical distance measures and their spectral counterparts are presented they have strong physical foundations since they are based on the combinatorial law leading to bose einstein statistics in statistical physics it is shown that these distance measures are very closely related to the recently introduced jensen shannon divergence measure the kullback leibler information 1951 number is found to be a limit case of this class

divergence measures based on the shannon entropy a novel class of information theoretic divergence measures based on the shannon entropy is introduced unlike the well known kullback divergences the new measures do not require the condition of absolute continuity to be satisfied by the probability distributions involved more importantly their close relationship with the variational distance and the probability of misclassification error are established in terms of bounds these bounds are crucial in many applications of divergence measures the measures are also well characterized by the properties of nonnegativity finiteness semiboundedness and boundedness

imposing essential boundary conditions in the natural element method by means of density scaled shapes a new method is proposed for the exact imposition of essential boundary conditions in the context of the natural element method nem this is a new technique in the field of computational mechanics and can be considered as a meshless method unlike most of these methods the ne shape functions are strictly interpolant and the essential boundary conditions can be imposed by directly substituting the corresponding terms in the system of equations however these shape functions are not strictly linear over non convex boundaries and the approach does not make the test functions vanish over the whole essential boundary region a modification of the initial nem version is considered based on shapes and complexes which are widely used in the field of scientific visualization using shapes in the context of the nem allows the construction of models entirely in terms of nodes and also ensures the linear precision of the interpolant over convex and non convex boundaries results on some benchmark problems are presented after a theoretical description of the method copyright 2000 john wiley sons ltd