confidence regions for stochastic variational inequalities the sample average approximation saa method is a basic approach for solving stochastic variational inequalities svi it is well known that under appropriate conditions the saa solutions provide asymptotically consistent point estimators for the true solution to an svi it is of fundamental interest to use such point estimators along with suitable central limit results to develop confidence regions of prescribed level of significance for the true solution however standard procedures are not applicable because the central limit theorem that governs the asymptotic behavior of saa solutions involves a discontinuous function evaluated at the true solution of the svi this paper overcomes such a difficulty by exploiting the precise geometric structure of the variational inequalities and by appealing to certain large deviations probability estimates and proposes a method to build asymptotically exact confidence regions for the true solution that are computable from the saa solutions we justify this method theoretically by establishing a precise limit theorem apply it to complementarity problems and test it with a linear complementarity problem

efficient solution of rational conics we present efficient algorithms for solving legendre equations over q equivalently for finding rational points on rational conics and parametrizing all solutions unlike existing algorithms no integer factorization is required provided that the prime factors of the discriminant are known

symmetrization approach to concentration inequalities for empirical processes we introduce a symmetrization technique that allows us to translate a problem of controlling the deviation of some functionals on a product space from their mean into a problem of controlling the deviation between two independent copies of the functional as an application we give a new easy proof of talagrand s concentration inequality for empirical processes where besides symmetrization we use only talagrand s concentration inequality on the discrete cube 0 1 n as another application of this technique we prove new vapnik chervonenkis type inequalities for example for vc classes of functions we prove a classical inequality of vapnik and chervonenkis only with normalization by the sum of variance and sample variance

variance based extragradient methods with line search for stochastic variational inequalities in this paper we propose dynamic sampled stochastic approximated ds sa extragradient methods for stochastic variational inequalities svis that are robust with respect to an unknown lipschitz c

information theoretic lower bounds on the oracle complexity of stochastic convex optimization relative to the large literature on upper bounds on complexity of convex optimization lesser attention has been paid to the fundamental hardn4516420ess of these problems given the extensive use of convex optimization in machine learning and statistics gaining an understanding of these complexity theoretic issues is important in this paper we study the complexity of stochastic convex optimization in an oracle model of computation we introduce a new notion of discrepancy between functions and use it to reduce problems of stochastic convex optimization to statistical parameter estimation which can be lower bounded using information theoretic methods using this approach we improve upon known results and obtain tight minimax complexity estimates for various function classes

robust stochastic approximation approach to stochastic programming in this paper we consider optimization problems where the objective function is given in a form of the expectation a basic difficulty of solving such stochastic optimization problems is that the involved multidimensional integrals expectations cannot be computed with high accuracy the aim of this paper is to compare two computational approaches based on monte carlo sampling techniques namely the stochastic approximation sa and the sample average approximation saa methods both approaches the sa and saa methods have a long history current opinion is that the saa method can efficiently use a specific say linear structure of the considered problem while the sa approach is a crude subgradient method which often performs poorly in practice we intend to demonstrate that a properly modified sa approach can be competitive and even significantly outperform the saa method for a certain class of convex stochastic problems we extend the analysis to the case of convex concave stochastic saddle point problems and present in our opinion highly encouraging results of numerical experiments

an optimal method for stochastic composite optimization this paper considers an important class of convex programming cp problems namely the stochastic composite optimization sco whose objective function is given by the summation of general nonsmooth and smooth stochastic components since sco covers non smooth smooth and stochastic cp as certain special cases a valid lower bound on the rate of convergence for solving these problems is known from the classic complexity theory of convex programming note however that the optimization algorithms that can achieve this lower bound had never been developed in this paper we show that the simple mirror descent stochastic approximation method exhibits the best known rate of convergence for solving these problems our major contribution is to introduce the accelerated stochastic approximation ac sa algorithm based on nesterov s optimal method for smooth cp nesterov in doklady an sssr 269 543 547 1983 nesterov in math program 103 127 152 2005 and show that the ac sa algorithm can achieve the aforementioned lower bound on the rate of convergence for sco to the best of our knowledge it is also the first universally optimal algorithm in the literature for solving non smooth smooth and stochastic cp problems we illustrate the significant advantages of the ac sa algorithm over existing methods in the context of solving a special but broad class of stochastic programming problems

adaptivity of averaged stochastic gradient descent to local strong convexity for logistic regression in this paper we consider supervised learning problems such as logistic regression and study the stochastic gradient method with averaging in the usual stochastic approximation setting where observations are used only once we show that after n iterations with a constant step size proportional to 1 r2 n where n is the number of observations and r is the maximum norm of the observations the convergence rate is always of order o 1 n and improves to o r2 n where is the lowest eigenvalue of the hessian at the global optimum when this eigenvalue is greater than r2 n since does not need to be known in advance this shows that averaged stochastic gradient is adaptive to unknown local strong convexity of the objective function our proof relies on the generalized self concordance properties of the logistic loss and thus extends to all generalized linear models with uniformly bounded features

learning by mirror averaging given a collection of m different estimators or classifiers we study the problem of model selection type aggregation i e we construct a new estimator or classifier called aggregate which is nearly as good as the best among them with respect to a given risk criterion we define our aggregate by a simple recursive procedure which solves an auxiliary stochastic linear programming problem related to the original non linear one and constitutes a special case of the mirror averaging algorithm we show that the aggregate satisfies sharp oracle inequalities under some general assumptions the results allow one to construct in an easy way sharp adaptive nonparametric estimators for several problems including regression classification and density estimation

extragradient method with variance reduction for stochastic variational inequalities we propose an extragradient method with stepsizes bounded away from zero for stochastic variational inequalities requiring only pseudomonotonicity we provide convergence and complexity analysis a

accelerated gradient methods for nonconvex nonlinear and stochastic programming in this paper we generalize the well known nesterov s accelerated gradient ag method originally designed for convex smooth optimization to solve nonconvex and possibly stochastic optimization problems we demonstrate that by properly specifying the stepsize policy the ag method exhibits the best known rate of convergence for solving general nonconvex smooth optimization problems by using first order information similarly to the gradient descent method we then consider an important class of composite optimization problems and show that the ag method can solve them uniformly i e by using the same aggressive stepsize policy as in the convex case even if the problem turns out to be nonconvex we demonstrate that the ag method exhibits an optimal rate of convergence if the composite problem is convex and improves the best known rate of convergence if the problem is nonconvex based on the ag method we also present new nonconvex stochastic approximation methods and show that they can improve a few existing rates of convergence for nonconvex stochastic optimization to the best of our knowledge this is the first time that the convergence of the ag method has been established for solving nonconvex nonlinear programming in the literature

beyond the regret minimization barrier optimal algorithms for stochastic strongly convex optimization we give novel algorithms for stochastic strongly convex optimization in the gradient oracle model which return a o 1 t approximate solution after t iterations the first algorithm is deterministic and achieves this rate via gradient updates and historical averaging the second algorithm is randomized and is based on pure gradient steps with a random step size his rate of convergence is optimal in the gradient oracle model this improves upon the previously known best rate of o log t t which was obtained by applying an online strongly convex optimization algorithm with regret o log t to the batch setting we complement this result by proving that any algorithm has expected regret of log t in the online stochastic strongly convex optimization setting this shows that any online to batch conversion is inherently suboptimal for stochastic strongly convex optimization this is the first formal evidence that online convex optimization is strictly more difficult than batch stochastic convex optimization

solving monotone stochastic variational inequalities and complementarity problems by progressive hedging the concept of a stochastic variational inequality has recently been articulated in a new way that is able to cover in particular the optimality conditions for a multistage stochastic programming problem one of the long standing methods for solving such an optimization problem under convexity is the progressive hedging algorithm that approach is demonstrated here to be applicable also to solving multistage stochastic variational inequality problems under monotonicity thus increasing the range of applications for progressive hedging stochastic complementarity problems as a special case are explored numerically in a linear two stage formulation

solving lagrangian variational inequalities with applications to stochastic programming lagrangian variational inequalities feature both primal and dual elements in expressing first order conditions for optimality in a wide variety of settings where multipliers in a very general sense need to be brought in their stochastic version relates to problems of stochastic programming and covers not only classical formats with inequality constraints but also composite models with nonsmooth objectives the progressive hedging algorithm as a means of solving stochastic programming problems has however focused so far only on optimality conditions that correspond to variational inequalities in primal variables alone here that limitation is removed by appealing to a recent extension of progressive hedging to multistage stochastic variational inequalities in general

on variance reduction for stochastic smooth convex optimization with multiplicative noise we propose dynamic sampled stochastic approximation sa methods for stochastic optimization with a heavy tailed distribution with finite 2nd moment the objective is the sum of a smooth convex function with a convex regularizer typically it is assumed an oracle with an upper bound sigma 2 on its variance oubv differently we assume an oracle with multiplicative noise this rarely addressed setup is more aggressive but realistic where the variance may not be uniformly bounded our methods achieve optimal iteration complexity and near optimal oracle complexity for the smooth convex class we use an accelerated sa method a la fista which achieves given tolerance varepsilon 0 the optimal iteration complexity of mathscr o varepsilon frac 1 2 with a near optimal oracle complexity of mathscr o varepsilon 2 ln varepsilon frac 1 2 2 this improves upon ghadimi and lan math program 156 59 99 2016 where it is assumed an oubv for the strongly convex class our method achieves optimal iteration complexity of mathscr o ln varepsilon 1 and optimal oracle complexity of mathscr o varepsilon 1 this improves upon byrd et al math program 134 127 155 2012 where it is assumed an oubv in terms of variance our bounds are local they depend on variances sigma x 2 at solutions x and the per unit distance multiplicative variance sigma 2 l for the smooth convex class there exist policies such that our bounds resemble up to absolute constants those obtained in the mentioned papers if it was assumed an oubv with sigma 2 sigma x 2 for the strongly convex class such property is obtained exactly if the condition number is estimated or in the limit for better conditioned problems or for larger initial batch sizes in any case if it is assumed an oubv our bounds are thus sharper since typically max sigma x 2 sigma l 2 ll sigma 2

writing projective representations over subfields let g x be an absolutely irreducible subgroup of gl d k and let f be a proper subfield of the finite field k we present a practical algorithm to decide constructively whether or not g is conjugate to a subgroup of gl d f k where k denotes the centre of gl d k if the derived group of g also acts absolutely irreducibly then the algorithm is las vegas and costs o x d3 d2log f arithmetic operations in k this work forms part of a recognition project based on aschbacher s classification of maximal subgroups of gl d k

performance measures for optimal constrained beamformers in this paper the closed form expression is derived which relates the output signal to interference plus noise ratio to the minimum mean square error of constrained beamformers for narrowband and wideband signal environments

minimizing buffered probability of exceedance by progressive hedging stochastic programming problems have for a long time been posed in terms of minimizing the expected value of a random variable influenced by decision variables but alternative objectives can also be considered such as minimizing a measure of risk here something different is introduced minimizing the buffered probability of exceedance for a specified loss threshold the buffered version of the traditional concept of probability of exceedance has recently been developed with many attractive properties that are conducive to successful optimization in contrast to the usual concept which is often posed simply as the probability of failure the main contribution here is to demonstrate that in minimizing buffered probability of exceedance the underlying convexities in a stochastic programming problem can be maintained and the progressive hedging algorithm can be employed to compute a solution

equilibrium uncertainty and risk in hydro thermal electricity systems the correspondence of competitive partial equilibrium with a social optimum is well documented in the welfare theorems of economics these theorems can be applied to single period electricity pool auctions in which price taking agents maximize profits at competitive prices and extend naturally to standard models with locational marginal prices in hydro thermal markets where the auctions are repeated over many periods agents seek to optimize their current and future profit accounting for future prices that depend on uncertain inflows this makes the agent problems multistage stochastic optimization models but perfectly competitive partial equilibrium still corresponds to a social optimum when all agents are risk neutral and share common knowledge of the probability distribution governing future inflows the situation is complicated when agents are risk averse in this setting we show under mild conditions that a social optimum corresponds to a competitive market equilibrium if agents have time consistent dynamic coherent risk measures and there are enough traded market instruments to hedge inflow uncertainty we illustrate some of the consequences of risk aversion on market outcomes using a simple two stage competitive equilibrium model with three agents

a simple fast adaptive algorithm for broad band null steering arrays a simple algorithm for adaptively steering the broadband null in a two element power inversion array which uses a tapped delay line filter as the processing element is presented and investigated by performing a simple signal transformation the algorithm constrains some of the filter weights so that a variable length delay line is approximated in the maximally flat sense by using the least mean square lms algorithm to adjust the weights controlling the length and gain of the delay line a broadband null is then adaptively formed to track and reject the broadband jammer as studied analytically and by using simulation results the convergence behavior of the algorithm is fast and depends only on the tolerable level of misadjustment noise its implementation complexity is comparable to that of using the lms algorithm directly

hybrid deterministic stochastic methods for data fitting many structured data fitting applications require the solution of an optimization problem involving a sum over a potentially large number of measurements incremental gradient algorithms offer inexpensive iterations by sampling a subset of the terms in the sum these methods can make great progress initially but often slow as they approach a solution in contrast full gradient methods achieve steady convergence at the expense of evaluating the full objective and gradient on each iteration we explore hybrid methods that exhibit the benefits of both approaches rate of convergence analysis shows that by controlling the sample size in an incremental gradient algorithm it is possible to maintain the steady convergence rates of full gradient methods we detail a practical quasi newton implementation based on this approach numerical experiments illustrate its potential benefits

a fast null steering algorithm for linearly constrained adaptive arrays a fast null steering algorithm for adaptive arrays with a look direction constraint is investigated in this algorithm the least mean square lms algorithm is used to adaptively adjust the positions of the array response zeros one after another in a time multiplexed manner to track unknown jammers to ensure that the look direction response and the other zeros remain unchanged when a particular null is being updated the element signals are suitably transformed before the lms updating despite this signal transformation the new algorithm has almost the same implementation complexity as when the lms algorithm is employed directly as only one zero is being adaptively updated at any instant however with each null being independently steered the convergence behavior of the new algorithm is significantly faster and is almost independent of the external noise environment

adaptive beam forming using a cascade configuration a new technique for adaptive beam forming based on the davies cascade array configuration 11 is presented first of all it is shown that an n element linear array can be reconfigured as a number of subarrays of two elements each this two element array is capable of steering one null in an arbitrary direction while maintaining a desired gain in some other prespecified look direction to adaptively perform this function at any cascade stage the inverse of a 2 2 correlation matrix of the signal vector at a subarray is computed and substituted in the expression for the constrained optimum weight vector similar processing is performed at subsequent stages of the cascade configuration at each stage of adaptation a noise source contributing maximum power at that stage is nulled out simulation studies carried out verify the validity of this approach and demonstrate the fast convergence property of the proposed technique

adaptive zero tracking complemented by lms weight update for guaranteed convergence a low complexity zero tracking algorithm with fast and guaranteed convergence behavior is proposed and investigated the algorithm implements a zero tracking algorithm and a least mean square lms weight update algorithm in parallel with the former adjusting the zeros of the array in a time multiplexed manner to achieve fast asymptotic and tracking behavior while the latter improves the initial transient behavior of the algorithm and guarantees its convergence to the global optimum by comparing their output powers the relative performances of the two component algorithms are monitored and re initialization of one algorithm by the other may occur periodically this gives the algorithm both fast and guaranteed convergence behavior even though the zeros are directly available and the implementation complexity is only two times that for the conventional lms algorithm

performance analysis of a wavelet based generalized sidelobe canceller in this paper we carry out a detailed performance analysis of the recently addressed wavelet based generalized sidelobe canceller gsc which employs a set of p regular m band wavelet filters in the design of the blocking matrix involved we begin our analysis by developing a closed form expression of the output signal to interference plus noise ratio sinr of partially adaptive beamformers with the gsc as the underlying structure under a one jammer or multiple jammer environment this new expression is shown to encompass several previous works as special cases we then examine the relationship between the choices of the parameters p and m in the wavelet based gsc and the corresponding output sinr based on this expression some guidelines for the choices of these parameters can thus be induced to achieve superior performance simulation results are also furnished to justify these suggestions

optimal stochastic approximation algorithms for strongly convex stochastic composite optimization i a generic algorithmic framework in this paper we present a generic algorithmic framework namely the accelerated stochastic approximation ac sa algorithm for solving strongly convex stochastic composite optimization sco problems while the classical stochastic approximation algorithms are asymptotically optimal for solving differentiable and strongly convex problems the ac sa algorithm when employed with proper stepsize policies can achieve optimal or nearly optimal rates of convergence for solving different classes of sco problems during a given number of iterations moreover we investigate these ac sa algorithms in more detail such as by establishing the large deviation results associated with the convergence rates and introducing an efficient validation procedure to check the accuracy of the generated solutions

writing representations over proper division subrings abstract let e be a division ring and g a finite group of automorphisms of e whose elements are distinct modulo inner automorphisms of e let f e g be the division subring of elements of e fixed by g given a representation a e d d of an f algebra a we give necessary and sufficient conditions for to be writable over f here e d d denotes the algebra of d d matrices over e and a matrix a writes over f if a 1 a a f d d we give an algorithm for constructing an a or proving that no a exists the case of particular interest to us is when e is a field and is absolutely irreducible the algorithm relies on an explicit formula for a and a generalization of hilbert s theorem 90 that arises in galois cohomology the algorithm has applications to the construction of absolutely irreducible group representations especially for solvable groups and to the recognition of class c 5 in aschbacher s matrix group classification scheme m aschbacher on the maximal subgroups of the finite classical groups invent math 76 1984 469 514 mr0746539 shangzhi li on the subgroup structure of classical groups in group theory in china in math appl china ser vol 365 kluwer acad publ dordrecht 1996 pp 70 90 mr1447199 1 13

variable number sample path optimization the sample path method is one of the most important tools in simulation based optimization the basic idea of the method is to approximate the expected simulation output by the average of sample observations with a common random number sequence in this paper we describe a new variant of powell s unconstrained optimization by quadratic approximation uobyqa method which integrates a bayesian variable number sample path vnsp scheme to choose appropriate number of samples at each iteration the statistically accurate scheme determines the number of simulation runs and guarantees the global convergence of the algorithm the vnsp scheme saves a significant amount of simulation operations compared to general purpose fixed number sample path methods we present numerical results based on the new algorithm

on solving relative norm equations in algebraic number fields let q f be algebraic number fields and m f a free o e module we prove a theorem which enables us to determine whether a given relative norm equation of the form n f e has any solutions m at all and if so to compute a complete set of nonassociate solutions finally we formulate an algorithm using this theorem consider its algebraic complexity and give some examples

splitting full matrix algebras over algebraic number fields abstract let k be a fixed algebraic number field and let a be an associative algebra over k given by structure constants such that a m n k holds for some positive integer n suppose that n is bounded then an isomorphism a m n k can be constructed by a polynomial time ff algorithm an ff algorithm is a deterministic procedure which is allowed to call oracles for factoring integers and factoring univariate polynomials over finite fields as a consequence we obtain a polynomial time ff algorithm to compute isomorphisms of central simple algebras of bounded degree over k

accelerated schemes for a class of variational inequalities we propose a novel stochastic method namely the stochastic accelerated mirror prox samp method for solving a class of monotone stochastic variational inequalities svi the main idea of the proposed algorithm is to incorporate a multi step acceleration scheme into the stochastic mirror prox method the developed samp method computes weak solutions with the optimal iteration complexity for svis in particular if the operator in svi consists of the stochastic gradient of a smooth function the iteration complexity of the samp method can be accelerated in terms of their dependence on the lipschitz constant of the smooth function for svis with bounded feasible sets the bound of the iteration complexity of the samp method depends on the diameter of the feasible set for unbounded svis we adopt the modified gap function introduced by monteiro and svaiter for solving monotone inclusion and show that the iteration complexity of the samp method depends on the distance from the initial point to the set of strong solutions it is worth noting that our study also significantly improves a few existing complexity results for solving deterministic variational inequality problems we demonstrate the advantages of the samp method over some existing algorithms through our preliminary numerical experiments

confidence level solutions for stochastic programming we propose an alternative approach to stochastic programming based on monte carlo sampling and stochastic gradient optimization the procedure is by essence probabilistic and the computed solution is a random variable we propose a solution concept in which the probability that the random algorithm produces a solution with an expected objective value departing from the optimal one by more than e is small enough we derive complexity bounds on the number of iterations of this process we show that by repeating the basic process on independent samples one can significantly reduce the number of iterations

discrete approximation of two stage stochastic and distributionally robust linear complementarity problems in this paper we propose a discretization scheme for the two stage stochastic linear complementarity problem lcp where the underlying random data are continuously distributed under some moderate conditions we derive qualitative and quantitative convergence for the solutions obtained from solving the discretized two stage stochastic lcp slcp we explain how the discretized two stage slcp may be solved by the well known progressive hedging method phm moreover we extend the discussion by considering a two stage distributionally robust lcp drlcp with moment constraints and proposing a discretization scheme for the drlcp as an application we show how the slcp and drlcp models can be used to study equilibrium arising from two stage duopoly game where each player plans to set up its optimal capacity at present with anticipated competition for production in future

primal dual subgradient methods for convex problems in this paper we present a new approach for constructing subgradient schemes for different types of nonsmooth problems with convex structure our methods are primal dual since they are always able to generate a feasible approximation to the optimum of an appropriately formulated dual problem besides other advantages this useful feature provides the methods with a reliable stopping criterion the proposed schemes differ from the classical approaches divergent series methods mirror descent methods by presence of two control sequences the first sequence is responsible for aggregating the support functions in the dual space and the second one establishes a dynamically updated scale between the primal and dual spaces this additional flexibility allows to guarantee a boundedness of the sequence of primal test points even in the case of unbounded feasible set however we always assume the uniform boundedness of subgradients we present the variants of subgradient schemes for nonsmooth convex minimization minimax problems saddle point problems variational inequalities and stochastic optimization in all situations our methods are proved to be optimal from the view point of worst case black box lower complexity bounds

risk minimization regret minimization and progressive hedging algorithms this paper begins with a study on the dual representations of risk and regret measures and their impact on modeling multistage decision making under uncertainty a relationship between risk envelopes and regret envelopes is established by using the lagrangian duality theory such a relationship opens a door to a decomposition scheme called progressive hedging for solving multistage risk minimization and regret minimization problems in particular the classical progressive hedging algorithm is modified in order to handle a new class of linkage constraints that arises from reformulations and other applications of risk and regret minimization problems numerical results are provided to show the efficiency of the progressive hedging algorithms

improved methods for calculating vectors of short length in a lattice including a complexity analysis the standard methods for calculating vectors of short length in a lattice use a reduction procedure followed by enumerating all vectors of z in a suitable box however it suffices to consider those x e z which lie in a suitable ellipsoid having a much smaller volume than the box we show in this paper that searching through that ellipsoid is in many cases much more efficient if combined with an appropriate reduction procedure our method allows to do computations in lattices of much higher dimensions several randomly constructed numerical examples illustrate the superiority of our new method over the known ones

optimal stochastic approximation algorithms for strongly convex stochastic composite optimization ii shrinking procedures and optimal algorithms in this paper we study new stochastic approximation sa type algorithms namely the accelerated sa ac sa for solving strongly convex stochastic composite optimization sco problems specifically by introducing a domain shrinking procedure we significantly improve the large deviation results associated with the convergence rate of a nearly optimal ac sa algorithm presented by ghadimi and lan in siam j optim 22 2012 pp 1469 1492 moreover we introduce a multistage ac sa algorithm which possesses an optimal rate of convergence for solving strongly convex sco problems in terms of the dependence on not only the target accuracy but also a number of problem parameters and the selection of initial points to the best of our knowledge this is the first time that such an optimal method has been presented in the literature from our computational results these ac sa algorithms can substantially outperform the classical sa and some other sa type algorithms for solving certain classes of strongly

scenarios and policy aggregation in optimization under uncertainty a common approach in coping with multiperiod optimization problems under uncertainty where statistical information is not really enough to support a stochastic programming model has been to set up and analyze a number of scenarios the aim then is to identify trends and essential features on which a robust decision policy can be based this paper develops for the first time a rigorous algorithmic procedure for determining such a policy in response to any weighting of the scenarios the scenarios are bundled at various levels to reflect the availability of information and iterative adjustments are made to the decision policy to adapt to this structure and remove the dependence on hindsight

on perturbed proximal gradient algorithms we study a version of the proximal gradient algorithm for which the gradient is intractable and is approximated by monte carlo methods and in particular markov chain monte carlo we derive conditions on the step size and the monte carlo batch size under which convergence is guaranteed both increasing batch size and constant batch size are considered we also derive non asymptotic bounds for an averaged version our results cover both the cases of biased and unbiased monte carlo approximation to support our findings we discuss the inference of a sparse generalized linear model with random effect and the problem of learning the edge structure and parameters of sparse undirected graphical models

on the performance of the generalized sidelobe canceller in coherent situations the convergence rate for the adaptive weights to reach the optimum value in an adaptive array system depends on the eigenvalue spread ratio of the autocovariance matrix how the eigenvalue spread ratio in a generalized sidelobe canceller gsc is affected by the various parameters is studied expressions for the output power of the gsc in coherent situations are derived and numerical results are included

sample size selection in optimization methods for machine learning this paper presents a methodology for using varying sample sizes in batch type optimization methods for large scale machine learning problems the first part of the paper deals with the delicate issue of dynamic sample selection in the evaluation of the function and gradient we propose a criterion for increasing the sample size based on variance estimates obtained during the computation of a batch gradient we establish an o 1 epsilon complexity bound on the total cost of a gradient method the second part of the paper describes a practical newton method that uses a smaller sample to compute hessian vector products than to evaluate the function and the gradient and that also employs a dynamic sampling technique the focus of the paper shifts in the third part of the paper to l 1 regularized problems designed to produce sparse solutions we propose a newton like method that consists of two phases a minimalistic gradient projection phase that identifies zero variables and subspace phase that applies a subsampled hessian newton iteration in the free variables numerical tests on speech recognition problems illustrate the performance of the algorithms

a lie algebra method for rational parametrization of severi brauer surfaces it is well known that a severi brauer surface has a rational point if and only if it is isomorphic to the projective plane given a severi brauer surface we study the problem to decide whether such an isomorphism to the projective plane or such a rational point does exist and to construct such an isomorphism or such a point in the affirmative case we give an algorithm using lie algebra techniques the algorithm has been implemented in magma

on the arithmetic of del pezzo surfaces of degree 2 we study the arithmetic of certain del pezzo surfaces of degree 2 we produce examples of brauer manin obstruction to the hasse principle coming from 2 and 4 torsion elements in the brauer group

acceleration of stochastic approximation by averaging a new recursive algorithm of stochastic approximation type with the averaging of trajectories is investigated convergence with probability one is proved for a variety of classical optimization and identification problems it is also demonstrated for these problems that the proposed algorithm achieves the highest possible rate of convergence

the output sinr of the linearly constrained beamformer the authors derive expressions of the output signal to interference plus noise ratio sinr of the linearly constrained narrowband beamformer in noncoherent situations using a vector approach numerical results are included

solving quadratic equations using reduced unimodular quadratic forms let q be an n x n symmetric matrix with integral entries and with det q 0 but not necesarily positive definite we describe a generalized lll algorithm to reduce this quadratic form this algorithm either reduces the quadratic form or stops with some isotropic vector it is proved to run in polynomial time we also describe an algorithm for the minimization of a ternary quadratic form when a quadratic equation q x y z 0 is solvable over q a solution can be deduced from another quadratic equation of determinant 1 the combination of these algorithms allows us to solve efficiently any general ternary quadratic equation over q and this gives a polynomial time algorithm as soon as the factorization of the determinant of q is known

solving stochastic complementarity problems in energy market modeling using scenario reduction in this paper we analyze market equilibrium models with random aspects that lead to stochastic complementarity problems while the models presented depict energy markets the results are believed to be applicable to more general stochastic complementarity problems the contribution is the development of new heuristic scenario reduction approaches that iteratively work towards solving the full extensive form stochastic market model the methods are tested on three representative models and supporting numerical results are provided as well as derived mathematical bounds

a nonsmooth version of newton s method newton s method for solving a nonlinear equation of several variables is extended to a nonsmooth case by using the generalized jacobian instead of the derivative this extension includes the b derivative version of newton s method as a special case convergence theorems are proved under the condition of semismoothness it is shown that the gradient function of the augmented lagrangian forc2 nonlinear programming is semismooth thus the extended newton s method can be used in the augmented lagrangian method for solving nonlinear programs

manifold identification in dual averaging for regularized stochastic online learning iterative methods that calculate their steps from approximate subgradient directions have proved to be useful for stochastic learning problems over large and streaming data sets when the objective consists of a loss function plus a nonsmooth regularization term the solution often lies on a low dimensional manifold of parameter space along which the regularizer is smooth when an l1 regularizer is used to induce sparsity in the solution for example this manifold is defined by the set of nonzero components of the parameter vector this paper shows that a regularized dual averaging algorithm can identify this manifold with high probability before reaching the solution this observation motivates an algorithmic strategy in which once an iterate is suspected of lying on an optimal or near optimal manifold we switch to a local phase that searches in this manifold thus converging rapidly to a near optimal point computational results are presented to verify the identification property and to illustrate the effectiveness of this approach

an lll algorithm with quadratic complexity the lenstra lenstra lovasz lattice basis reduction algorithm called lll or rm l 3 is a fundamental tool in computational number theory and theoretical computer science which can be viewed as an efficient algorithmic version of hermite s inequality on hermite s constant given an integer d dimensional lattice basis with vectors of euclidean norm less than b in an n dimensional space the rm l 3 algorithm outputs a reduced basis in o d 3n rm log b cdot mathcal m d rm log b bit operations where mathcal m k denotes the time required to multiply k bit integers this worst case complexity is problematic for applications where d or and rm log b are often large as a result the original rm l 3 algorithm is almost never used in practice except in tiny dimension instead one applies floating point variants where the long integer arithmetic required by gram schmidt orthogonalization is replaced by floating point arithmetic unfortunately this is known to be unstable in the worst case the usual floating point rm l 3 algorithm is not even guaranteed to terminate and the output basis may not be rm l 3 reduced at all in this article we introduce the rm l 2 algorithm a new and natural floating point variant of the rm l 3 algorithm which provably outputs rm l 3 reduced bases in polynomial time o d 2n d rm log b rm log b cdot mathcal m d this is the first rm l 3 algorithm whose running time without fast integer arithmetic provably grows only quadratically with respect to rm log b like euclid s gcd algorithm and lagrange s two dimensional algorithm

solving norm equations in relative number fields using s units in this paper we are interested in solving the so called norm equation m l k x a where l k is a given arbitrary extension of number fields and a a given algebraic number of k by considering s units and relative class groups we show that if there exists at least one solution in l but not necessarily in z l then there exists a solution for which we can describe precisely its prime ideal factorization in fact we prove that under some explicit conditions the s units that are norms are norms of s units this allows us to limit the search for rational solutions to a finite number of tests and we give the corresponding algorithm when a is an algebraic integer we also study the existence of an integral solution and we can adapt the algorithm to this case

incremental constraint projection methods for monotone stochastic variational inequalities we consider stochastic variational inequalities vis with monotone operators where the feasible set is an intersection of a large number of convex sets we propose a stochastic approximation metho

two stage quadratic games under uncertainty and their solution by progressive hedging algorithms a model of a two stage n person noncooperative game under uncertainty is studied in which at the first stage each player solves a quadratic program parameterized by other players decisions and