hub location as the minimization of a supermodular set function this paper highlights how a general class of hub location problems can be modeled as the minimization of a real valued supermodular set function well known problems such as uncapacitated hub location p hub median and hub arc location among others are shown to be particular cases of this class two integer programming formulations are introduced and compared one uses path based variables frequently employed in hub location whereas the other exploits properties of supermodular functions in addition several worst case bounds for a greedy and a local improvement heuristic are obtained for the general class and for some particular cases in which sharper bounds can be devised computational experiments are performed to compare both formulations when used with a general purpose solver computational results obtained on benchmark instances confirm the superiority of the supermodular formulation

an iterative reweighted method for tucker decomposition of incomplete tensors we consider the problem of low rank decomposition of incomplete tensors since many real world data lie on an intrinsically low dimensional subspace tensor low rank decomposition with missing entries has applications in many data analysis problems such as recommender systems and image inpainting in this paper we focus on tucker decomposition which represents an n th order tensor in terms of n factor matrices and a core tensor via multilinear operations to exploit the underlying multilinear low rank structure in high dimensional datasets we propose a group based log sum penalty functional to place structural sparsity over the core tensor which leads to a compact representation with smallest core tensor the proposed method is developed by iteratively minimizing a surrogate function that majorizes the original objective function this iterative optimization leads to an iteratively reweighted least squares algorithm in addition to reduce the computational complexity an over relaxed monotone fast iterative shrinkage thresholding technique is adapted and embedded in the iterative reweighted process the proposed method is able to determine the model complexity i e multilinear rank in an automatic way simulation results show that the proposed algorithm offers competitive performance compared with other existing algorithms

solving the uncapacitated multiple allocation hub location problem by means of a dual ascent technique this paper deals with the uncapacitated multiple allocation hub location problem the dual problem of a four indexed formulation is considered and a heuristic method based on a dual ascent technique is designed this heuristic which is reinforced with several specifical subroutines and does not require any external linear problem solver is the core tool embedded in an exact branch and bound framework besides the heuristic provides the branch and bound algorithm with good lower bounds for the nodes of the branching tree the results of the computational experience with the classical cab and ap data sets are included showing the great effectiveness of this approach instances with up to 120 nodes are solved

rank revealing qr factorizations abstract an algorithm is presented for computing a column permutation pi and a qr factorization a pi qr of an m by n m or n matrix a such that a possible rank deficiency of a will be revealed in the triangular factor r having a small lower right block for low rank deficient matrices the algorithm is guaranteed to reveal the rank of a and the cost is only slightly more than the cost of one regular qr factorization a posteriori upper and lower bounds on the singular values of a are derived and can be used to infer the numerical rank of a keywords qr factorization rank deficient matrices least squares computation subset selection rank singular values

online categorical subspace learning for sketching big data with misses with the scale of data growing every day reducing the dimensionality a k a sketching of high dimensional data has emerged as a task of paramount importance relevant issues to address in this context include the sheer volume of data that may consist of categorical observations the typically streaming format of acquisition and the possibly missing entries to cope with these challenges this paper develops a novel categorical subspace learning approach to unravel the latent structure for three prominent categorical bilinear models namely probit tobit and logit the deterministic probit and tobit models treat data as quantized values of an analog valued process lying in a low dimensional subspace while the probabilistic logit model relies on low dimensionality of the data log likelihood ratios leveraging the low intrinsic dimensionality of the sought models a rank regularized maximum likelihood estimator is devised which is then solved recursively via alternating majorization minimization to sketch high dimensional categorical data on the fly the resultant lightweight first order algorithms entail highly parallelizable tasks per iteration in addition the quantization thresholds are also learned jointly with the subspace to enhance the predictive power of the sought models performance of the subspace iterates is analyzed for both infinite and finite data streams where for the former asymptotic convergence to the stationary point set of the batch estimator is established while for the latter sublinear regret bounds are derived for the empirical cost simulated tests with both synthetic and real world datasets corroborate the merits of the novel schemes for real time movie recommendation and chess game classification

subspace learning and imputation for streaming big data matrices and tensors extracting latent low dimensional structure from high dimensional data is of paramount importance in timely inference tasks encountered with big data analytics however increasingly noisy heterogeneous and incomplete datasets as well as the need for real time processing of streaming data pose major challenges to this end in this context the present paper permeates benefits from rank minimization to scalable imputation of missing data via tracking low dimensional subspaces and unraveling latent possibly multi way structure from incomplete streaming data for low rank matrix data a subspace estimator is proposed based on an exponentially weighted least squares criterion regularized with the nuclear norm after recasting the nonseparable nuclear norm into a form amenable to online optimization real time algorithms with complementary strengths are developed and their convergence is established under simplifying technical assumptions in a stationary setting the asymptotic estimates obtained offer the well documented performance guarantees of the batch nuclear norm regularized estimator under the same unifying framework a novel online adaptive algorithm is developed to obtain multi way decompositions of low rank tensors with missing entries and perform imputation as a byproduct simulated tests with both synthetic as well as real internet and cardiac magnetic resonance imagery mri data confirm the efficacy of the proposed algorithms and their superior performance relative to state of the art alternatives

the tree of hubs location problem this paper presents the tree of hubs location problem it is a network hub location problem with single assignment where a fixed number of hubs have to be located with the particularity that it is required that the hubs are connected by means of a tree the problem combines several aspects of location network design and routing problems potential applications appear in telecommunications and transportation systems when set up costs for links between hubs are so high that full interconnection between hub nodes is prohibitive we propose an integer programming formulation for the problem furthermore we present some families of valid inequalities that reinforce the formulation and we give an exact separation procedure for them finally we present computational results using the well known ap and cab data sets

synchronizing sample curves nonparametrically more and more often the outcome of a study is not a random variable but a noisy function for each experimental unit resulting in a sample of curves typically the individual curves vary not only in amplitude or intensity but also with respect to the time axis different subjects experience certain events sooner or later analyzing such data involves finding out the time changes or curve registration among curves following our previous work where modified dynamic time warping is applied to align two curves we formulate a global minimization problem to align all curves in a sample and to compute the aligned average curve algorithms for solving the minimization problem are presented and tested with simulated and real data the test results are promising the method which involves kernel smoothing of regression functions estimates the time changes and the average of the aligned curves from noisy data large sample asymptotics is derived

rank regularization and bayesian inference for tensor completion and extrapolation a novel regularizer of the parafac decomposition factors capturing the tensor s rank is proposed in this paper as the key enabler for completion of three way data arrays with missing entries set in a bayesian framework the tensor completion method incorporates prior information to enhance its smoothing and prediction capabilities this probabilistic approach can naturally accommodate general models for the data distribution lending itself to various fitting criteria that yield optimum estimates in the maximum a posteriori sense in particular two algorithms are devised for gaussian and poisson distributed data that minimize the rank regularized least squares error and kullback leibler divergence respectively the proposed technique is able to recover the ground truth tensor rank when tested on synthetic data and to complete brain imaging and yeast gene expression datasets with 50 and 15 of missing entries respectively resulting in recovery errors at 11 db and 15 db

sampling from large matrices an approach through geometric functional analysis we study random submatrices of a large matrix a we show how to approximately compute a from its random submatrix of the smallest possible size o rlog r with a small error in the spectral norm where r e vav2f vav22 is the numerical rank of a the numerical rank is always bounded by and is a stable relaxation of the rank of a this yields an asymptotically optimal guarantee in an algorithm for computing low rank approximations of a we also prove asymptotically optimal estimates on the spectral norm and the cut norm of random submatrices of a the result for the cut norm yields a slight improvement on the best known sample complexity for an approximation algorithm for max 2csp problems we use methods of probability in banach spaces in particular the law of large numbers for operator valued random variables

critical cutsets of graphs and canonical facets of set packing polytopes a cutset in a graph g with node set n is the set of edges joining the nodes in a subset of n to those not in the subset we call a cutset critical if its removal produces a graph whose independence number is greater than that of g we use the concept of a critical cutset to give first a necessary then a sufficient condition for an inequality with 0 1 coefficients to be a facet of the set packing polytope defined by g

fast monte carlo algorithms for finding low rank approximations we consider the problem of approximating a given m n matrix a by another matrix of specified rank k which is smaller than m and n the singular value decomposition svd can be used to find the best such approximation however it takes time polynomial in m n which is prohibitive for some modern applications in this article we develop an algorithm that is qualitatively faster provided we may sample the entries of the matrix in accordance with a natural probability distribution in many applications such sampling can be done efficiently our main result is a randomized algorithm to find the description of a matrix d of rank at most k so that holds with probability at least 1 where v vf is the frobenius norm the algorithm takes time polynomial in k 1 e log 1 only and is independent of m and n in particular this implies that in constant time it can be determined if a given matrix of arbitrary size has a good low rank approximation

subspace orbit randomized decomposition for low rank matrix approximations an efficient accurate and reliable approximation of a matrix by one of lower rank is a fundamental task in numerical linear algebra and signal processing applications in this paper we introduce a new matrix decomposition approach termed subspace orbit randomized singular value decomposition sor svd which makes use of random sampling techniques to give a low rank approximation to an input matrix given a large and dense data matrix of size m times n the algorithm requires a few passes through data to compute a rank k approximation in o mnk floating point operations moreover the sor svd algorithm can utilize advanced computer architectures and as a result it can be optimized for maximum efficiency the sor svd algorithm is simple accurate and provably correct and outperforms previously reported techniques in terms of accuracy and efficiency our numerical experiments support these claims

finding structure with randomness probabilistic algorithms for constructing approximate matrix decompositions low rank matrix approximations such as the truncated singular value decomposition and the rank revealing qr decomposition play a central role in data analysis and scientific computing this work surveys and extends recent research which demonstrates that randomization offers a powerful tool for performing low rank matrix approximation these techniques exploit modern computational architectures more fully than classical methods and open the possibility of dealing with truly massive data sets this paper presents a modular framework for constructing randomized algorithms that compute partial matrix decompositions these methods use random sampling to identify a subspace that captures most of the action of a matrix the input matrix is then compressed either explicitly or implicitly to this subspace and the reduced matrix is manipulated deterministically to obtain the desired low rank factorization in many cases this approach beats its classical competitors in terms of accuracy robustness and or speed these claims are supported by extensive numerical experiments and a detailed error analysis the specific benefits of randomized techniques depend on the computational environment consider the model problem of finding the k dominant components of the singular value decomposition of an m times n matrix i for a dense input matrix randomized algorithms require bigo mn log k floating point operations flops in contrast to bigo mnk for classical algorithms ii for a sparse input matrix the flop count matches classical krylov subspace methods but the randomized approach is more robust and can easily be reorganized to exploit multiprocessor architectures iii for a matrix that is too large to fit in fast memory the randomized techniques require only a constant number of passes over the data as opposed to bigo k passes for classical algorithms in fact it is sometimes possible to perform matrix approximation with a single pass over the data

projection approximation subspace tracking subspace estimation plays an important role in a variety of modern signal processing applications we present a new approach for tracking the signal subspace recursively it is based on a novel interpretation of the signal subspace as the solution of a projection like unconstrained minimization problem we show that recursive least squares techniques can be applied to solve this problem by making an appropriate projection approximation the resulting algorithms have a computational complexity of o nr where n is the input vector dimension and r is the number of desired eigencomponents simulation results demonstrate that the tracking capability of these algorithms is similar to and in some cases more robust than the computationally expensive batch eigenvalue decomposition relations of the new algorithms to other subspace tracking methods and numerical issues are also discussed

on uniqueness in candecomp parafac one of the basic issues in the analysis of three way arrays by candecomp parafac cp has been the question of uniqueness of the decomposition kruskal 1977 has proved that uniqueness is guaranteed when the sum of thek ranks of the three component matrices involved is at least twice the rank of the solution plus 2 since then little has been achieved that might further qualify kruskal s sufficient condition attempts to prove that it is also necessary for uniqueness except for rank 1 or 2 have failed but counterexamples to necessity have not been detected the present paper gives a method for generating the class of all solutions or at least a subset of that class given a cp solution that satisfies certain conditions this offers the possibility to examine uniqueness for a great variety of specific cp solutions it will be shown that kruskal s condition is necessary and sufficient when the rank of the solution is three but that uniqueness may hold even if the condition is not satisfied when the rank is four or higher

adaptive eigensubspace algorithms for direction or frequency estimation and tracking the authors present an adaptive estimator of the complete noise or signal subspace of a sample covariance matrix as well as the estimator s practical implementations the general formulation of the proposed estimator results from an asymptotic argument which shows the signal or noise subspace computation to be equivalent to a constrained gradient search procedure a highly parallel algorithm denoted the inflation method is introduced for the estimation of the noise subspace the simulation results of these adaptive estimators show that the adaptive subspace algorithms perform substantially better than p a thompson s 1980 adaptive version of v f pisarenko s technique 1973 in estimating frequencies or directions of arrival doa of plane waves for tracking nonstationary parameters the simulation results also show that the adaptive subspace algorithms are better than direct eigendecomposition methods for which computational complexity is much higher than the adaptive versions

coherence pursuit fast simple and robust principal component analysis this paper presents a remarkably simple yet powerful algorithm termed coherence pursuit cop to robust principal component analysis pca as inliers lie in a low dimensional subspace and are mostly correlated an inlier is likely to have strong mutual coherence with a large number of data points by contrast outliers either do not admit low dimensional structures or form small clusters in either case an outlier is unlikely to bear strong resemblance to a large number of data points given that cop sets an outlier apart from an inlier by comparing their coherence with the rest of the data points the mutual coherences are computed by forming the gram matrix of the normalized data points subsequently the sought subspace is recovered from the span of the subset of the data points that exhibit strong coherence with the rest of the data as cop only involves one simple matrix multiplication it is significantly faster than the state of the art robust pca algorithms we derive analytical performance guarantees for cop under different models for the distributions of inliers and outliers in both noise free and noisy settings cop is the first robust pca algorithm that is simultaneously non iterative provably robust to both unstructured and structured outliers and can tolerate a large number of unstructured outliers

efficient low rank approximation of matrices based on randomized pivoted decomposition given a matrix bf a with numerical rank k the two sided orthogonal decomposition tsod computes a factorization bf a bf udv t where bf u and bf v are orthogonal and bf d is upper lower triangular tsod is rank revealing as the middle factor bf d reveals the rank of bf a the computation of tsod however is demanding in this paper we present an algorithm called randomized pivoted tsod rp tsod where the middle factor is lower triangular key in our work is the exploitation of randomization and rp tsod is primarily devised to efficiently construct an approximation to a low rank matrix we provide three different types of bounds for rp tsod i we furnish upper bounds on the error of the low rank approximation ii we bound the k approximate principal singular values and iii we derive bounds for the canonical angles between the approximate and the exact singular subspaces our bounds describe the characteristics and behavior of our proposed algorithm through numerical tests we show the effectiveness of the devised bounds as well as our proposed algorithm

nonparametric maximum likelihood estimation for shifted curves the analysis of a sample of curves can be done by self modelling regression methods within this framework we follow the ideas of nonparametric maximum likelihood estimation known from event history analysis and the counting process set up we derive an infinite dimensional score equation and from there we suggest an algorithm to estimate the shape function for a simple shape invariant model the nonparametric maximum likelihood estimator that we find turns out to be a nadaraya watson like estimator but unlike in the usual kernel smoothing situation we do not need to select a bandwidth or even a kernel function since the score equation automatically selects the shape and the smoothing parameter for the estimation we apply the method to a sample of electrophoretic spectra to illustrate how it works

tight linear programming relaxations of uncapacitated p hub median problems abstract the problem of locating hub facilities and allocating non hub nodes to those hubs arises frequently in the design of communication networks airline passenger flow and parcel delivery networks in this paper we consider uncapacitated multiple and single allocation p hub median problems we develop new mixed 0 1 linear formulations with tight linear programming relaxations the approach is tested on a well known and heavily used benchmark data set of real world problems with resulting lp relaxations ranging from 10010 to 391 250 variables and from 2 101 to 31 901 constraints which proved to be difficult linear programs yet this approach proved to be very effective in almost all instances the linear programming solution was integer in cases with fractional solutions the integrality was achieved by adding a small partial set of integrality constraints therefore we extended the range of optimally solvable instances of these np hard hub location problems which have defied researchers for the last ten years as an additional result for the single allocation case we were able to establish optimality of all heuristic solutions obtained via tabu search algorithm from a previous study for the more difficult single allocation p hub median problem we also used the best known heuristic solution as a guidance in adding integrality constraints this novel linkage between optimal and heuristic solutions has a potential impact in a number of other problem settings where efficient heuristic solutions exist and are probably but not provably optimal

tensor decompositions and applications this survey provides an overview of higher order tensor decompositions their applications and available software a tensor is a multidimensional or n way array decompositions of higher order tensors i e n way arrays with n geq 3 have applications in psycho metrics chemometrics signal processing numerical linear algebra computer vision numerical analysis data mining neuroscience graph analysis and elsewhere two particular tensor decompositions can be considered to be higher order extensions of the matrix singular value decomposition candecomp parafac cp decomposes a tensor as a sum of rank one tensors and the tucker decomposition is a higher order form of principal component analysis there are many other tensor decompositions including indscal parafac2 candelinc dedicom and paratuck2 as well as nonnegative variants of all of the above the n way toolbox tensor toolbox and multilinear engine are examples of software packages for working with tensors

stochastic set packing problem abstract in this paper a stochastic version of the set packing problem spp is studied via scenario analysis we consider a one stage recourse approach to deal with the uncertainty in the coefficients it consists of maximizing in the stochastic spp a composite function of the expected value minus the weighted risk of obtaining a scenario whose objective function value is worse than a given threshold the splitting variable representation is decomposed by dualizing the nonanticipativity constraints that link the deterministic spp with a 0 1 knapsack problem for each scenario under consideration as a result a structured larger pure 0 1 model is created we present several procedures for obtaining good feasible solutions as well as a preprocessing approach for fixing variables the lagrange multipliers updating is performed by using the volume algorithm computational experience is reported for a broad variety of instances which shows that the new approach usually outperforms a state of the art optimization engine producing a comparable optimality gap with smaller several orders of magnitude computing time

latent variable graphical model selection via convex optimization suppose we observe samples of a subset of a collection of random variables no additional information is provided about the number of latent variables nor of the relationship between the latent and observed variables is it possible to discover the number of latent components and to learn a statistical model over the entire collection of variables we address this question in the setting in which the latent and observed variables are jointly gaussian with the conditional statistics of the observed variables conditioned on the latent variables being specified by a graphical model as a first step we give natural conditions under which such latent variable gaussian graphical models are identifiable given marginal statistics of only the observed variables essentially these conditions require that the conditional graphical model among the observed variables is sparse while the effect of the latent variables is spread out over most of the observed variables next we propose a tractable convex program based on regularized maximum likelihood for model selection in this latent variable setting the regularizer uses both the ell 1 norm and the nuclear norm our modeling framework can be viewed as a combination of dimensionality reduction to identify latent variables and graphical modeling to capture remaining statistical structure not attributable to the latent variables and it consistently estimates both the number of latent components and the conditional graphical model structure among the observed variables these results are applicable in the high dimensional setting in which the number of latent observed variables grows with the number of samples of the observed variables the geometric properties of the algebraic varieties of sparse matrices and of low rank matrices play an important role in our analysis

a quadratic integer program for the location of interacting hub facilities abstract this paper reports a new formulation of a general hub location model as a quadratic integer program non convexity of the objective function makes the problem difficult a variety of alternative solution strategies are discussed computational results from two simple heuristics are presented for the task of siting 2 3 or 4 hubs to serve interactions between sets of 10 15 20 and 25 u s cities the effects of different computational shortcuts are examined

local minima and convergence in low rank semidefinite programming the low rank semidefinite programming problem lrsdpr is a restriction of the semidefinite programming problem sdp in which a bound r is imposed on the rank of x and it is well known that lrsdpr is equivalent to sdp if r is not too small in this paper we classify the local minima of lrsdpr and prove the optimal convergence of a slight variant of the successful yet experimental algorithm of burer and monteiro 5 which handles lrsdpr via the nonconvex change of variables x rrt in addition for particular problem classes we describe a practical technique for obtaining lower bounds on the optimal solution value during the execution of the algorithm computational results are presented on a set of combinatorial optimization relaxations including some of the largest quadratic assignment sdps solved to date

searching for structure in curve samples abstract the shape of a regression curve can to a large extent be characterized by the succession of structural features like extrema inflection points and so on when analyzing a sample of regression curves it is often important to know at an early stage of data analysis which structural features are occurring consistently in each curve of the sample such a definition is usually not easy due to substantial interindividual variation both in the x and the y axis and due to the influence of noise a method is proposed for identifying typical features without relying on an a priori specified functional model for the curves the approach is based on the frequencies of occurrence of structural features as for example maxima in the curve sample along the x axis important tools are nonparametric regression and differentiation and kernel density estimation apart from a theoretical foundation the usefulness of the method is documented by application to two interesting biomedical areas growth and develop

petrels parallel subspace estimation and tracking by recursive least squares from partial observations many real world datasets exhibit an embedding of low dimensional structure in a high dimensional manifold examples include images videos and internet traffic data it is of great significance to estimate and track the low dimensional structure with small storage requirements and computational complexity when the data dimension is high therefore we consider the problem of reconstructing a data stream from a small subset of its entries where the data is assumed to lie in a low dimensional linear subspace possibly corrupted by noise we further consider tracking the change of the underlying subspace which can be applied to applications such as video denoising network monitoring and anomaly detection our setting can be viewed as a sequential low rank matrix completion problem in which the subspace is learned in an online fashion the proposed algorithm dubbed parallel estimation and tracking by recursive least squares petrels first identifies the underlying low dimensional subspace and then reconstructs the missing entries via least squares estimation if required subspace identification is performed via a recursive procedure for each row of the subspace matrix in parallel with discounting for previous observations numerical examples are provided for direction of arrival estimation and matrix completion comparing petrels with state of the art batch algorithms

weber problems with alternative transportation systems abstract in this paper we address a planar p facility location problem where together with a metric induced by a gauge there exists a series of rapid transit lines which can be used as alternative transportation system to reduce the total transportation cost the location problem is reduced to solving a finite number of multi weber problems from which localization results are obtained in particular it is shown that if the gauge in use is polyhedral then the problem is reduced to finding a p median

the effects of triangle inequality on the vehicle routing problem abstract routing problems often utilize experimental networks to represent real world scenarios however most ignore the inclusion of triangle inequality violations a phenomenon resulting from delays or rounding errors within a network this work evaluates the effect of both frequency the number of violations and severity the degree of intensity of a violation of triangle inequality and evaluates both solution quality and solution time based on simulated annealing ant colony optimization and savings algorithm methods findings indicate that while both frequency and severity degrade solution quality increased levels of frequency and severity together result in significant adverse affects to solution quality solution time however is not impacted by the presence of triangle inequality violations within the network this information should encourage practitioners to identify delays and maintain the presence of triangle inequality violations in a network to ensure accuracy of solution quality

fixed point continuation for ell_1 minimization methodology and convergence we present a framework for solving the large scale ell 1 regularized convex minimization problem min x 1 mu f x our approach is based on two powerful algorithmic ideas operator splitting and continuation operator splitting results in a fixed point algorithm for any given scalar mu continuation refers to approximately following the path traced by the optimal value of x as mu increases in this paper we study the structure of optimal solution sets prove finite convergence for important quantities and establish q linear convergence rates for the fixed point algorithm applied to problems with f x convex but not necessarily strictly convex the continuation framework motivated by our convergence results is demonstrated to facilitate the construction of practical algorithms

statistical tools to analyze data representing a sample of curves the paper is concerned with data representing a sample of smooth curves which can be considered as independent realizations of an underlying biological chemical ldots process such samples of curves often possess the following features there is a typical structural pattern common to all curves of the sample on the other hand individual realizations of the typical shape show different dynamics and intensity in particular typical peaks are shifted from individual to individual differences in dynamics complicate the analysis of samples of curves for example the cross sectional average usually does not reflect an average pattern due to shifts structure is smeared or might even disappear our approach consists in synchronizing the individual curves before determining the average or any further statistics pointwise averaging of the synchronized curves then leads to an average curve which represents the common structure with average dynamics and average intensity the method requires the introduction of new statistical objects they are defined mathematically their properties are discussed and possible estimators are proposed the asymptotic bias and variance of the estimators are derived an application to visually evoked brain potentials illustrates the approach

adaptive reduced rank processing based on joint and iterative interpolation decimation and filtering we present an adaptive reduced rank signal processing technique for performing dimensionality reduction in general adaptive filtering problems the proposed method is based on the concept of joint and iterative interpolation decimation and filtering we describe an iterative least squares ls procedure to jointly optimize the interpolation decimation and filtering tasks for reduced rank adaptive filtering in order to design the decimation unit we present the optimal decimation scheme and also propose low complexity decimation structures we then develop low complexity least mean squares lms and recursive least squares rls algorithms for the proposed scheme along with automatic rank and branch adaptation techniques an analysis of the convergence properties and issues of the proposed algorithms is carried out and the key features of the optimization problem such as the existence of multiple solutions are discussed we consider the application of the proposed algorithms to interference suppression in code division multiple access cdma systems simulations results show that the proposed algorithms outperform the best known reduced rank schemes with lower complexity

allocation strategies in hub networks in this paper we study allocation strategies and their effects on total routing costs in hub networks given a set of nodes with pairwise traffic demands the p hub median problem is the problem of choosing p nodes as hub locations and routing traffic through these hubs at minimum cost this problem has two versions in single allocation problems each node can send and receive traffic through a single hub whereas in multiple allocation problems there is no such restriction and a node may send and receive its traffic through all p hubs this results in high fixed costs and complicated networks in this study we introduce the r allocation p hub median problem where each node can be connected to at most r hubs this new problem generalizes the two versions of the p hub median problem we derive mixed integer programming formulations for this problem and perform a computational study using well known datasets for these datasets we conclude that single allocation solutions are considerably more expensive than multiple allocation solutions but significant savings can be achieved by allowing nodes to be allocated to two or three hubs rather than one we also present models for variations of this problem with service quality considerations flow thresholds and non stop service

adaptive algorithms to track the parafac decomposition of a third order tensor the parafac decomposition of a higher order tensor is a powerful multilinear algebra tool that becomes more and more popular in a number of disciplines existing parafac algorithms are computationally demanding and operate in batch mode both serious drawbacks for on line applications when the data are serially acquired or the underlying model changes with time adaptive parafac algorithms that can track the sought decomposition at low complexity would be highly desirable this is a challenging task that has not been addressed in the literature and the topic of this paper given an estimate of the parafac decomposition of a tensor at instant t we propose two adaptive algorithms to update the decomposition at instant t 1 the new tensor being obtained from the old one after appending a new slice in the time dimension the proposed algorithms can yield estimation performance that is very close to that obtained via repeated application of state of art batch algorithms at orders of magnitude lower complexity the effectiveness of the proposed algorithms is illustrated using a mimo radar application tracking of directions of arrival and directions of departure as an example

robust subspace tracking with missing entries the set theoretic approach in this paper an adaptive projected subgradient method apsm based algorithm for robust subspace tracking is introduced a properly chosen cost function is constructed at each time instance and the goal is to seek for points which belong to the zero level set of this function i e the set of points which score a zero loss at each iteration an outlier detection mechanism is employed in order to conclude whether the current data vector contains outlier noise or not in the sequel a sparsity promoting greedy algorithm is employed for the outlier vector estimation allowing the purification of the corrupted data from the outlier noise prior to any further processing furthermore the case where the observation vectors are partially observed is attacked via a prediction procedure which estimates the values of the unobserved missing coefficients a theoretical analysis is carried out and the simulation experiments within the contexts of robust subspace estimation and robust matrix completion demonstrate the enhanced performance of the proposed scheme compared to recently developed state of the art algorithms

low rank approximation and regression in input sparsity time we design a new distribution over m n matrices s so that for any fixed n d matrix a of rank r with probability at least 9 10 psaxp2 e 1 e paxp2 simultaneously for all x rd here m is bounded by a polynomial in re 1 and the parameter e 0 1 such a matrix s is called a subspace embedding furthermore sa can be computed in o nnz a time where nnz a is the number of nonzero entries of a this improves over all previous subspace embeddings for which computing sa required at least ndlog d time we call these s sparse embedding matrices using our sparse embedding matrices we obtain the fastest known algorithms for overconstrained least squares regression low rank approximation approximating all leverage scores and ep regression more specifically let b be an n 1 vector e 0 a small enough value and integers k p g 1 our results include the following regression the regression problem is to find d 1 vector x for which pax bpp l 1 e min xpax bpp for the euclidean case p e 2 we obtain an algorithm running in o nnz a o d3e 2 time and another in o nnz a log 1 e o d3 log 1 e time here o f e f log o 1 f for p 1 more generally we obtain an algorithm running in o nnz a log n o r e 1 c time for a fixed c low rank approximation we give an algorithm to obtain a rank k matrix k such that pa kpf 1 e p a akpf where ak is the best rank k approximation to a that is ak is the output of principal components analysis produced by a truncated singular value decomposition useful for latent semantic indexing and many other statistical problems our algorithm runs in o nnz a o nk2e 4 k3e 5 time leverage scores we give an algorithm to estimate the leverage scores of a up to a constant factor in o nnz a log n o r3 time

an improved benders decomposition algorithm for the tree of hubs location problem abstract the tree of hubs location problem is a particularly hard variant of the so called hub location problems when solving this problem by a benders decomposition approach it is necessary to deal with both optimality and feasibility cuts while modern implementations of the benders decomposition method rely on pareto optimal optimality cuts or on rendering feasibility cuts based on minimal infeasible subsystems a new cut selection scheme is devised here under the guiding principle of extracting useful information even when facing infeasible subproblems the proposed algorithm outperforms two other modern variants of the method and it is capable of optimally solving instances five times larger than the ones previously reported on the literature

integer programming formulations of discrete hub location problems abstract hubs are facilities that serve as transshipment and switching points for transportation and telecommunication systems with many origins and destinations this paper present integer programming formulations for four types of discrete hub location problems the p hub median problem the uncapacitated hub location problem p hub center problems and hub covering problems hub median and uncapacitated hub location problems have received limited attention from researchers this paper introduces discrete hub center and hub covering problems basic formulations and formulations with flow thresholds for spokes are presented

efficient algorithms for computing a strong rank revealing qr factorization given anm n matrixm withm n it is shown that there exists a permutation fi and an integer k such that the qr factorization myi q ak ckbk reveals the numerical rank of m the k k upper triangular matrix ak is well conditioned ilckll2 is small and bk is linearly dependent on ak with coefficients bounded by a low degree polynomial in n existing rank revealing qr rrqr algorithms are related to such factorizations and two algorithms are presented for computing them the new algorithms are nearly as efficient as qr with column pivoting for most problems and take o ran2 floating point operations in the worst case

learning with coefficient based regularized regression on markov resampling big data research has become a globally hot topic in recent years one of the core problems in big data learning is how to extract effective information from the huge data in this paper we propose a markov resampling algorithm to draw useful samples for handling coefficient based regularized regression cbrr problem the proposed markov resampling algorithm is a selective sampling method which can automatically select uniformly ergodic markov chain u e m c samples according to transition probabilities based on u e m c samples we analyze the theoretical performance of cbrr algorithm and generalize the existing results on independent and identically distributed observations to be specific when the kernel is infinitely differentiable the learning rate depending on the sample size m can be arbitrarily close to mathcal o m 1 under a mild regularity condition on the regression function the good generalization ability of the proposed method is validated by experiments on simulated and real data sets

a discrepancy measure for improved clustering a discrepancy measure is proposed to improve the clustering of patterns which experience non linear distortions the discrepancy measure is an outcome of a non linear alignment procedure which optimally aligns the elements of patterns in order to minimize the dissimilarity between the patterns the k means clustering algorithm is modified to use the discrepancy measure to compute the similarity between patterns and the cluster centers a series of clustering experiments were conducted on identical data using the modified and standard k means algorithm the results obtained show that clustering performance of the modified algorithm is significantly superior to that of the standard algorithm

steady state statistical performance analysis of subspace tracking methods subspace tracking using low complexity methods is a common research area in signal processing while numerical properties of available methods are extensively addressed in literature there is a paucity of statistical results on limiting behavior of subspace tracking methods in this contribution we will study steady state error of dpm malase champagne s pa karasalo and past methods we show that dpm malase and past are approximations of the pa method while karasalo s method is equivalent to the pa method in convergence region a tradeoff between steady state error of methods and their local convergence rate is demonstrated a nonstationary signal model is assumed in which true subspace is slowly and randomly varying simulation results show pa and past are asymptotically close to the optimum sample covariance matrix scm while this is not true for dpm and malase in stationary signal case in the steady state these methods can be expressed with a unified approach using stochastic first order difference equations whose fixed points are calculated the results can be used to determine optimal step size in terms of tolerable total signal subspace projection error simulation results confirm the validity of theoretical error calculations

the qlp approximation to the singular value decomposition in this paper we introduce a new decomposition called the pivoted qlp decomposition it is computed by applying pivoted orthogonal triangularization to the columns of the matrix x in question to get an upper triangular factor r and then applying the same procedure to the rows of r to get a lower triangular matrix l the diagonal elements of r are called the r values of x those of l are called the l values numerical examples show that the l values track the singular values of x with considerable fidelity far better than the r values at a gap in the l values the decomposition provides orthonormal bases of analogues of row column and null spaces provided of x the decomposition requires no more than twice the work required for a pivoted qr decomposition the computation of r and l can be interleaved so that the computation can be terminated at any suitable point which makes the decomposition especially suitable for low rank determination problems the interleaved algorithm also suggests a new efficient 2 norm estimator

benders decomposition for large scale uncapacitated hub location this paper describes an exact algorithm capable of solving large scale instances of the well known uncapacitated hub location problem with multiple assignments the algorithm applies benders decomposition to a strong path based formulation of the problem the standard decomposition algorithm is enhanced through the inclusion of several features such as the use of a multicut reformulation the generation of strong optimality cuts the integration of reduction tests and the execution of a heuristic procedure extensive computational experiments were performed to evaluate the efficiency and robustness of the algorithm computational results obtained on classical benchmark instances with up to 200 nodes and on a new and more difficult set of instances with up to 500 nodes confirm the efficiency of the algorithm

subspace iteration randomization and singular value problems a classical problem in matrix computations is the efficient and reliable approximation of a given matrix by a matrix of lower rank the truncated singular value decomposition svd is known to provide the best such approximation for any given fixed rank however the svd is also known to be very costly to compute among the different approaches in the literature for computing low rank approximations randomized algorithms have attracted researchers attention recently due to their surprising reliability and computational efficiency in different application areas typically such algorithms are shown to compute with very high probability low rank approximations that are within a constant factor from optimal and are known to perform even better in many practical situations in this paper we present a novel error analysis that considers randomized algorithms within the subspace iteration framework and show with very high probability that highly accurate low rank approximations as well as singular values ca

hankel matrix nuclear norm regularized tensor completion for n dimensional exponential signals signals are generally modeled as a superposition of exponential functions in spectroscopy of chemistry biology and medical imaging for fast data acquisition or other inevitable reasons however only a small amount of samples may be acquired and thus how to recover the full signal becomes an active research topic but existing approaches cannot efficiently recover n dimensional exponential signals with n geq 3 in this paper we study the problem of recovering n dimensional particularly n geq 3 exponential signals from partial observations and formulate this problem as a low rank tensor completion problem with exponential factor vectors the full signal is reconstructed by simultaneously exploiting the candecomp parafac tensor structure and the exponential structure of the associated factor vectors the latter is promoted by minimizing an objective function involving the nuclear norm of hankel matrices experimental results on simulated and real magnetic resonance spectroscopy data show that the proposed approach can successfully recover full signals from very limited samples and is robust to the estimated tensor rank

self modelling warping functions the paper introduces a semiparametric model for functional data the warping functions are assumed to be linear combinations ofqcommon components which are estimated from the data hence the name self modelling even small values ofqprovide remarkable model flexibility comparable with nonparametric methods at the same time this approach avoids overfitting because the common components are estimated combining data across individuals as a convenient by product component scores are often interpretable and can be used for statistical inference an example of classification based on scores is given abstract from author

single pass randomized algorithms for lu decomposition abstract in this paper we present some single pass randomized algorithms to compute lu decomposition these algorithms need only one pass over the original matrix and hence are very suitable for extremely large and high dimensional matrix stored outside of core memory or generated in a streaming fashion rigorous error bounds and complexity of these algorithms are provided numerical experiments show that these single pass algorithms have the similar accuracy and runtime excluding the cost of matrix transfer compared with the state of the art randomized algorithms for lu decomposition

stochastic subsampling for factorizing huge matrices we present a matrix factorization algorithm that scales to input matrices with both huge number of rows and columns learned factors may be sparse or dense and or nonnegative which makes our algorithm suitable for dictionary learning sparse component analysis and nonnegative matrix factorization our algorithm streams matrix columns while subsampling them to iteratively learn the matrix factors at each iteration the row dimension of a new sample is reduced by subsampling resulting in lower time complexity compared to a simple streaming algorithm our method comes with convergence guarantees to reach a stationary point of the matrix factorization problem we demonstrate its efficiency on massive functional magnetic resonance imaging data 2 tb and on patches extracted from hyperspectral images 103 gb for both problems which involve different penalties on rows and columns we obtain significant speed ups compared to state of the art algorithms

integral shape averaging and structural average estimation a comparative study classical signal averaging is an important operation in signal processing applications but is meaningless if shape or time variability is present among the signals being averaged structural average estimation sae and integral shape averaging isa are recent methods that deal with these types of variability by providing a structural average for sae and a mean shape signal for isa that can act as a complement to the classical average in signal analysis in this paper these two techniques are regarded from the perspective of the time warping problem we approach isa using a new formalism that emphasizes the theoretical connections between the two techniques a simulation study is then carried out to compare the two methods in three kinds of time fluctuation two sae algorithms are employed namely dynamic time warping dtw and self modeling registration smr the discussion of the results obtained in this comparison offers specific guidelines for the application dependent use of each approach

new formulation and a branch and cut algorithm for the multiple allocation p hub median problem abstract this article deals with the uncapacitated multiple allocation p hub median problem where p facilities hubs must be located among n available sites in order to minimize the transportation cost of sending a product between all pairs of sites each path between an origin and a destination can traverse any pair of hubs for the first time in the literature an integer programming formulation with o n2 variables has been devised to approach this problem based on this formulation a branch and cut algorithm has been developed which allows to solve larger instances than those previously solved in the literature the proposed algorithm performs specially well for relatively large values of p

new formulations for the uncapacitated multiple allocation hub location problem in this paper we review the integer linear formulations of the uncapacitated multiple allocation hub location problem we study the scope of validity of these formulations and give new ones that generalize the older formulations our formulations allow one or two visits to hubs and include a more general cost structure that needs not satisfy the triangle inequality we prove that the constraints defined by cliques of a related intersection graph are tighter constraints than the classical ones we also discuss a pre processing of the problem which is very useful for reducing its size finally we check the strength of the new formulations and compare them with others in the literature by solving instances of two commonly used data sets the cab civil aeronautics board and ap australian post and also randomly generated instances our computational results clearly show that our formulations outperform all others previously used for small and medium problems

fast monte carlo algorithms for matrices ii computing a low rank approximation to a matrix in many applications the data consist of or may be naturally formulated as an m times n matrix a it is often of interest to find a low rank approximation to a i e an approximation d to the matrix a of rank not greater than a specified rank k where k is much smaller than m and n methods such as the singular value decomposition svd may be used to find an approximation to a which is the best in a well defined sense these methods require memory and time which are superlinear in m and n for many applications in which the data sets are very large this is prohibitive two simple and intuitive algorithms are presented which when given an m times n matrix a compute a description of a low rank approximation d to a and which are qualitatively faster than the svd both algorithms have provable bounds for the error matrix a d for any matrix x let x f and x 2 denote its frobenius norm and its spectral norm respectively in the first algorithm c columns of a are randomly chosen if the m times c matrix c consists of those c columns of a after appropriate rescaling then it is shown that from c tc approximations to the top singular values and corresponding singular vectors may be computed from the computed singular vectors a description d of the matrix a may be computed such that mathrm rank d le k and such that left a d right xi 2 le min d mathrm rank d le k left a d right xi 2 poly k 1 c left a right 2 f holds with high probability for both xi 2 f this algorithm may be implemented without storing the matrix a in random access memory ram provided it can make two passes over the matrix stored in external memory and use o cm c 2 additional ram the second algorithm is similar except that it further approximates the matrix c by randomly sampling r rows of c to form a r times c matrix w thus it has additional error but it can be implemented in three passes over the matrix using only constant additional ram to achieve an additional error beyond the best rank k approximation that is at most epsilon a 2 f both algorithms take time which is polynomial in k 1 epsilon and log 1 delta where delta 0 is a failure probability the first takes time linear in mbox max m n and the second takes time independent of m and n our bounds improve previously published results with respect to the rank parameter k for both the frobenius and spectral norms in addition the proofs for the error bounds use a novel method that makes important use of matrix perturbation theory the probability distribution over columns of a and the rescaling are crucial features of the algorithms which must be chosen judiciously

exact and heuristic algorithms for the uncapacitated multiple allocation p hub median problem in this paper new milp formulations for the multiple allocation p hub median problem are presented these require fewer variables and constraints than those traditionally used in the literature an efficient heuristic algorithm based on shortest paths is described lp based solution methods as well as an explicit enumeration algorithm are developed to obtain exact solutions computational results are presented for well known problems from the literature which show that exact solutions can be found in a reasonable amount of computational time our algorithms are also benchmarked on a different data set this data set which includes problems that are larger than those used in the literature is based on a postal delivery network and has been treated by the authors in an earlier paper

robust subspace clustering subspace clustering refers to the task of nding a multi subspace representation that best ts a collection of points taken from a high dimensional space this paper introduces an algorithm inspired by sparse subspace clustering ssc 25 to cluster noisy data and develops some novel theory demonstrating its correctness in particular the theory uses ideas from geometric functional analysis to show that the algorithm can accurately recover the underlying subspaces under minimal requirements on their orientation and on the number of samples per subspace synthetic as well as real data experiments complement our theoretical study illustrating our approach and demonstrating its eectiveness

stochastic uncapacitated hub location we study stochastic uncapacitated hub location problems in which uncertainty is associated to demands and transportation costs we show that the stochastic problems with uncertain demands or dependent transportation costs are equivalent to their associated deterministic expected value problem evp in which random variables are replaced by their expectations in the case of uncertain independent transportation costs the corresponding stochastic problem is not equivalent to its evp and specific solution methods need to be developed we describe a monte carlo simulation based algorithm that integrates a sample average approximation scheme with a benders decomposition algorithm to solve problems having stochastic independent transportation costs numerical results on a set of instances with up to 50 nodes are reported

tensor decompositions for identifying directed graph topologies and tracking dynamic networks directed networks are pervasive both in nature and engineered systems often underlying the complex behavior observed in biological systems microblogs and social interactions over the web as well as global financial markets since their structures are often unobservable in order to facilitate network analytics one generally resorts to approaches capitalizing on measurable nodal processes to infer the unknown topology structural equation models sems are capable of incorporating exogenous inputs to resolve inherent directional ambiguities however conventional sems assume full knowledge of exogenous inputs which may not be readily available in some practical settings this paper advocates a novel sem based topology inference approach that entails factorization of a three way tensor constructed from the observed nodal data using the well known parallel factor parafac decomposition it turns out that second order piecewise stationary statistics of exogenous variables suffice to identify the hidden topology capitalizing on the uniqueness properties inherent to high order tensor factorizations it is shown that topology identification is possible under reasonably mild conditions in addition to facilitate real time operation and inference of time varying networks an adaptive parafac tensor decomposition scheme that tracks the topology revealing tensor factors is developed extensive tests on simulated and real stock quote data demonstrate the merits of the novel tensor based approach

efficient algorithms on robust low rank matrix completion against outliers this paper considers robust low rank matrix completion in the presence of outliers the objective is to recover a low rank data matrix from a small number of noisy observations we exploit the bilinear factorization formulation and develop a novel algorithm fully utilizing parallel computing resources our main contributions are i providing two smooth loss functions that promote robustness against two types of outliers namely dense outliers drawn from some elliptical distribution and sparse spike like outliers with small additive gaussian noise and ii an efficient algorithm with provable convergence to a stationary solution based on a parallel update scheme numerical results show that the proposed algorithm obtains a better solution with faster convergence speed than the benchmark algorithms in both synthetic and real data scenarios