a newton cg augmented lagrangian method for semidefinite programming we consider a newton cg augmented lagrangian method for solving semidefinite programming sdp problems from the perspective of approximate semismooth newton methods in order to analyze the rate of convergence of our proposed method we characterize the lipschitz continuity of the corresponding solution mapping at the origin for the inner problems we show that the positive definiteness of the generalized hessian of the objective function in these inner problems a key property for ensuring the efficiency of using an inexact semismooth newton cg method to solve the inner problems is equivalent to the constraint nondegeneracy of the corresponding dual problems numerical experiments on a variety of large scale sdp problems with the matrix dimension n up to 4 110 and the number of equality constraints m up to 2 156 544 show that the proposed method is very efficient we are also able to solve the sdp problem fap36 with n 4 110 and m 1 154 467 in the seventh dimacs implementation challenge much more accurately than in previous attempts

semismoothness of solutions to generalized equations and the moreau yosida regularization we show that a locally lipschitz homeomorphism function is semismooth at a given point if and only if its inverse function is semismooth at its image point we present a sufficient condition for the semismoothness of solutions to generalized equations over cone reducible nonpolyhedral convex sets we prove that the semismoothness of solutions to the moreau yosida regularization of a lower semicontinuous proper convex function is implied by the semismoothness of the metric projector over the epigraph of the convex function

benchmarking optimization software with performance profiles we propose performance profiles distribution functions for a performance metric as a tool for benchmarking and comparing optimization software we show that performance profiles combine the best features of other tools for performance evaluation

a statistical decision rule with incomplete knowledge about classes abstract a statistical decision rule among m classes is presented when the a priori knowledge about classes is not complete either the number of classes is not the true one or it is not possible to obtain samples from all the possible classes the reject option proposed by chow is extended by defining an ambiguity reject option and a distance reject option these two types of reject can be defined in a parametric as well as in a non parametric way an example is given in r in order to illustrate this rule this method has been developed essentially to solve diagnostic problems

relaxing nonconvex quadratic functions by multiple adaptive diagonal perturbations the current bottleneck of globally solving mixed integer nonconvex quadratically constrained programming problems miqcps is still to construct strong but computationally cheap convex relaxations especially when dense quadratic functions are present we propose a cutting surface method based on multiple diagonal perturbations to derive convex quadratic relaxations for nonconvex quadratic problems with separable constraints our relaxations can also be realized as outer approximations of a semi infinite convex formulation of a semidefinite relaxation proposed by buchheim and wiegele the corresponding separation problem is a highly structured semidefinite program sdp with a convex but nonsmooth objective function we propose solving this separation problem with a specialized barrier coordinate minimization algorithm numerical experiments on randomly generated instances show that our approach is very promising we also discuss how to apply our method to more general cases of miqcps

an ergodic theorem for filtering with applications to stability abstract ergodic properties of the signal filtering pair are studied for continuous time finite markov chains observed in white noise the obtained law of large numbers is applied to the stability problem of the nonlinear filter with respect to initial conditions the furstenberg khasminskii formula is derived for the top lyapunov exponent of the zakai equation and is used to estimate the stability index of the filter

computational experience with a bundle approach for semidefinite cutting plane relaxations of max cut and equipartition we propose a dynamic version of the bundle method to get approximate solutions to semidefinite programs with a nearly arbitrary number of linear inequalities our approach is based on lagrangian duality where the inequalities are dualized and only a basic set of constraints is maintained explicitly this leads to function evaluations requiring to solve a relatively simple semidefinite program our approach provides accurate solutions to semidefinite relaxations of the max cut and the equipartition problem which are not achievable by direct approaches based only on interior point methods

a nonlinear programming algorithm for solving semidefinite programs via low rank factorization in this paper we present a nonlinear programming algorithm for solving semidefinite programs sdps in standard form the algorithm s distinguishing feature is a change of variables that replaces the symmetric positive semidefinite variable x of the sdp with a rectangular variable r according to the factorization x rr t the rank of the factorization i e the number of columns of r is chosen minimally so as to enhance computational speed while maintaining equivalence with the sdp fundamental results concerning the convergence of the algorithm are derived and encouraging computational results on some large scale test problems are also presented

connections between semidefinite relaxations of the max cut and stable set problems we describe links between a recently introduced semidefinite relaxation for the max cut problem and the well known semidefinite relaxation for the stable set problem underlying the lovasz s theta function it turns out that the connection between the convex bodies defining the semidefinite relaxations mimics the connection existing between the corresponding polyhedra we also show how the semidefinite relaxations can be combined with the classical linear relaxations in order to obtain tighter relaxations

on the rank of extreme matrices in semidefinite programs and the multiplicity of optimal eigenvalues we derive some basic results on the geometry of semidefinite programming sdp and eigenvalue optimization i e the minimization of the sum of the k largest eigenvalues of a smooth matrix valued function we provide upper bounds on the rank of extreme matrices in sdps and the first theoretically solid explanation of a phenomenon of intrinsic interest in eigenvalue optimization in the spectrum of an optimal matrix the kth and k 1 st largest eigenvalues tend to be equal and frequently have multiplicity greater than two this clustering is intuitively plausible and has been observed as early as 1975 when the matrix valued function is affine we prove that clustering must occur at extreme points of the set of optimal solutions if the number of variables is sufficiently large we also give a lower bound on the multiplicity of the critical eigenvalue these results generalize to the case of a general matrix valued function under appropriate conditions

the spatiotemporal meg covariance matrix modeled as a sum of kronecker products the single kronecker product kp model for the spatiotemporal covariance of meg residuals is extended to a sum of kronecker products this sum of kp is estimated such that it approximates the spatiotemporal sample covariance best in matrix norm contrary to the single kp this extension allows for describing multiple independent phenomena in the ongoing background activity whereas the single kp model can be interpreted by assuming that background activity is generated by randomly distributed dipoles with certain spatial and temporal characteristics the sum model can be physiologically interpreted by assuming a composite of such processes taking enough terms into account the spatiotemporal sample covariance matrix can be described exactly by this extended model in the estimation of the sum of kp model it appears that the sum of the first 2 kp describes between 67 and 93 moreover these first two terms describe two physiological processes in the background activity focal frequency specific alpha activity and more widespread non frequency specific activity furthermore temporal nonstationarities due to trial to trial variations are not clearly visible in the first two terms and hence play only a minor role in the sample covariance matrix in terms of matrix power considering the dipole localization the single kp model appears to describe around 80 of the noise and seems therefore adequate the emphasis of further improvement of localization accuracy should be on improving the source model rather than the covariance model

an unconstrained minimization method for solving low rank sdp relaxations of the maxcut problem in this paper we consider low rank semidefinite programming lrsdp relaxations of combinatorial quadratic problems that are equivalent to the maxcut problem using the gramian representation of a positive semidefinite matrix the lrsdp problem can be formulated as the nonconvex nonlinear programming problem of minimizing a quadratic function with quadratic equality constraints for the solution of this problem we propose a continuously differentiable exact merit function that exploits the special structure of the constraints and we use this function to define an efficient and globally convergent algorithm finally we test our code on an extended set of instances of the maxcut problem and we report comparisons with other existing codes

detecting spatio temporal modes in multivariate data by entropy field decomposition a new data analysis method that addresses a general problem of detecting spatio temporal variations in multivariate data is presented the method utilizes two recent and complimentary general approaches to data analysis information field theory ift and entropy spectrum pathways esp both methods reformulate and incorporate bayesian theory thus use prior information to uncover underlying structure of the unknown signal unification of esp and ift creates an approach that is non gaussian and non linear by construction and is found to produce unique spatio temporal modes of signal behavior that can be ranked according to their significance from which space time trajectories of parameter variations can be constructed and quantified two brief examples of real world applications of the theory to the analysis of data bearing completely different unrelated nature lacking any underlying similarity are also presented the first example provides an analysis of resting state functional magnetic resonance imaging rsfmri data that allowed us to create an efficient and accurate computational method for assessing and categorizing brain activity the second example demonstrates the potential of the method in the application to the analysis of a strong atmospheric storm circulation system during the complicated stage of tornado development and formation using data recorded by a mobile doppler radar reference implementation of the method will be made available as a part of the quest toolkit that is currently under development at the center for scientific computation in imaging

solving max cut to optimality by intersecting semidefinite and polyhedral relaxations we present a method for finding exact solutions of max cut the problem of finding a cut of maximum weight in a weighted graph we use a branch and bound setting that applies a dynamic version of the bundle method as bounding procedure this approach uses lagrangian duality to obtain a nearly optimal solution of the basic semidefinite max cut relaxation strengthened by triangle inequalities the expensive part of our bounding procedure is solving the basic semidefinite relaxation of the max cut problem which has to be done several times during the bounding process we review other solution approaches and compare the numerical results with our method we also extend our experiments to instances of unconstrained quadratic 0 1 optimization and to instances of the graph equipartition problem the experiments show that our method nearly always outperforms all other approaches in particular for dense graphs where linear programming based methods fail our method performs very well exact solutions are obtained in a reasonable time for any instance of size up to n 100 independent of the density for some problems of special structure we can solve even larger problem classes we could prove optimality for several problems of the literature where to the best of our knowledge no other method is able to do so

information pathways in a disordered lattice the maximum entropy random walk in a disordered lattice is obtained as a consequence of the principle of maximum entropy for a particular type of prior information without restriction on the number of steps this novel result demonstrates that transition probabilities defining the random walk represent a general characterization of information on a defective lattice and does not necessarily reflect a physical process the localization phenomenon is shown to be a consequence of solution of the laplacian on the lattice hence it contradicts the previous interpretation as a spherical lifshitz state and naturally generalizes to multiple modes whose order reflects the significance of information the dynamics of information flow on the microscale is related to the macroscopic structure of the lattice through a fokker planck formalism this newly derived theoretical framework is opening doors for a wide range of applications in analysis of information flow in disordered systems that includes potentially breakthrough resolution of the outstanding problem of inferring connectivity from discrete imaging i e neural data

approximation and limit results for nonlinear filters over an infinite time interval this paper is concerned with approximations to nonlinear filtering problems that are of interest over a very long time interval since the optimal filter can rarely be constructed one needs to compute with numerically feasible approximations the signal model can be a jump diffusion or just a process that is approximated by a jump diffusion the observation noise can be either white or of wide bandwidth the observations can be taken in either discrete or continuous time the cost of interest is the pathwise error per unit time over a long time interval it is shown under quite reasonable conditions on the approximating filter and the signal and noise processes that as time bandwidth process and filter approximation etc go to their limit in any way at all the limit of the pathwise average costs per unit time is just what one would get if the approximating processes were replaced by their ideal values and the optimal filter were used analogous results are obtained with appropriate scaling if the observations are taken in discrete time and the sampling interval also goes to zero for these cases the approximating filter is a numerical approximation to the optimal filter for the presumed limit signal observation noise problem

complementarity and nondegeneracy in semidefinite programming primal and dual nondegeneracy conditions are defined for semidefinite programming given the existence of primal and dual solutions it is shown that primal nondegeneracy implies a unique dual solution and that dual nondegeneracy implies a unique primal solution the converses hold if strict complementarity is assumed primal and dual nondegeneracy assumptions do not imply strict complementarity as they do in lp the primal and dual nondegeneracy assumptions imply a range of possible ranks for primal and dual solutions x and z this is in contrast with lp where nondegeneracy assumptions exactly determine the number of variables which are zero it is shown that primal and dual nondegeneracy and strict complementarity all hold generically numerical experiments suggest probability distributions for the ranks of x and z which are consistent with the nondegeneracy conditions

a spectral bundle method for semidefinite programming a central drawback of primal dual interior point methods for semidefinite programs is their lack of ability to exploit problem structure in cost and coefficient matrices this restricts applicability to problems of small dimension typically semidefinite relaxations arising in combinatorial applications have sparse and well structured cost and coefficient matrices of huge order we present a method that allows us to compute acceptable approximations to the optimal solution of large problems within reasonable time semidefinite programming problems with constant trace on the primal feasible set are equivalent to eigenvalue optimization problems these are convex nonsmooth programming problems and can be solved by bundle methods we propose replacing the traditional polyhedral cutting plane model constructed from subgradient information by a semidefinite model that is tailored to eigenvalue problems convergence follows from the traditional approach but a proof is included for completeness we present numerical examples demonstrating the efficiency of the approach on combinatorial examples

what are the differences between bayesian classifiers and mutual information classifiers in this paper both bayesian and mutual information classifiers are examined for binary classifications with or without a reject option the general decision rules are derived for bayesian classifiers with distinctions on error types and reject types a formal analysis is conducted to reveal the parameter redundancy of cost terms when abstaining classifications are enforced the redundancy implies an intrinsic problem of nonconsistency for interpreting cost terms if no data are given to the cost terms we demonstrate the weakness of bayesian classifiers in class imbalanced classifications on the contrary mutual information classifiers are able to provide an objective solution from the given data which shows a reasonable balance among error types and reject types numerical examples of using two types of classifiers are given for confirming the differences including the extremely class imbalanced cases finally we briefly summarize the bayesian and mutual information classifiers in terms of their application advantages and disadvantages respectively

quadratic programming feature selection identifying a subset of features that preserves classification accuracy is a problem of growing importance because of the increasing size and dimensionality of real world data sets we propose a new feature selection method named quadratic programming feature selection qpfs that reduces the task to a quadratic optimization problem in order to limit the computational complexity of solving the optimization problem qpfs uses the nystrom method for approximate matrix diagonalization qpfs is thus capable of dealing with very large data sets for which the use of other methods is computationally expensive in experiments with small and medium data sets the qpfs method leads to classification accuracy similar to that of other successful techniques for large data sets qpfs is superior in terms of computational efficiency

numerical evaluation of sbmethod we report numerical results for sbmethod a publically available implementation of the spectral bundle method applied to the 7 th dimacs challenge test sets that are semidefinite relaxations of combinatorial optimization problems the performance of the code is heavily influenced by parameters that control bundle update and eigenvalue computation unfortunately no mathematically sound guidelines for setting them are known based on our experience with sbmethod we propose heuristics for dynamically updating the parameters as well as a heuristc for improving the starting point these are now the default settings of sbmethod version 1 1 we compare their performance on the dimacs instances to our previous best choices for version 1 0 sbmethod version 1 1 is also part of the independent dimacs benchmark by h mittelmann based on these results we try to analyze strengths and weaknesses of our approach in comparison to other codes for large scale semidefinite programming

a generalized constraint neural network model associating partially known relationships for nonlinear regressions in an attempt to enhance the neural network technique so that it can evolve from a black box tool into a semi analytical one we propose a novel modeling approach of imposing generalized constraints on a standard neural network we redefine approximation problems by use of a new formalization with the aim of embedding prior knowledge explicitly into the model to the maximum extent a generalized constraint neural network gcnn model has therefore been developed which basically consists of two submodels one is constructed by the standard neural network technique to approximate the unknown part of the target function the other is formed from partially known relationships to impose generalized constraints on the whole model three issues arising after combination of the two submodels are discussed a the better approximation provided by the gcnn model compared with a standard neural network b the identifiability of parameters in the partially known relationships and c the discrepancy in the approximation due to removable singularities in the target function numerical studies of three benchmark problems show important findings that have not previously been reported in the literature significant benefits were observed from using the gcnn model in comparison with a standard neural network

on filtering of markov chains in strong noise the filtering problem for finite state markov chains is revisited in the low signal to noise regime we give a description of conditional measure concentration around the invariant distribution of the signal and derive asymptotic expressions for the performance indices of the minimum mean square error mmse and minimum a posteriori probability map filtering estimates

probabilistic independent component analysis for functional magnetic resonance imaging we present an integrated approach to probabilistic independent component analysis ica for functional mri fmri data that allows for nonsquare mixing in the presence of gaussian noise in order to avoid overfitting we employ objective estimation of the amount of gaussian noise through bayesian analysis of the true dimensionality of the data i e the number of activation and non gaussian noise sources this enables us to carry out probabilistic modeling and achieves an asymptotically unique decomposition of the data it reduces problems of interpretation as each final independent component is now much more likely to be due to only one physical or physiological process we also describe other improvements to standard ica such as temporal prewhitening and variance normalization of timeseries the latter being particularly useful in the context of dimensionality reduction when weak activation is present we discuss the use of prior information about the spatiotemporal nature of the source processes and an alternative hypothesis testing approach for inference using gaussian mixture models the performance of our approach is illustrated and evaluated on real and artificial fmri data and compared to the spatio temporal accuracy of results obtained from classical ica and glm analyses

discussion probability theory and fuzzy logic are complementary rather than competitive the relationship between probability theory and fuzzy logic has long been an object of discussion and some controversy the position articulated in this article is that probability theory by itself is not sufficient for dealing with uncertainty and imprecision in real world settings to enhance its effectiveness probability theory needs an infusion of concepts and techniques drawn from fuzzy logic especially the concept of a linguistic variable and the calculus of fuzzy if then rules in the final analysis probability theory and fuzzy logic are complementary rather than competitive

mixtures of probabilistic principal component analyzers principal component analysis pca is one of the most popular techniques for processing compressing and visualizing data although its effectiveness is limited by its global linearity while nonlinear variants of pca have been proposed an alternative paradigm is to capture data complexity by a combination of local linear pca projections however conventional pca does not correspond to a probability density and so there is no unique way to combine pca models therefore previous attempts to formulate mixture models for pca have been ad hoc to some extent in this article pca is formulated within a maximum likelihood framework based on a specific form of gaussian latent variable model this leads to a well defined mixture model for probabilistic principal component analyzers whose parameters can be determined using an expectationmaximization algorithm we discuss the advantages of this model in the context of clustering density modeling and local dimensionality reduction and we demonstrate its application to image compression and handwritten digit recognition

the optimum class selective rejection rule class selective rejection is an extension of simple rejection that is when an input pattern cannot be reliably assigned to one of the n classes in an n class problem it is assigned to a subset of classes that are most likely to issue the pattern instead of simply being rejected by selecting more classes the risk of making an error can be reduced at the price of subsequently having a larger remaining number of classes the optimality of class selective rejection is therefore defined as the best trade off between error rate and average number of selected classes formally the trade off study is embedded in the framework of decision theory the average expected loss is expressed as a linear combination of error rate and average number of classes the minimization of the average expected loss therefore provides the best trade off the complexity of the resulting optimum rule is reduced via a discrete convex minimization to be linear in the number of classes upper bounds on error rate and average number of classes are derived an example is provided to illustrate various aspects of the optimum decision rule finally the implications of the new decision rule are discussed

extracting qualitative dynamics from experimental data abstract we consider the notion of qualitative information and the practicalities of extracting it from experimental data our approach based on a theorem of takens draws on ideas from the generalized theory of information known as singular system analysis due to bertero pike and co workers we illustrate our technique with numerical data from the chaotic regime of the lorenz model

on the feature selection criterion based on an approximation of multidimensional mutual information we derive the feature selection criterion presented in 1 and 2 from the multidimensional mutual information between features and the class our derivation 1 specifies and validates the lower order dependency assumptions of the criterion and 2 mathematically justifies the utility of the criterion by relating it to bayes classification error

solving large scale sparse semidefinite programs for combinatorial optimization we present a dual scaling interior point algorithm and show how it exploits the structure and sparsity of some large scale problems we solve the positive semidefinite relaxation of combinatorial and quadratic optimization problems subject to boolean constraints we report the first computational results of interior point algorithms for approximating maximum cut semidefinite programs with dimension up to 3 000

discovering diverse subset for unsupervised hyperspectral band selection band selection as a special case of the feature selection problem tries to remove redundant bands and select a few important bands to represent the whole image cube this has attracted much attention since the selected bands provide discriminative information for further applications and reduce the computational burden though hyperspectral band selection has gained rapid development in recent years it is still a challenging task because of the following requirements 1 an effective model can capture the underlying relations between different high dimensional spectral bands 2 a fast and robust measure function can adapt to general hyperspectral tasks and 3 an efficient search strategy can find the desired selected bands in reasonable computational time to satisfy these requirements a multigraph determinantal point process mdpp model is proposed to capture the full structure between different bands and efficiently find the optimal band subset in extensive hyperspectral applications there are three main contributions 1 graphical model is naturally transferred to address band selection problem by the proposed mdpp 2 multiple graphs are designed to capture the intrinsic relationships between hyperspectral bands and 3 mixture dpp is proposed to model the multiple dependencies in the proposed multiple graphs and offers an efficient search strategy to select the optimal bands to verify the superiority of the proposed method experiments have been conducted on three hyperspectral applications such as hyperspectral classification anomaly detection and target detection the reliability of the proposed method in generic hyperspectral tasks is experimentally proved on four real world hyperspectral data sets

local estimation of posterior class probabilities to minimize classification errors decision theory shows that the optimal decision is a function of the posterior class probabilities more specifically in binary classification the optimal decision is based on the comparison of the posterior probabilities with some threshold therefore the most accurate estimates of the posterior probabilities are required near these decision thresholds this paper discusses the design of objective functions that provide more accurate estimates of the probability values taking into account the characteristics of each decision problem we propose learning algorithms based on the stochastic gradient minimization of these loss functions we show that the performance of the classifier is improved when these algorithms behave like sample selectors samples near the decision boundary are the most relevant during learning

parameter identifiability in statistical machine learning a review this review examines the relevance of parameter identifiability for statistical models used in machine learning in addition to defining main concepts we address several issues of identifiability closely related to machine learning showing the advantages and disadvantages of state of the art research and demonstrating recent progress first we review criteria for determining the parameter structure of models from the literature this has three related issues parameter identifiability parameter redundancy and reparameterization second we review the deep influence of identifiability on various aspects of machine learning from theoretical and application viewpoints in addition to illustrating the utility and influence of identifiability we emphasize the interplay among identifiability theory machine learning mathematical statistics information theory optimization theory information geometry riemann geometry symbolic computation bayesian inference algebraic geometry and others finally we present a new perspective together with the associated challenges

conditional likelihood maximisation a unifying framework for information theoretic feature selection we present a unifying framework for information theoretic feature selection bringing almost two decades of research on heuristic filter criteria under a single theoretical interpretation this is in response to the question what are the implicit statistical assumptions of feature selection criteria based on mutual information to answer this we adopt a different strategy than is usual in the feature selection literature instead of trying to define a criterion we derive one directly from a clearly specified objective function the conditional likelihood of the training labels while many hand designed heuristic criteria try to optimize a definition of feature relevancy and redundancy our approach leads to a probabilistic framework which naturally incorporates these concepts as a result we can unify the numerous criteria published over the last two decades and show them to be low order approximations to the exact but intractable optimisation problem the primary contribution is to show that common heuristics for information based feature selection including markov blanket algorithms as a special case are approximate iterative maximisers of the conditional likelihood a large empirical study provides strong evidence to favour certain classes of criteria in particular those that balance the relative size of the relevancy redundancy terms overall we conclude that the jmi criterion yang and moody 1999 meyer et al 2008 provides the best tradeoff in terms of accuracy stability and flexibility with small data samples

optimal decision rule with class selective rejection and performance constraints the problem of defining a decision rule which takes into account performance constraints and class selective rejection is formalized in a general framework in the proposed formulation the problem is defined using three kinds of criteria the first is the cost to be minimized which defines the objective function the second are the decision options determined by the admissible assignment classes or subsets of classes and the third are the performance constraints the optimal decision rule within the statistical decision theory framework is obtained by solving the stated optimization problem two examples are provided to illustrate the formulation and the decision rule is obtained

speedp an algorithm to compute sdp bounds for very large max cut instances we consider low rank semidefinite programming lrsdp relaxations of unconstrained 1 1 quadratic problems or equivalently of max cut problems that can be formulated as the non convex nonlinear programming problem of minimizing a quadratic function subject to separable quadratic equality constraints we prove the equivalence of the lrsdp problem with the unconstrained minimization of a new merit function and we define an efficient and globally convergent algorithm called speedp for finding critical points of the lrsdp problem we provide evidence of the effectiveness of speedp by comparing it with other existing codes on an extended set of instances of the max cut problem we further include speedp within a simply modified version of the goemans williamson algorithm and we show that the corresponding heuristic speedp mc can generate high quality cuts for very large sparse graphs of size up to a million nodes in a few hours

low rank optimization on the cone of positive semidefinite matrices we propose an algorithm for solving optimization problems defined on a subset of the cone of symmetric positive semidefinite matrices this algorithm relies on the factorization x yy t where the number of columns of y fixes an upper bound on the rank of the positive semidefinite matrix x it is thus very effective for solving problems that have a low rank solution the factorization x yy t leads to a reformulation of the original problem as an optimization on a particular quotient manifold the present paper discusses the geometry of that manifold and derives a second order optimization method with guaranteed quadratic convergence it furthermore provides some conditions on the rank of the factorization to ensure equivalence with the original problem in contrast to existing methods the proposed algorithm converges monotonically to the sought solution its numerical efficiency is evaluated on two applications the maximal cut of a graph and the problem of sparse principal component analysis

a semidefinite programming based search strategy for feature selection with mutual information measure feature subset selection as a special case of the general subset selection problem has been the topic of a considerable number of studies due to the growing importance of data mining applications in the feature subset selection problem there are two main issues that need to be addressed i finding an appropriate measure function than can be fairly fast and robustly computed for high dimensional data ii a search strategy to optimize the measure over the subset space in a reasonable amount of time in this article mutual information between features and class labels is considered to be the measure function two series expansions for mutual information are proposed and it is shown that most heuristic criteria suggested in the literature are truncated approximations of these expansions it is well known that searching the whole subset space is an np hard problem here instead of the conventional sequential search algorithms we suggest a parallel search strategy based on semidefinite programming sdp that can search through the subset space in polynomial time by exploiting the similarities between the proposed algorithm and an instance of the maximum cut problem in graph theory the approximation ratio of this algorithm is derived and is compared with the approximation ratio of the backward elimination method the experiments show that it can be misleading to judge the quality of a measure solely based on the classification accuracy without taking the effect of the non optimum search strategy into account

asymptotic stability of the wonham filter ergodic and nonergodic signals the stability problem of the wonham filter with respect to initial conditions is addressed the case of ergodic signals is revisited in view of a gap in the classic work of h kunita 1971 we give new bounds for the exponential stability rates which do not depend on the observations in the nonergodic case the stability is implied by identifiability conditions formulated explicitly in terms of the transition intensities matrix and the observation structure

extremal correlation matrices abstract let r n denote the convex compact set of all real n by n positive semidefinite matrices with main diagonal entries equal to 1 we examine the extreme points of r n focusing mainly on their rank the principal result is that r n contains extreme points of rank k if and only if k k 1 2 n

laplacian eigenvalues and the maximum cut problem we introduce and study an eigenvalue upper bound g on the maximum cut mc g of a weighted graph the function g has several interesting properties that resemble the behaviour of mc g the following results are presented we show that is subadditive with respect to amalgam and additive with respect to disjoint sum and 1 sum we prove that g is never worse that 1 131 mc g for a planar or more generally a weakly bipartite graph with nonnegative edge weights we give a dual characterization of g and show that g is computable in polynomial time with an arbitrary precision

semi definite matrix constraints in optimization positive semi definite matrix constraints arise in a number of optimization problems in which some or all of the elements of a matrix are variables such as the educational testing and matrix modification problems the structure of such constraints is developed including expressions for the normal cone feasible directions and their application to optimality conditions a computational framework is given within which these concepts can be exploited and which permits the quantification of second order effects the matrix of lagrange multipliers in this formulation is shown to have an important relationship to the characterization of the normal cone modifications of various iterative schemes in nonlinear programming are considered in order to develop an effective algorithm for the educational testing problem and comparative numerical experiments are described it is shown that a particular choice of the penalty parameter for an l 1 exact penalty function is appropriate for this type of problem the be

a nonsmooth version of newton s method newton s method for solving a nonlinear equation of several variables is extended to a nonsmooth case by using the generalized jacobian instead of the derivative this extension includes the b derivative version of newton s method as a special case convergence theorems are proved under the condition of semismoothness it is shown that the gradient function of the augmented lagrangian forc2 nonlinear programming is semismooth thus the extended newton s method can be used in the augmented lagrangian method for solving nonlinear programs

a group model for stable multi subject ica on fmri datasets spatial independent component analysis ica is an increasingly used data driven method to analyze functional magnetic resonance imaging fmri data to date it has been used to extract sets of mutually correlated brain regions without prior information on the time course of these regions some of these sets of regions interpreted as functional networks have recently been used to provide markers of brain diseases and open the road to paradigm free population comparisons such group studies raise the question of modeling subject variability within ica how can the patterns representative of a group be modeled and estimated via ica for reliable inter group comparisons in this paper we propose a hierarchical model for patterns in multi subject fmri datasets akin to mixed effect group models used in linear model based analysis we introduce an estimation procedure canica canonical ica based on i probabilistic dimension reduction of the individual data ii canonical correlation analysis to identify a data subspace common to the group iii ica based pattern extraction in addition we introduce a procedure based on cross validation to quantify the stability of ica patterns at the level of the group we compare our method with state of the art multi subject fmri ica methods and show that the features extracted using our procedure are more reproducible at the group level on two datasets of 12 healthy controls a resting state and a functional localizer study

solving a class of semidefinite programs via nonlinear programming in this paper we introduce a transformation that converts a class of linear and nonlinear semidefinite programming sdp problems into nonlinear optimization problems for those problems of interest the transformation replaces matrix valued constraints by vector valued ones hence reducing the number of constraints by an order of magnitude the class of transformable problems includes instances of sdp relaxations of combinatorial optimization problems with binary variables as well as other important sdp problems we also derive gradient formulas for the objective function of the resulting nonlinear optimization problem and show that both function and gradient evaluations have affordable complexities that effectively exploit the sparsity of the problem data this transformation together with the efficient gradient formulas enables the solution of very large scale sdp problems by gradient based nonlinear optimization techniques in particular we propose a first order log barrier method designed for solving a class of large scale linear sdp problems this algorithm operates entirely within the space of the transformed problem while still maintaining close ties with both the primal and the dual of the original sdp problem global convergence of the algorithm is established under mild and reasonable assumptions

an information maximization approach to blind separation and blind deconvolution we derive a new self organizing learning algorithm that maximizes the information transferred in a network of nonlinear units the algorithm does not assume any knowledge of the input distributions and is defined here for the zero noise limit under these conditions information maximization has extra properties not found in the linear case linsker 1989 the nonlinearities in the transfer function are able to pick up higher order moments of the input distributions and perform something akin to true redundancy reduction between units in the output representation this enables the network to separate statistically independent components in the inputs a higher order generalization of principal components analysis we apply the network to the source separation or cocktail party problem successfully separating unknown mixtures of up to 10 speakers we also show that a variant on the network architecture is able to perform blind deconvolution cancellation of unknown echoes and reverberation in a speech signal finally we derive dependencies of information transfer on time delays we suggest that information maximization provides a unifying framework for problems in blind signal processing

lyapunov exponents for finite state nonlinear filtering consider the wonham optimal filtering problem for a finite state ergodic markov process in both discrete and continuous time and let sigma be the noise intensity for the observation we examine the sensitivity of the solution with respect to the filter s initial conditions in terms of the gap between the first two lyapunov exponents of the zakai equation for the unnormalized conditional probability this gap is studied in the limit as sigma to 0 by techniques involving considerations of nonlinear filtering and the stochastic feynman kac formula conditions are given for the limit to be either negative or infty asymptotic bounds are derived in the latter case

using svm to combine global heuristics for the standard quadratic problem abstract the standard quadratic problem stqp is an np hard problem with many local minimizers stationary points in the literature heuristics based on unconstrained continuous non convex formulations have been proposed bomze palagi 2005 bomze grippo palagi 2012 but none dominates the other in terms of best value found following cassioli dilorenzo locatelli schoen sciandrone 2012 we propose to use support vector machines svms to define a multistart global strategy which selects the best heuristic we test our method on stqp arising from the maximum clique problem on a graph which is a challenging combinatorial problem we use as benchmark the clique problems in the dimacs challenge