constructing local generic formal fibers abstract let t m be a complete local noetherian unique factorization domain of dimension at least two such that the cardinality of the residue field t m is at least the cardinality of the real numbers suppose 0 p 0 p 1 p n m is a chain of distinct prime ideals of t such that p n intersected with the prime subring of t is the zero ideal we construct a chain of local unique factorization domains a n a n 1 a 1 a 0 t such that the completion of each a i is t and for every i the generic formal fiber of a i is local this means that the ring t a i k i is a local ring where k i is the quotient field of a i with p i a i k i its maximal ideal

large scale multiple kernel learning while classical kernel based learning algorithms are based on a single kernel in practice it is often desirable to use multiple kernels lanckriet et al 2004 considered conic combinations of kernel matrices for classification leading to a convex quadratically constrained quadratic program we show that it can be rewritten as a semi infinite linear program that can be efficiently solved by recycling the standard svm implementations moreover we generalize the formulation and our method to a larger class of problems including regression and one class classification experimental results show that the proposed algorithm works for hundred thousands of examples or hundreds of kernels to be combined and helps for automatic model selection improving the interpretability of the learning result in a second part we discuss general speed up mechanism for svms especially when used with sparse feature maps as appear for string kernels allowing us to train a string kernel svm on a 10 million real world splice data set from computational biology we integrated multiple kernel learning in our machine learning toolbox shogun for which the source code is publicly available at http www fml tuebingen mpg de raetsch projects shogun

the efficiency of ballstep subgradient level methods for convex optimization we study subgradient methods for convex optimization that use projections onto successive approximations of level sets of the objective corresponding to estimates of the optimal value we establish convergence and efficiency estimates for simple ballstep level controls without requiring that the feasible set be compact our framework may handle accelerations based on cheap projections surrogate constraints and conjugate subgradient techniques

noetherian rings between a semilocal domain and its completion we consider a general technique for constructing local noetherian integral domains letrbe a semilocal noetherian domain with jacobson radicalmand field of fractionsk letybe a nonzero element ofmand letr be the y adic completion ofr for elements 1 s yr algebraically independent overk we obtain a necessary and sufficient condition fora k 1 s r to be simultaneously noetherian and a directed union of localized polynomial rings insvariables overr we specify conditions in order that excellence be preserved and we use the construction to obtain a non noetherian ringaof the formk r which is a directed union of localized polynomial rings overr

smooth optimization with approximate gradient we show that the optimal complexity of nesterov s smooth first order optimization algorithm is preserved when the gradient is computed only up to a small uniformly bounded error in applications of this method to semidefinite programs this means in some instances computing only a few leading eigenvalues of the current iterate instead of a full matrix exponential which significantly reduces the method s computational cost this also allows sparse problems to be solved efficiently using sparse maximum eigenvalue packages

on the convergence rate of l p norm multiple kernel learning we derive an upper bound on the local rademacher complexity of lp norm multiple kernel learning which yields a tighter excess risk bound than global approaches previous local approaches analyzed the case p 1 only while our analysis covers all cases 1 p assuming the different feature mappings corresponding to the different kernels to be uncorrelated we also show a lower bound that shows that the bound is tight and derive consequences regarding excess loss namely fast convergence rates of the order o n 1 a where is the minimum eigenvalue decay rate of the individual kernels

consistency property of extended least squares parameter estimation for stochastic diffusion equation the extended least squares parameter estimate for stochastic heat diffusion equations is considered the unknown parameter is a heat diffusion coefficient which is a function of a spatial variable almost sure convergence for the estimated parameter is proved a numerical example is demonstrated for supporting the theoretical results developed here

new perspectives on k support and cluster norms we study a regularizer which is defined as a parameterized in mum of quadratics and which we call the box norm we show that the k support norm a regularizer proposed by argyriou et al 2012 for sparse vector prediction problems belongs to this family and the box norm can be generated as a perturbation of the former we derive an improved algorithm to compute the proximity operator of the squared box norm and we provide a method to compute the norm we extend the norms to matrices introducing the spectral k support norm and spectral box norm we note that the spectral box norm is essentially equivalent to the cluster norm a multitask learning regularizer introduced by jacob et al 2009a and which in turn can be interpreted as a perturbation of the spectral k support norm centering the norm is important for multitask learning and we also provide a method to use centered versions of the norms as regularizers numerical experiments indicate that the spectral k support and box norms and their centered variants provide state of the art performance in matrix completion and multitask learning problems respectively

completions of local rings with an isolated singularity abstract it is shown that all equicharacteristic complete local rings can be realized as completions of local rings whose proper localizations are all regular the only restriction in the mixed characteristic case is that the prime integer cannot be a nonzero zerodivisor in any proper localization of the complete ring the constructed examples are unique factorization domains whenever possible and in general are as close to being ufds as the hypothesis will allow

a new class of incremental gradient methods for least squares problems the least mean squares lms method for linear least squares problems differs from the steepest descent method in that it processes data blocks one by one with intermediate adjustment of the parameter vector under optimization this mode of operation often leads to faster convergence when far from the eventual limit and to slower sublinear convergence when close to the optimal solution we embed both lms and steepest descent as well as other intermediate methods within a one parameter class of algorithms and we propose a hybrid class of methods that combine the faster early convergence rate of lms with the faster ultimate linear convergence rate of steepest descent these methods are well suited for neural network training problems with large data sets furthermore these methods allow the effective use of scaling based for example on diagonal or other approximations of the hessian matrix

l p norm multiple kernel learning learning linear combinations of multiple kernels is an appealing strategy when the right choice of features is unknown previous approaches to multiple kernel learning mkl promote sparse kernel combinations to support interpretability and scalability unfortunately this l1 norm mkl is rarely observed to outperform trivial baselines in practical applications to allow for robust kernel mixtures that generalize well we extend mkl to arbitrary norms we devise new insights on the connection between several existing mkl formulations and develop two efficient interleaved optimization strategies for arbitrary norms that is lp norms with p 1 this interleaved optimization is much faster than the commonly used wrapper approaches as demonstrated on several data sets a theoretical analysis and an experiment on controlled artificial data shed light on the appropriateness of sparse non sparse and l norm mkl in various scenarios importantly empirical applications of lp norm mkl to three real world problems from computational biology show that non sparse mkl achieves accuracies that surpass the state of the art data sets source code to reproduce the experiments implementations of the algorithms and further information are available at http doc ml tu berlin de nonsparse mkl

parameter identification for hyperbolic stochastic systems parameter identification is studied for hyperbolic stochastic systems explicit sufficient conditions ensuring consistency of the maximum likelihood estimate are given

accelerated image reconstruction using ordered subsets of projection data the authors define ordered subset processing for standard algorithms such as expectation maximization em for image restoration from projections ordered subsets methods group projection data into an ordered sequence of subsets or blocks an iteration of ordered subsets em is defined as a single pass through all the subsets in each subset using the current estimate to initialize application of em with that data subset this approach is similar in concept to block kaczmarz methods introduced by eggermont et al 1981 for iterative reconstruction simultaneous iterative reconstruction sirt and multiplicative algebraic reconstruction mart techniques are well known special cases ordered subsets em os em provides a restoration imposing a natural positivity condition and with close links to the em algorithm os em is applicable in both single photon spect and positron emission tomography pet in simulation studies in spect the os em algorithm provides an order of magnitude acceleration over em with restoration quality maintained

multitask classification hypothesis space with improved generalization bounds this paper presents a pair of hypothesis spaces hss of vector valued functions intended to be used in the context of multitask classification while both are parameterized on the elements of reproducing kernel hilbert spaces and impose a feature mapping that is common to all tasks one of them assumes this mapping as fixed while the more general one learns the mapping via multiple kernel learning for these new hss empirical rademacher complexity based generalization bounds are derived and are shown to be tighter than the bound of a particular hs which has appeared recently in the literature leading to improved performance as a matter of fact the latter hs is shown to be a special case of ours based on an equivalence to group lasso type hss the proposed hss are utilized toward corresponding support vector machine based formulations finally experimental results on multitask learning problems underline the quality of the derived bounds and validate this paper s analysis

maximum likelihood reconstruction for emission tomography previous models for emission tomography et do not distinguish the physics of et from that of transmission tomography we give a more accurate general mathematical model for et where an unknown emission density x y z generates and is to be reconstructed from the number of counts n d in each of d detector units d within the model we give an algorithm for determining an estimate of which maximizes the probability p n of observing the actual detector count data n over all possible densities let independent poisson variables n b with unknown means b b 1 b represent the number of unobserved emissions in each of b boxes pixels partitioning an object containing an emitter suppose each emission in box b is detected in detector unit d with probability p b d d 1 d with p b d a one step transition matrix assumed known we observe the total number n n d of emissions in each detector unit d and want to estimate the unknown b b 1 b for each the observed data n has probability or likelihood p n the em algorithm of mathematical statistics starts with an initial estimate 0 and gives the following simple iterative procedure for obtaining a new estimate new from an old estimate old to obtain k k 1 2 new b old b dd 1 n d p b d old b p b d b 1 b

generalized hierarchical kernel learning this paper generalizes the framework of hierarchical kernel learning hkl and illustrates its utility in the domain of rule learning hkl involves multiple kernel learning over a set of given base kernels assumed to be embedded on a directed acyclic graph this paper proposes a two fold generalization of hkl the first is employing a generic l1 lp block norm regularizer 1 2 that alleviates a key limitation of the hkl formulation the second is a generalization to the case of multi class multi label and more generally multi task applications the main technical contribution of this work is the derivation of a highly specialized partial dual of the proposed generalized hkl formulation and an efficient mirror descent based active set algorithm for solving it importantly the generic regularizer enables the proposed formulation to be employed in the rule ensemble learning rel where the goal is to construct an ensemble of conjunctive propositional rules experiments on benchmark rel data sets illustrate the efficacy of the proposed generalizations

convergence results for continuous time adaptive stochastic filtering algorithms abstract the adaptive stochastic filtering problem for gaussian processes is considered the self tuning synthesis procedure is used to derive two algorithms for this problem almost sure convergence for the parameter estimate and the filtering error will be established the convergence analysis is based on an almost supermartingale convergence lemma that allows a stochastic lyapunov like approach

regularized maximum likelihood estimate for an infinite dimensional parameter in stochastic parabolic systems the purpose of this paper is to study the identification problem of an infinite dimensional parameter more precisely a spatially varying parameter in stochastic diffusion equations in a previous study s i aihara and y sunahara siam j control optim 26 1988 pp 1062 1075 some explicit conditions for the consistency property of the maximum likelihood estimate mle is explored here an algorithm for generating the mle is developed with the aid of the regularization technique proposed by c kravaris and j h seinfeld siam j control optim 23 1985 pp 217 241 after the consistency property of the mle by a regularization is proved necessary conditions for the regularized mle rmle are derived proposed is an iterative algorithm for computing one of the solutions of the necessary conditions derived the convergence property of the sequence generated by the proposed algorithm is also shown finally numerical examples are presented

accelerated iterative transmission ct reconstruction using an ordered subsets convex algorithm iterative maximum likelihood ml transmission computed tomography algorithms have distinct advantages over fourier based reconstruction but unfortunately require increased computation time the convex algorithm is a relatively fast iterative ml algorithm but it is nevertheless too slow for many applications therefore an acceleration of this algorithm by using ordered subsets of projections is proposed ordered subsets convex algorithm osc osc applies the convex algorithm sequentially to subsets of projections osc was compared with the convex algorithm using simulated and physical thorax phantom data reconstructions were performed for osc using eight and 16 subsets eight and four projections subset respectively global errors image noise contrast recovery and likelihood increase were calculated results show that osc is faster than the convex algorithm the amount of acceleration being approximately proportional to the number of subsets in osc and it causes only a slight increase of noise and global errors in the reconstructions images and image profiles of the reconstructions were in good agreement in conclusion osc and the convex algorithm result in similar image quality but osc is more than an order of magnitude faster

on the convergence of the lms algorithm with adaptive learning rate for linear feedforward networks we consider the problem of training a linear feedforward neural network by using a gradient descent like lms learning algorithm the objective is to find a weight matrix for the network by repeatedly presenting to it a finite set of examples so that the sum of the squares of the errors is minimized kohonen showed that with a small but fixed learning rate or stepsize some subsequences of the weight matrices generated by the algorithm will converge to certain matrices close to the optimal weight matrix in this paper we show that by dynamically decreasing the learning rate during each training cycle the sequence of matrices generated by the algorithm will converge to the optimal weight matrix we also show that for any given 0 the lms algorithm with decreasing learning rates will generate an optimal weight matrix i e a matrix of distance at most away from the optimal matrix after o 1 training cycles this is in contrast to 1 log 1 training cycles needed to generate an optima

variable sparsity kernel learning this paper presents novel algorithms and applications for a particular class of mixed norm regularization based multiple kernel learning mkl formulations the formulations assume that the given kernels are grouped and employ l1 norm regularization for promoting sparsity within rkhs norms of each group and ls s 2 norm regularization for promoting non sparse combinations across groups various sparsity levels in combining the kernels can be achieved by varying the grouping of kernels hence we name the formulations as variable sparsity kernel learning vskl formulations while previous attempts have a non convex formulation here we present a convex formulation which admits efficient mirror descent md based solving techniques the proposed md based algorithm optimizes over product of simplices and has a computational complexity of o m2ntot log nmax e2 where m is no training data points nmax ntot are the maximum no kernels in any group total no kernels respectively and e is the error in approximating the objective a detailed proof of convergence of the algorithm is also presented experimental results show that the vskl formulations are well suited for multi modal learning tasks like object categorization results also show that the md based algorithm outperforms state of the art mkl solvers in terms of computational efficiency

characterization of completions of unique factorization domains it is shown that a complete local ring is the completion of a unique factorization domain if and only if it is a field a discrete valuation ring or it has depth at least two and no element of its prime ring is a zerodivisor it is also shown that the normal chain conjecture is false and that there exist local noncatenary ufds

some results on tight closure and completion abstract we construct two examples of nonexcellent local noetherian domains which demonstrate that tight closure and completion do not commute the first example is a local normal domain a with a height one principal prime ideal p such that pa p a we also construct an example of a complete local normal gorenstein domain which is not f regular but is the completion of an f regular local ring

lovasz ϑ function svms and finding dense subgraphs in this paper we establish that the lovasz function on a graph can be restated as a kernel learning problem we introduce the notion of svm graphs on which lovasz function can be approximated well by a support vector machine svm we show that erdos renyi random g n p graphs are svm graphs for log4 n n p 1 even if we embed a large clique of size np 1 p in a g n p graph the resultant graph still remains a svm graph this immediately suggests an svm based algorithm for recovering a large planted clique in random graphs associated with the function is the notion of orthogonal labellings we introduce common orthogonal labellings which extends the idea of orthogonal labellings to multiple graphs this allows us to propose a multiple kernel learning mkl based solution which is capable of identifying a large common dense subgraph in multiple graphs both in the planted clique case and common subgraph detection problem the proposed solutions beat the state of the art by an order of magnitude

excellent rings with local generic formal fibers abstract let t m be a complete regular local ring of dimension at least 2 containing the rationals such that the cardinality of the residue field t m is at least the cardinality of the real numbers suppose p m is a prime ideal of t we construct an excellent regular local ring a such that the completion of a is t and the generic formal fiber of a is local this means that the ring t a k is a local ring where k is the quotient field of a with p a k its maximal ideal in addition if q is a prime ideal of t with q a p 0 then q pt

a singularity theory approach to a semilinear boundary value problem where a b 3 are real parameters with 0 a b 1 this problem arises in population genetics with the unknown function u corresponding to the relative frequency of a gene the parameter a is given by a combination of the death rates of the various genotypes involved b gives the frequency of the gene on the boundary of the region and 1 is the reciprocal of the diffusion rate of the gene see aronson and weinberger 1 for further details solutions of 1 1 correspond to possible steady states of the gene populations we are interested mainly in how the bifurcation diagram changes as a and b vary and we investigate this question by exploiting singularity theory for bifurcation problems as developed by golubitsky and schaeffer c51 equation 1 1 is a simple example of a semilinear elliptic boundary value problem and a standard method of investigating the bifurcation of

incremental least squares methods and the extended kalman filter in this paper we propose and analyze nonlinear least squares methods which process the data incrementally one data block at a time such methods are well suited for large data sets and real time operation and have received much attention in the context of neural network training problems we focus on the extended kalman filter which may be viewed as an incremental version of the gauss newton method we provide a nonstochastic analysis of its convergence properties and we discuss variants aimed at accelerating its convergence

learning the kernel function via regularization we study the problem of finding an optimal kernel from a prescribed convex set of kernels k for learning a real valued function by regularization we establish for a wide variety of regularization functionals that this leads to a convex optimization problem and for square loss regularization we characterize the solution of this problem we show that although k may be an uncountable set the optimal kernel is always obtained as a convex combination of at most m 2 basic kernels where m is the number of data examples in particular our results apply to learning the optimal radial kernel or the optimal dot product kernel

pareto path multitask multiple kernel learning a traditional and intuitively appealing multitask multiple kernel learning mt mkl method is to optimize the sum thus the average of objective functions with partially shared kernel function which allows information sharing among the tasks we point out that the obtained solution corresponds to a single point on the pareto front pf of a multiobjective optimization problem which considers the concurrent optimization of all task objectives involved in the multitask learning mtl problem motivated by this last observation and arguing that the former approach is heuristic we propose a novel support vector machine mt mkl framework that considers an implicitly defined set of conic combinations of task objectives we show that solving our framework produces solutions along a path on the aforementioned pf and that it subsumes the optimization of the average of objective functions as a special case using the algorithms we derived we demonstrate through a series of experimental results that the framework is capable of achieving a better classification performance when compared with other similar mtl approaches

the ordered subsets mirror descent optimization method with applications to tomography we describe an optimization problem arising in reconstructing three dimensional medical images from positron emission tomography pet a mathematical model of the problem based on the maximum likelihood principle is posed as a problem of minimizing a convex function of several million variables over the standard simplex to solve a problem of these characteristics we develop and implement a new algorithm ordered subsets mirror descent and demonstrate theoretically and computationally that it is well suited for solving the pet reconstruction problem

an incremental gradient projection method with momentum term and adaptive stepsize rule we consider an incremental gradient method with momentum term for minimizing the sum of continuously differentiable functions this method uses a new adaptive stepsize rule that decreases the stepsize whenever sufficient progress is not made we show that if the gradients of the functions are bounded and lipschitz continuous over a certain level set then every cluster point of the iterates generated by the method is a stationary point in addition if the gradient of the functions have a certain growth property then the method is either linearly convergent in some sense or the stepsizes are bounded away from zero the new stepsize rule is much in the spirit of heuristic learning rules used in practice for training neural networks via backpropagation as such the new stepsize rule may suggest improvements on existing learning rules finally extension of the method and the convergence results to constrained minimization is discussed as are some implementation issues and numerical experience

tight closure in non excellent rings abstract some examples are constructed to demonstrate that tight closure need not satisfy several key properties when the ring is not excellent in particular there are counterexamples to colon capturing and persistence a consequence is that r need not be cohen macaulay when r is a local ring of characteristic p even when r is universally catenarian a new closure operation for non excellent local rings of characteristic p is introduced and it is seen to have most of the properties associated with tight closure

learning the kernel matrix with semidefinite programming kernel based learning algorithms work by embedding the data into a euclidean space and then searching for linear relations among the embedded data points the embedding is performed implicitly by specifying the inner products between each pair of points in the embedding space this information is contained in the so called kernel matrix a symmetric and positive semidefinite matrix that encodes the relative positions of all points specifying this matrix amounts to specifying the geometry of the embedding space and inducing a notion of similarity in the input space classical model selection problems in machine learning in this paper we show how the kernel matrix can be learned from data via semidefinite programming sdp techniques when applied to a kernel matrix associated with both training and test data this gives a powerful transductive algorithm using the labeled part of the data one can learn an embedding also for the unlabeled part the similarity between test points is inferred from training points and their labels importantly these learning problems are convex so we obtain a method for learning both the model class and the function without local minima furthermore this approach leads directly to a convex method for learning the 2 norm soft margin parameter in support vector machines solving an important open problem

adaptive control via a simple switching algorithm in this paper we present an adaptive stabilization control for systems with unknown constant parameters and stochastic disturbances which may be neither open loop stable nor minimum phase the ideas come from previous works j f zhang and h f chen adaptive stabilization under the weakest condition proc 31st control and design conference december 14 18 1992 pp 3620 3621 and h f chen continuous time stochastic adaptive control stabilizing the system and minimizing the quadratic loss function tech report institute of systems science academia sinica beijing 1992 but here we not only simplify the construction procedure of an adaptive control but also reduce the computational load significantly so that the adaptive control in this paper is more practical furthermore parameter estimation is carried out in only a finite time period and unlike previous work the parameter estimates are generated by ordinary differential equations rather than stochastic differential equations

a statistical model for positron emission tomography abstract positron emission tomography pet still in its research stages is a technique that promises to open new medical frontiers by enabling physicians to study the metabolic activity of the body in a pictorial manner much as in x ray transmission tomography and other modes of computerized tomography the quality of the reconstructed image in pet is very sensitive to the mathematical algorithm to be used for reconstruction in this article we tailor a mathematical model to the physics of positron emissions and we use the model to describe the basic image reconstruction problem of pet as a standard problem in statistical estimation from incomplete data we describe various estimation procedures such as the maximum likelihood ml method using the em algorithm the method of moments and the least squares method a computer simulation of a pet experiment is then used to demonstrate the ml and the least squares reconstructions the main purposes of this article are to report on what we believe is an

test ideals and base change problems in tight closure theory test ideals are an important concept in tight closure theory and their behavior via flat base change can be very difficult to understand our paper presents results regarding this behavior under flat maps with reasonably nice but far from smooth fibers this involves analyzing in depth a special type of ideal of test elements called the cs test ideal besides providing new results the paper also contains extensions of a theorem by g lyubeznik and k e smith on the completely stable test ideal and of theorems by f enescu and independently m hashimoto on the behavior of f rationality under flat base change

band reweighed gabor kernel embedding for face image representation and recognition face recognition with illumination or pose variation is a challenging problem in image processing and pattern recognition a novel algorithm using band reweighed gabor kernel embedding to deal with the problem is proposed in this paper for a given image it is first transformed by a group of gabor filters which output gabor features using different orientation and scale parameters fisher scoring function is used to measure the importance of features in each band and then the features with the largest scores are preserved for saving memory requirements the reduced bands are combined by a vector which is determined by a weighted kernel discriminant criterion and solved by a constrained quadratic programming method and then the weighted sum of these nonlinear bands is defined as the similarity between two images compared with existing concatenation based gabor feature representation and the uniformly weighted similarity calculation approaches our method provides a new way to use gabor features for face recognition and presents a reasonable interpretation for highlighting discriminant orientations and scales the minimum mahalanobis distance considering the spatial correlations within the data is exploited for feature matching and the graphical lasso is used therein for directly estimating the sparse inverse covariance matrix experiments using benchmark databases show that our new algorithm improves the recognition results and obtains competitive performance