tensor representation learning based image patch analysis for text identification and recognition in this paper we introduce a novel framework for text identification and recognition called tensor representation learning based image patch analysis trl ipa unlike most of previous text identification approaches which can only be applied to binarized images trl ipa can be directly applied to gray level and color images trl ipa is built on a general formulation of the convergent tensor representation learning ctrl algorithms in the implementation of trl ipa image patches are represented in the form of tensors while low dimensional representations of these tensors are learned via a ctrl algorithm to identify text regions in new coming document images a random forest classifier is trained in the learned tensor subspace moreover the trl ipa framework can be straightforwardly applied to recognition problems such as handwritten digits recognition we conducted extensive experiments on ancient chinese arabic and cyrillic document images to evaluate trl ipa on text identification tasks experimental results demonstrate its effectiveness and robustness in addition recognition results on images of handwritten digits show its advantage over state of the art vector and tensor representation based approaches highlightsa novel model tensor representation learning based image patch analysis trl ipa is proposed for document understanding trl ipa is built on a general formulation of the convergent tensor representation learning ctrl algorithms the ctrl algorithms are theoretically guaranteed to converge to a local optimal solution of the learning problem extensive experiments demonstrate the superiority of trl ipa over related vector and tensor representation based approaches

multilinear calderon zygmund theory abstract a systematic treatment of multilinear calderon zygmund operators is presented the theory developed includes strong type and endpoint weak type estimates interpolation a multilinear t 1 theorem and a variety of results regarding multilinear multiplier operators

dictionary pair learning on grassmann manifolds for image denoising image denoising is a fundamental problem in computer vision and image processing that holds considerable practical importance for real world applications the traditional patch based and sparse coding driven image denoising methods convert 2d image patches into 1d vectors for further processing thus these methods inevitably break down the inherent 2d geometric structure of natural images to overcome this limitation pertaining to the previous image denoising methods we propose a 2d image denoising model namely the dictionary pair learning dpl model and we design a corresponding algorithm called the dpl on the grassmann manifold dplg algorithm the dplg algorithm first learns an initial dictionary pair i e the left and right dictionaries by employing a subspace partition technique on the grassmann manifold wherein the refined dictionary pair is obtained through a sub dictionary pair merging the dplg obtains a sparse representation by encoding each image patch only with the selected sub dictionary pair the non zero elements of the sparse representation are further smoothed by the graph laplacian operator to remove the noise consequently the dplg algorithm not only preserves the inherent 2d geometric structure of natural images but also performs manifold smoothing in the 2d sparse coding space we demonstrate that the dplg algorithm also improves the structural similarity values of the perceptual visual quality for denoised images using the experimental evaluations on the benchmark images and berkeley segmentation data sets moreover the dplg also produces the competitive peak signal to noise ratio values from popular image denoising algorithms

stdp compatible approximation of backpropagation in an energy based model we show that langevin markov chain monte carlo inference in an energy based model with latent variables has the property that the early steps of inference starting from a stationary point correspond to propagating error gradients into internal layers similar to backpropagation the backpropagated error is with respect to output units that have received an outside driving force pushing them away from the stationary point backpropagated error gradients correspond to temporal derivatives with respect to the activation of hidden units these lead to a weight update proportional to the product of the presynaptic firing rate and the temporal rate of change of the postsynaptic firing rate simulations and a theoretical argument suggest that this rate based update rule is consistent with those associated with spike timing dependent plasticity the ideas presented in this article could be an element of a theory for explaining how brains perform credit assignment in deep hierarchies as efficiently as backpropagation does with neural computation corresponding to both approximate inference in continuous valued latent variables and error backpropagation at the same time

large margin low rank tensor analysis we present a supervised model for tensor dimensionality reduction which is called large margin low rank tensor analysis lmlrta in contrast to traditional vector representation based dimensionality reduction methods lmlrta can take any order of tensors as input and unlike previous tensor dimensionality reduction methods which can learn only the low dimensional embeddings with a priori specified dimensionality lmlrta can automatically and jointly learn the dimensionality and the low dimensional representations from data moreover lmlrta delivers low rank projection matrices while it encourages data of the same class to be close and of different classes to be separated by a large margin of distance in the low dimensional tensor space lmlrta can be optimized using an iterative fixed point continuation algorithm which is guaranteed to converge to a local optimal solution of the optimization problem we evaluate lmlrta on an object recognition application where the data are represented as 2d tensors and a face recognition application where the data are represented as 3d tensors experimental results show the superiority of lmlrta over state of the art approaches

darboux transformation of the variable coefficient nonlocal equation abstract in this paper we investigate an integrable variable coefficient nonlocal nonlinear schrodinger equation nnls for the first time with the help of lax pair darboux transformations and exact solutions to the variable coefficient nnls are studied it is shown that we can get many more general solutions than equation with constant coefficient in addition some solutions of variable coefficient nnls equation with specific coefficients are given

justifying and generalizing contrastive divergence we study an expansion of the log likelihood in undirected graphical models such as the restricted boltzmann machine rbm where each term in the expansion is associated with a sample in a gibbs chain alternating between two random variables the visible vector and the hidden vector in rbms we are particularly interested in estimators of the gradient of the log likelihood obtained through this expansion we show that its residual term converges to zero justifying the use of a truncation running only a short gibbs chain which is the main idea behind the contrastive divergence cd estimator of the log likelihood gradient by truncating even more we obtain a stochastic reconstruction error related through a mean field approximation to the reconstruction error often used to train autoassociators and stacked autoassociators the derivation is not specific to the particular parametric forms used in rbms and requires only convergence of the gibbs chain we present theoretical and empirical evidence linking the number of gibbs steps k and the magnitude of the rbm parameters to the bias in the cd estimator these experiments also suggest that the sign of the cd estimator is correct most of the time even when the bias is large so that cd k is a good descent direction even for small k

graph regularized restricted boltzmann machine the restricted boltzmann machine rbm has received an increasing amount of interest in recent years it determines good mapping weights that capture useful latent features in an unsupervised manner the rbm and its generalizations have been successfully applied to a variety of image classification and speech recognition tasks however most of the existing rbm based models disregard the preservation of the data manifold structure in many real applications the data generally reside on a low dimensional manifold embedded in high dimensional ambient space in this brief we propose a novel graph regularized rbm to capture features and learning representations explicitly considering the local manifold structure of the data by imposing manifold based locality that preserves constraints on the hidden layer of the rbm the model ultimately learns sparse and discriminative representations the representations can reflect data distributions while simultaneously preserving the local manifold structure of data we test our model using several benchmark image data sets for unsupervised clustering and supervised classification problem the results demonstrate that the performance of our method exceeds the state of the art alternatives

the effects of adding noise during backpropagation training on a generalization performance we study the effects of adding noise to the inputs outputs weight connections and weight changes of multilayer feedforward neural networks during backpropagation training we rigorously derive and analyze the objective functions that are minimized by the noise affected training processes we show that input noise and weight noise encourage the neural network output to be a smooth function of the input or its weights respectively in the weak noise limit noise added to the output of the neural networks only changes the objective function by a constant hence it cannot improve generalization input noise introduces penalty terms in the objective function that are related to but distinct from those found in the regularization approaches simulations have been performed on a regression and a classification problem to further substantiate our analysis input noise is found to be effective in improving the generalization performance for both problems however weight noise is found to be effective in improving the generalization performance only for the classification problem other forms of noise have practically no effect on generalization

the existence of least area surfaces in 3 manifolds this paper presents a new and unified approach to the existence theorems for least area surfaces in 3 manifolds introduction a surface f smoothly embedded or immersed in a riemannian manifold m is minimal if it has mean curvature zero at all points it is a least area surface in a class of surfaces if it has finite area which realizes the infimum of all possible areas for surfaces in this class the connection between these two ideas is that a surface which is of least area in a reasonable class of surfaces must be minimal the converse is false minimal surfaces are in general only critical points for the area function there are close analogies between these two concepts and the theory of geodesics in a riemannian manifold minimal surfaces correspond to geodesics and least area surfaces correspond to geodesic arcs or closed geodesics which have shortest length in some class of paths any geodesic a in a riemannian manifold m has the property that it is locally shortest i e if p and q are nearby points on a then the subarc of a which joins p and q is the shortest path in m from p to q it can also be proved that minimal surfaces are locally of least area but the proof is difficult and involves substantial knowledge of the theory of partial differential equations there are now a large number of theorems asserting the existence of surfaces of least area in various classes surfaces of this type have become an important tool in 3 dimensional topology in this paper we present a new approach to the proofs of these existence theorems this yields a simplified and unified method for the proof of the existence of minimal surfaces in 3 dimensional riemannian manifolds in 1930 douglas do and rado ra independently showed that a simple closed curve in ri which bounds a disk of finite area bounds a disk of least possible area this result was extended by morrey moi in 1948 to a general class of riemannian manifolds the homogeneously regular manifolds a class which includes all closed manifolds work of osserman 0 and gulliver g later showed that least area disks in 3 manifolds were immersed in their interiors more recently there has been a series of new existence results for closed surfaces of least area in manifolds of any dimension it follows from work of sacks and uhlenbeck s ui that if m is closed and 7r2 m is nonzero then there is an essential map of the 2 sphere into m which has least area among all essential maps sacks and uhlenbeck s uii and schoen and yau s y independently showed that if f f m is a map of a closed orientable surface f not the 2 sphere into a closed riemannian manifold m such received by the editors june 23 1986 1980 mathematics subject classification 1985 revision primary 53a10 secondary 57m35 the first author was supported by nsf postdoctoral fellowship dms84 14097 1988 american mathematical society 0002 9947 88 1 00 25 per page

dimension reduction with extreme learning machine data may often contain noise or irrelevant information which negatively affect the generalization capability of machine learning algorithms the objective of dimension reduction algorithms such as principal component analysis pca non negative matrix factorization nmf random projection rp and auto encoder ae is to reduce the noise or irrelevant information of the data the features of pca eigenvectors and linear ae are not able to represent data as parts e g nose in a face image on the other hand nmf and non linear ae are maimed by slow learning speed and rp only represents a subspace of original data this paper introduces a dimension reduction framework which to some extend represents data as parts has fast learning speed and learns the between class scatter subspace to this end this paper investigates a linear and non linear dimension reduction framework referred to as extreme learning machine ae elm ae and sparse elm ae selm ae in contrast to tied weight ae the hidden neurons in elm ae and selm ae need not be tuned and their parameters e g input weights in additive neurons are initialized using orthogonal and sparse random weights respectively experimental results on usps handwritten digit recognition data set cifar 10 object recognition and norb object recognition data set show the efficacy of linear and non linear elm ae and selm ae in terms of discriminative capability sparsity training time and normalized mean square error

learning consensus representation for weak style classification style classification e g baroque and gothic architecture style is grabbing increasing attention in many fields such as fashion architecture and manga most existing methods focus on extracting discriminative features from local patches or patterns however the spread out phenomenon in style classification has not been recognized yet it means that visually less representative images in a style class are usually very diverse and easily getting misclassified we name them weak style images another issue when employing multiple visual features towards effective weak style classification is lack of consensus among different features that is weights for different visual features in the local patch should have been allocated similar values to address these issues we propose a consensus style centralizing auto encoder cscae for learning robust style features representation especially for weak style classification first we propose a style centralizing auto encoder scae which centralizes weak style features in a progressive way then based on scae we propose both the non linear and linear version cscae which adaptively allocate weights for different features during the progressive centralization process consensus constraints are added based on the assumption that the weights of different features of the same patch should be similar specifically the proposed linear counterpart of cscae motivated by the shared weights idea as well as group sparsity improves both efficacy and efficiency for evaluations we experiment extensively on fashion manga and architecture style classification problems in addition we collect a new dataset online shopping for fashion style classification which will be publicly available for vision based fashion style research experiments demonstrate the effectiveness of the scae and cscae on both public and newly collected datasets when compared with the most recent state of the art works

dual autoencoders features for imbalance classification problem many classification problems encountered in real world applications exhibit a profile of imbalanced data current methods depend on data resampling in fact if the feature set provides a clear decision boundary resampling may not be needed to solve the imbalanced classification problem therefore this work proposes a feature learning method based on the autoencoder to learn a set of features with better classification capabilities of the minority and the majority classes to address the imbalanced classification problems two sets of features are learned by two stacked autoencoders with different activation functions to capture different characteristics of the data and they are combined to form the dual autoencoding features samples are then classified in the new feature space learned in this manner instead of the original input space experimental results show that the proposed method outperforms current resampling based methods with statistical significance for imbalanced pattern classification problems highlightsto our best knowledge this paper proposes the first feature learning based method for dealing with imbalance pattern classification using stacked autoencoders we provide a new viewpoint for solving imbalanced pattern classification problems by projecting the input space onto a learned feature space with better representation using stacked autoencoders the concatenation of learned features from two different stacked autoencoders further enhances the classification accuracy by providing a better representation with different characteristics of the data to the classifier to learn more specifically the sigmoid activation function is less sensitive to input changes and provides robust representation while the tanh activation function is more sensitive and provides detailed information of the data experimental results show that the daf yields statistical significant improvement in the auc the f1 score and the g mean in comparison to representative methods of resampling based and feature projection methods for dealing with the imbalanced pattern classification problems

on calderon s conjecture with constants cfi p1 p2 depending only on fi p1 p2 and p p1p2 p1 p2 hold the flrst result of this type is proved in 4 and the purpose of the current paper is to extend the range of exponents p1 and p2 for which 2 is known in particular the case p1 2 p2 1 is solved to the a rmative this was

graph regularized auto encoders for image representation image representation has been intensively explored in the domain of computer vision for its significant influence on the relative tasks such as image clustering and classification it is valuable to learn a low dimensional representation of an image which preserves its inherent information from the original image space at the perspective of manifold learning this is implemented with the local invariant idea to capture the intrinsic low dimensional manifold embedded in the high dimensional input space inspired by the recent successes of deep architectures we propose a local invariant deep nonlinear mapping algorithm called graph regularized auto encoder gae with the graph regularization the proposed method preserves the local connectivity from the original image space to the representation space while the stacked auto encoders provide explicit encoding model for fast inference and powerful expressive capacity for complex modeling theoretical analysis shows that the graph regularizer penalizes the weighted frobenius norm of the jacobian matrix of the encoder mapping where the weight matrix captures the local property in the input space furthermore the underlying effects on the hidden representation space are revealed providing insightful explanation to the advantage of the proposed method finally the experimental results on both clustering and classification tasks demonstrate the effectiveness of our gae as well as the correctness of the proposed theoretical analysis and it also suggests that gae is a superior solution to the current deep representation learning techniques comparing with variant auto encoders and existing local invariant methods

bilinear integral operators with certain hypersingularities abstract in this paper we establish the sharp l p boundedness for the bilinear integral operators with certain hypersingularities that generalize the bilinear hilbert transform

locator checker scaler object tracking using spatially ordered and weighted patch descriptor in this paper we propose a simple yet effective object descriptor and a novel tracking algorithm to track a target object accurately for the object description we divide the bounding box of a target object into multiple patches and describe them with color and gradient histograms then we determine the foreground weight of each patch to alleviate the impacts of background information in the bounding box to this end we perform random walk with restart rwr simulation we then concatenate the weighted patch descriptors to yield the spatially ordered and weighted patch sowp descriptor for the object tracking we incorporate the proposed sowp descriptor into a novel tracking algorithm which has three components locator checker and scaler lcs the locator and the scaler estimate the center location and the size of a target respectively the checker determines whether it is safe to adjust the target scale in a current frame these three components cooperate with one another to achieve robust tracking experimental results demonstrate that the proposed lcs tracker achieves excellent performance on recent benchmarks

a fuzzy restricted boltzmann machine novel learning algorithms based on the crisp possibilistic mean value of fuzzy numbers a fuzzy restricted boltzmann machine frbm is extended from a restricted boltzmann machine rbm by replacing all the real valued parameters with fuzzy numbers a new frbm that employs the crisp possibilistic mean value of a fuzzy number to defuzzify the fuzzy free energy function is presented this approach is much clearer and easier to obtain the expression of the defuzzified free energy function and its approximation than the centroid method several theorems that discuss the error bounds of the approximation to ensure the rationality and validity are also investigated learning algorithms are given for the designed frbm with symmetric triangular fuzzy numbers stfns asymmetric triangular fuzzy numbers and gaussian fuzzy numbers by appropriately choosing the parameters a theorem is concluded that all frbms with symmetric fuzzy numbers will have identical learning algorithm to that of frbms with stfns this is illustrated by a case of frbm with gaussian fuzzy numbers two experiments including the mnist handwriting recognition and the bars and stripes benchmark are carried out the results show that the proposed frbms significantly outperform rbms in learning accuracy and generalization ability especially when encountering unlearned samples and recovering incomplete images

embedded minimal surfaces exotic spheres and manifolds with positive ricci curvature let n be a three dimensional riemannian manifold let e be a closed embedded surface in n then it is a question of basic interest to see whether one can deform in its isotopy class to some canonical embedded surface from the point of view of geometry a natural canonical surface will be the extremal surface of some functional defined on the space of embedded surfaces the simplest functional is the area functional the extremal surface of the area functional is called the minimal surface such minimal surfaces were used extensively by meeks yau my21 in studying group actions on three dimensional manifolds in my2 the theory of minimal surfaces was used to simplify and strengthen the classical dehn s lemma loop theorem and the sphere theorem in the setting there one minimizes area among all immersed surfaces and proves that the extremal object is embedded in this paper we minimize area among all embedded surfaces isotopic to a fixed embedded surface in the category of these surfaces we prove a general existence theorem theorem 1 a particular consequence of this theorem is that for irreducible manifolds an embedded incompressible surface is isotopic to an embedded incompressible surface with minimal area we also prove that there exists an embedded sphere of least area enclosing a fake cell provided the complementary volume is not a standard ball and provided there exists no embedded one sided rp2 by making use of the last result and a cutting and pasting argument we are able to settle a well known problem in the theory of three dimensional manifolds we prove that the covering space of any irreducible orientable three dimensional manifold is irreducible it is possible to exploit our existence theorem to study

cones embedded in hyperbolic manifolds we show that the existence of a maximal embedded tube in a hyperbolic n manifold implies the existence of a certain conical region one application is to establish a lower bound on the volume of the region outside the tube thereby improving estimates on volume and estimates on lengths of geodesics in small volume hyperbolic 3 manifolds we also provide new bounds on the injectivity radius and diameter of an n manifold

arithmeticity discreteness and volume we give an arithmetic criterion which is sufficient to imply the discreteness of various two generator subgroups of p sl 2 c we then examine certain two generator groups which arise as extremals in various geometric problems in the theory of kleinian groups in particular those encountered in efforts to determine the smallest co volume the margulis constant and the minimal distance between elliptic axes we establish the discreteness and arithmeticity of a number of these extremal groups the associated minimal volume arithmetic group in the commensurability class and we study whether or not the axis of a generator is simple

probabilistic low rank multitask learning in this paper we consider the problem of learning multiple related tasks simultaneously with the goal of improving the generalization performance of individual tasks the key challenge is to effectively exploit the shared information across multiple tasks as well as preserve the discriminative information for each individual task to address this we propose a novel probabilistic model for multitask learning mtl that can automatically balance between low rank and sparsity constraints the former assumes a low rank structure of the underlying predictive hypothesis space to explicitly capture the relationship of different tasks and the latter learns the incoherent sparse patterns private to each task we derive and perform inference via variational bayesian methods experimental results on both regression and classification tasks on real world applications demonstrate the effectiveness of the proposed method in dealing with the mtl problems

subspace clustering using a low rank constrained autoencoder the performance of subspace clustering is affected by data representation data representation for subspace clustering maps data from the original space into another space with the property of better separability many data representation methods have been developed in recent years typical among them are low rank representation lrr and an autoencoder lrr is a linear representation method that captures the global structure of data with low rank constraint alternatively an autoencoder nonlinearly maps data into a latent space using a neural network by minimizing the difference between the reconstruction and input to combine the advantages of an lrr globality and autoencoder self supervision based locality we propose a novel data representation method for subspace clustering the proposed method called low rank constrained autoencoder lrae forces the latent representation of the neural network to be of low rank and the low rank constraint is computed as a prior from the input space one major advantage of the lrae is that the learned data representation not only maintains the local features of the data but also preserves the underlying low rank global structure extensive experiments on several datasets for subspace clustering were conducted they demonstrated that the proposed lrae substantially outperformed state of the art subspace clustering methods

a latent manifold markovian dynamics gaussian process in this paper we propose a gaussian process gp model for analysis of nonlinear time series formulation of our model is based on the consideration that the observed data are functions of latent variables with the associated mapping between observations and latent representations modeled through gp priors in addition to capture the temporal dynamics in the modeled data we assume that subsequent latent representations depend on each other on the basis of a hidden markov prior imposed over them derivation of our model is performed by marginalizing out the model parameters in closed form using gp priors for observation mappings and appropriate stick breaking priors for the latent variable markovian dynamics this way we eventually obtain a nonparametric bayesian model for dynamical systems that accounts for uncertainty in the modeled data we provide efficient inference algorithms for our model on the basis of a truncated variational bayesian approximation we demonstrate the efficacy of our approach considering a number of applications dealing with real world data and compare it with the related state of the art approaches

use of a tracer specific deep artificial neural net to denoise dynamic pet images application of kinetic modeling km on a voxel level in dynamic pet images frequently suffers from high levels of noise drastically reducing the precision of parametric image analysis in this paper we investigate the use of machine learning and artificial neural networks to denoise dynamic pet images we train a deep denoising autoencoder dae using noisy and noise free spatiotemporal image patches extracted from the simulated images of 11c raclopride a dopamine d2 receptor agonist the dae processed dynamic and corresponding parametric images simulated and acquired are compared with those obtained with conventional denoising techniques including temporal and spatial gaussian smoothing iterative spatiotemporal smoothing deconvolution and the highly constrained backprojection processing hypr the simulated acquired parametric image non uniformity was 7 75 19 49 with temporal and 5 90 14 50 with spatial smoothing 5 82 16 21 with smoothing deconvolution 5 49 13 38 with hypr and 3 52 11 41 with dae the dae also produced the best results in terms of the coefficient of variation of voxel values and structural similarity index denoising induced bias in the regional mean binding potential was 7 8 with temporal and 26 31 with spatial smoothing 28 61 with smoothing deconvolution 27 63 with hypr and 14 8 with dae when the test data did not match the training data erroneous outcomes were obtained our results demonstrate that a deep dae can provide a substantial reduction in the voxel level noise compared with the conventional spatiotemporal denoising methods while introducing a similar or lower amount of bias the better dae performance comes at the cost of lower generality and requiring appropriate training data

a fast learning algorithm for deep belief nets we show how to use complementary priors to eliminate the explaining away effects that make inference difficult in densely connected belief nets that have many hidden layers using complementary priors we derive a fast greedy algorithm that can learn deep directed belief networks one layer at a time provided the top two layers form an undirected associative memory the fast greedy algorithm is used to initialize a slower learning procedure that fine tunes the weights using a contrastive version of the wake sleep algorithm after fine tuning a network with three hidden layers forms a very good generative model of the joint distribution of handwritten digit images and their labels this generative model gives better digit classification than the best discriminative learning algorithms the low dimensional manifolds on which the digits lie are modeled by long ravines in the free energy landscape of the top level associative memory and it is easy to explore these ravines by using the directed connections to display what the associative memory has in mind

progressive shape distribution encoder for learning 3d shape representation since there are complex geometric variations with 3d shapes extracting efficient 3d shape features is one of the most challenging tasks in shape matching and retrieval in this paper we propose a deep shape descriptor by learning shape distributions at different diffusion time via a progressive shape distribution encoder psde first we develop a shape distribution representation with the kernel density estimator to characterize the intrinsic geometry structures of 3d shapes then we propose to learn a deep shape feature through an unsupervised psde specially the unsupervised psde aims at modeling the complex non linear transform of the estimated shape distributions between consecutive diffusion time in order to characterize the intrinsic structures of 3d shapes more efficiently we stack multiple psdes to form a network structure finally we concatenate all neurons in the middle hidden layers of the unsupervised psde network to form an unsupervised shape descriptor for retrieval furthermore by imposing an additional constraint on the outputs of all hidden layers we propose a supervised psde to form a supervised shape descriptor for each hidden layer the similarity between a pair of outputs from the same class is as large as possible and the similarity between a pair of outputs from different classes is as small as possible the proposed method is evaluated on three benchmark 3d shape data sets with large geometric variations i e mcgill shrec 10 shapegoogle and shrec 14 human data sets and the experimental results demonstrate the superiority of the proposed method to the existing approaches

training with noise is equivalent to tikhonov regularization it is well known that the addition of noise to the input data of a neural network during training can in some circumstances lead to significant improvements in generalization performance previous work has shown that such training with noise is equivalent to a form of regularization in which an extra term is added to the error function however the regularization term which involves second derivatives of the error function is not bounded below and so can lead to difficulties if used directly in a learning algorithm based on error minimization in this paper we show that for the purposes of network training the regularization term can be reduced to a positive semi definite form that involves only first derivatives of the network mapping for a sum of squares error function the regularization term belongs to the class of generalized tikhonov regularizers direct minimization of the regularized error function provides a practical alternative to training with noise

scott s rigidity theorem for seifert fibered spaces revisited we will present a new proof of the rigidity theorem for seifert fibered spaces of infinite 1 by scott 1983 in the case when the base of the fibration is a hyperbolic triangle 2 orbifold our proof is based on arguments in the rigidity theorem for hyperbolic 3 manifolds by gabai 1997

a connection between score matching and denoising autoencoders denoising autoencoders have been previously shown to be competitive alternatives to restricted boltzmann machines for unsupervised pretraining of each layer of a deep architecture we show that a simple denoising autoencoder training criterion is equivalent to matching the score with respect to the data of a specific energy based model to that of a nonparametric parzen density estimator of the data this yields several useful insights it defines a proper probabilistic model for the denoising autoencoder technique which makes it in principle possible to sample from them or rank examples by their energy it suggests a different way to apply score matching that is related to learning to denoise and does not require computing second derivatives it justifies the use of tied weights between the encoder and decoder and suggests ways to extend the success of denoising autoencoders to a larger family of energy based models

nonlinear autoassociation is not equivalent to pca a common misperception within the neural network community is that even with nonlinearities in their hidden layer autoassociators trained with backpropagation are equivalent to linear methods such as principal component analysis pca our purpose is to demonstrate that nonlinear autoassociators actually behave differently from linear methods and that they can outperform these methods when used for latent extraction projection and classification while linear autoassociators emulate pca and thus exhibit a flat or unimodal reconstruction error surface autoassociators with nonlinearities in their hidden layer learn domains by building error reconstruction surfaces that depending on the task contain multiple local valleys this interpolation bias allows nonlinear autoassociators to represent appropriate classifications of nonlinear multimodal domains in contrast to linear autoassociators which are inappropriate for such tasks in fact autoassociators with hidden unit nonlinearities can be shown to perform nonlinear classification and nonlinear recognition

the smale conjecture for seifert fibered spaces with hyperbolic base orbifold let m be a closed orientable 3 manifold admitting an mathbb h 2 times mathbb r or widetilde mathrm sl 2 mathbb r geometry or equivalently a seifert fibered space with a hyperbolic base 2 orbifold our main result is that the connected component of the identity map in the diffeomorphism group mathrm diff m is either contractible or homotopy equivalent to s 1 according as the center of pi 1 m is trivial or infinite cyclic apart from the remaining case of non haken infranilmanifolds this completes the homeomorphism classifications of mathrm diff m and of the space of seifert fiberings mathrm sf m for compact orientable aspherical 3 manifolds we also prove that when m has an mathbb h 2 times mathbb r or widetilde mathrm sl 2 mathbb r geometry and the base orbifold has underlying manifold the 2 sphere with three cone points the inclusion mathrm isom m to mathrm diff m is a homotopy equivalence

dimensionality reduction using similarity induced embeddings the vast majority of dimensionality reduction dr techniques rely on the second order statistics to define their optimization objective even though this provides adequate results in most cases it comes with several shortcomings the methods require carefully designed regularizers and they are usually prone to outliers in this paper a new dr framework that can directly model the target distribution using the notion of similarity instead of distance is introduced the proposed framework called similarity embedding framework sef can overcome the aforementioned limitations and provides a conceptually simpler way to express optimization targets similar to existing dr techniques deriving a new dr technique using the sef becomes simply a matter of choosing an appropriate target similarity matrix a variety of classical tasks such as performing supervised dr and providing out of sample extensions as well as new novel techniques such as providing fast linear embeddings for complex techniques are demonstrated in this paper using the proposed framework six data sets from a diverse range of domains are used to evaluate the proposed method and it is demonstrated that it can outperform many existing dr techniques

homotopy hyperbolic 3 manifolds are hyperbolic this paper introduces a rigorous computer assisted procedure for analyzing hyperbolic 3 manifolds this procedure is used to complete the proof of several long standing rigidity conjectures in 3 manifold theory as well as to provide a new lower bound for the volume of a closed orientable hyperbolic 3 manifold

using additive noise in back propagation training the possibility of improving the generalization capability of a neural network by introducing additive noise to the training samples is discussed the network considered is a feedforward layered neural network trained with the back propagation algorithm back propagation training is viewed as nonlinear least squares regression and the additive noise is interpreted as generating a kernel estimate of the probability density that describes the training vector distribution two specific application types are considered pattern classifier networks and estimation of a nonstochastic mapping from data corrupted by measurement errors it is not proved that the introduction of additive noise to the training vectors always improves network generalization however the analysis suggests mathematically justified rules for choosing the characteristics of noise if additive noise is used in training results of mathematical statistics are used to establish various asymptotic consistency results for the proposed method numerical simulations support the applicability of the training method

noise injection theoretical prospects noise injection consists of adding noise to the inputs during neural network training experimental results suggest that it might improve the generalization ability of the resulting neural network a justification of this improvement remains elusive describing analytically the average perturbed cost function is difficult and controlling the fluctuations of the random perturbed cost function is hard hence recent papers suggest replacing the random perturbed cost by a deterministic taylor approximation of the average perturbed cost function this article takes a different stance when the injected noise is gaussian noise injection is naturally connected to the action of the heat kernel this provides indications on the relevance domain of traditional taylor expansions and shows the dependence of the quality of taylor approximations on global smoothness properties of neural networks under consideration the connection between noise injection and heat kernel also enables controlling the fluctuations of

higher rank hyperbolicity the large scale geometry of hyperbolic metric spaces exhibits many distinctive features such as the stability of quasi geodesics the morse lemma the visibility property and the homeomorphism between visual boundaries induced by a quasi isometry we prove a number of closely analogous results for spaces of rank n ge 2 in an asymptotic sense under some weak assumptions reminiscent of nonpositive curvature for this purpose we replace quasi geodesic lines with quasi minimizing locally finite n cycles of r n volume growth prime examples include n cycles associated with n quasiflats solving an asymptotic plateau problem and producing unique tangent cones at infinity for such cycles we show in particular that every quasi isometry between two proper text cat 0 spaces of asymptotic rank n extends to a class of n 1 cycles in the tits boundaries

stacked denoising autoencoders learning useful representations in a deep network with a local denoising criterion we explore an original strategy for building deep networks based on stacking layers of denoising autoencoders which are trained locally to denoise corrupted versions of their inputs the resulting algorithm is a straightforward variation on the stacking of ordinary autoencoders it is however shown on a benchmark of classification problems to yield significantly lower classification error thus bridging the performance gap with deep belief networks dbn and in several cases surpassing it higher level representations learnt in this purely unsupervised fashion also help boost the performance of subsequent svm classifiers qualitative experiments show that contrary to ordinary autoencoders denoising autoencoders are able to learn gabor like edge detectors from natural image patches and larger stroke detectors from digit images this work clearly establishes the value of using a denoising criterion as a tractable unsupervised objective to guide the learning of useful higher level representations

training products of experts by minimizing contrastive divergence it is possible to combine multiple latent variable models of the same data by multiplying their probability distributions together and then renormalizing this way of combining individual expert models makes it hard to generate samples from the combined model but easy to infer the values of the latent variables of each expert because the combination rule ensures that the latent variables of different experts are conditionally independent when given the data a product of experts poe is therefore an interesting candidate for a perceptual system in which rapid inference is vital and generation is unnecessary training a poe by maximizing the likelihood of the data is difficult because it is hard even to approximate the derivatives of the renormalization term in the combination rule fortunately a poe can be trained using a different objective function called contrastive divergence whose derivatives with regard to the parameters can be approximated accurately and efficiently examples are presented of contrastive divergence learning using several types of expert on several types of data

essential laminations in 3 manifolds it is a well established principle in manifold topology that by studying codimension 1 objects and their complementary pieces one obtains a great deal of topological and geometric information about the manifold itself this is particularly true for 3 dimensional manifolds using incompressible surfaces a great many advances have been made during the last 30 years the most spectacular results were obtained by haken ha waldhausen w and thurston t1 more recently work t2 g14 using taut foliations in 3 manifolds has proven fruitful unfortunately in some sense most closed 3 manifolds do not contain incompressible surfaces and it is currently not known exactly which 3 manifolds have taut foliations the purpose of this paper is to study another codimension 1 object the essential lamination which is a generalization of the incompressible surface as well as the taut foliation we will show that the existence of an essential lamination in a 3 manifold m implies that m has useful properties similar to those of manifolds having either an incompressible surface or a taut foliation we will also show that the universal cover of a manifold containing an essential lamination is r3 precise definitions of terms used in the following theorem will be given in section 1 here we give only a rough idea we begin with two alternative approximate definitions of essential lamination i an essential lamination in a 3 manifold m is a lamination satisfying four conditions the inclusion of leaves of the lamination into m should induce an injection on stj the complement of the lamination should be irreducible no leaf should be a sphere and the lamination should be end incompressible to say that a lamination is end incompressible means roughly that a folded leaf can be straightened using an isotopy there are no infinite folds ii alternatively

robust transfer metric learning for image classification metric learning has attracted increasing attention due to its critical role in image analysis and classification conventional metric learning always assumes that the training and test data are sampled from the same or similar distribution however to build an effective distance metric we need abundant supervised knowledge i e side label information which is generally inaccessible in practice because of the expensive labeling cost in this paper we develop a robust transfer metric learning rtml framework to effectively assist the unlabeled target learning by transferring the knowledge from the well labeled source domain specifically rtml exploits knowledge transfer to mitigate the domain shift in two directions i e sample space and feature space in the sample space domain wise and class wise adaption schemes are adopted to bridge the gap of marginal and conditional distribution disparities across two domains in the feature space our metric is built in a marginalized denoising fashion and low rank constraint which make it more robust to tackle noisy data in reality furthermore we design an explicit rank constraint regularizer to replace the rank minimization np hard problem to guide the low rank metric learning experimental results on several standard benchmarks demonstrate the effectiveness of our proposed rtml by comparing it with the state of the art transfer learning and metric learning algorithms

quasi minimizing surfaces in hyperbolic space let h 3 g denote 3 dimensional hyperbolic space with constant curvature 1 we are interested in surfaces f in h 3 with the following quasi minimality property if it f h 3 0 1 which does not depend on the homotopy if this condition is satisfied with q 1 then f is called homotopically minimizing a surface f c h 3 with this property is not necessarily area minimizing in contrast to the situation in higher dimensions cf 13 our main result is

convolutional covariance features conception integration and performance in person re identification abstract this paper introduces a novel type of features based on covariance descriptors the convolutional covariance features ccf differently from the traditional and handcrafted way to obtain covariance descriptors ccf is computed from adaptive and trainable features which come from a coarse to fine transfer learning cfl strategy cfl provides a generic to specific knowledge and noise invariant information for person re identification after training the deep features convolutional and flat features are extracted from respectively intermediate and top layers of a hybrid deep network intermediate layer features are then wrapped in covariance matrices composing the so called ccf which are integrated to the top layer features called here flat features integration of ccf and flat features demonstrated to improve the proposed person re identification in comparison with the use of the component features alone our person re identification method achieved the best top 1 performance when compared with other 18 state of the art methods over viper i lids cuhk01 and cuhk03 data sets the compared methods are based on deep learning covariance descriptors or handcrafted features and similarity functions

geodesic invariant feature a local descriptor in depth different from the photometric images depth images resolve the distance ambiguity of the scene while the properties such as weak texture high noise and low resolution may limit the representation ability of the well developed descriptors which are elaborately designed for the photometric images in this paper a novel depth descriptor geodesic invariant feature gif is presented for representing the parts of the articulate objects in depth images gif is a multilevel feature representation framework which is proposed based on the nature of depth images low level geodesic gradient is introduced to obtain the invariance to the articulate motion such as scale and rotation variation midlevel superpixel clustering is applied to reduce depth image redundancy resulting in faster processing speed and better robustness to noise high level deep network is used to exploit the nonlinearity of the data which further improves the classification accuracy the proposed descriptor is capable of encoding the local structures in the depth data effectively and efficiently comparisons with the state of the art methods reveal the superiority of the proposed method

three dimensional manifolds kleinian groups and hyperbolic geometry 1 a conjectural picture of 3 manifolds a major thrust of mathematics in the late 19th century in which poincare had a large role was the uniformization theory for riemann surfaces that every conformai structure on a closed oriented surface is represented by a riemannian metric of constant curvature for the typical case of negative euler characteristic genus greater than 1 such a metric gives a hyperbolic structure any small neighborhood in the surface is isometric to a neighborhood in the hyperbolic plane and the surface itself is the quotient of the hyperbolic plane by a discrete group of motions the exceptional cases the sphere and the torus have spherical and euclidean structures three manifolds are greatly more complicated than surfaces and i think it is fair to say that until recently there was little reason to expect any analogous theory for manifolds of dimension 3 or more except perhaps for the fact that so many 3 manifolds are beautiful the situation has changed so that i feel fairly confident in proposing the

an information maximization approach to blind separation and blind deconvolution we derive a new self organizing learning algorithm that maximizes the information transferred in a network of nonlinear units the algorithm does not assume any knowledge of the input distributions and is defined here for the zero noise limit under these conditions information maximization has extra properties not found in the linear case linsker 1989 the nonlinearities in the transfer function are able to pick up higher order moments of the input distributions and perform something akin to true redundancy reduction between units in the output representation this enables the network to separate statistically independent components in the inputs a higher order generalization of principal components analysis we apply the network to the source separation or cocktail party problem successfully separating unknown mixtures of up to 10 speakers we also show that a variant on the network architecture is able to perform blind deconvolution cancellation of unknown echoes and reverberation in a speech signal finally we derive dependencies of information transfer on time delays we suggest that information maximization provides a unifying framework for problems in blind signal processing

clustering of cell populations in flow cytometry data using a combination of gaussian mixtures we propose a supervised learning approach to automatic quantification of cell populations in flow cytometric samples one sample contains up to millions of measurement vectors with a dimensionality between 10 and 20 normally each measurement vector corresponds to a single cell in the biological sample identifying biologically meaningful cell populations is essentially a clustering problem however standard clustering methods are impractical because size shape and location of corresponding clusters may vary strongly between samples mainly due to phenotypic differences and inter laboratory variations in our holistic approach we implicitly employ the structural information such as relative locations and shape of sub populations a new input sample is reconstructed by a linear combination of artificial reference samples each represented by a gaussian mixture model gmm in which for each gaussian component the class label of the corresponding cluster of observations is known the reference samples are calculated from a larger set of training samples by non negative matrix factorization and can be regarded as the basis of a lower dimensional feature space in which input samples are reconstructed we show a method for calculating the feature space transformation based on minimization the l2 distance defined between two gmm the feature space representation of the sample is then used to assign each observation to one of the specified sub populations by a bayes decision we present classification results on a database of about 170 patients with acute lymphoblastic leukemia all where high accuracy in the prediction of relatively small leukemic populations is crucial the approach is not limited to our application it can be employed wherever analysis of large multi dimensional numerical data of a specific class of samples with related structure has to be performed highlightsa density model based on the interpolation of gaussian mixture models is presented non negative matrix factorization is used to compress the model affine registration of gaussian mixture models using the l2 distance is performed the applicability of the method is demonstrated on flow cytometry data

complete minimal varieties in hyperbolic space it is a basic problem in the subject of minimal varieties to determine the class of complete minimal and area minimizing varieties m k in a given riemannian manifold n besides being of their own intrinsic interest the cases r of euclidean space and mn sn of the sphere are fundamental in understanding the local structure of minimal varieties due to the existence of tangent cones to general minimal varieties there has been much study of the complete areaminimizing varieties in these cases in particular relating to the bernstein conjecture and minimizing cones e g l2 f1 nevertheless many deep problems remain unsolved this paper is centered around the question of existence of complete areaminimizing varieties in hyperbolic space h here by way of contrast to the situation in ir n or s we find a comparatively complete solution to the problem of existence in fact the main result of this paper exhibits a wealth of complete area minimizing varieties in h in all dimensions and codimensions more precisely we have