tensor representation learning based image patch analysis for text identification and recognition in this paper we introduce a novel framework for text identification and recognition called tensor representation learning based image patch analysis trl ipa unlike most of previous text identification approaches which can only be applied to binarized images trl ipa can be directly applied to gray level and color images trl ipa is built on a general formulation of the convergent tensor representation learning ctrl algorithms in the implementation of trl ipa image patches are represented in the form of tensors while low dimensional representations of these tensors are learned via a ctrl algorithm to identify text regions in new coming document images a random forest classifier is trained in the learned tensor subspace moreover the trl ipa framework can be straightforwardly applied to recognition problems such as handwritten digits recognition we conducted extensive experiments on ancient chinese arabic and cyrillic document images to evaluate trl ipa on text identification tasks experimental results demonstrate its effectiveness and robustness in addition recognition results on images of handwritten digits show its advantage over state of the art vector and tensor representation based approaches highlightsa novel model tensor representation learning based image patch analysis trl ipa is proposed for document understanding trl ipa is built on a general formulation of the convergent tensor representation learning ctrl algorithms the ctrl algorithms are theoretically guaranteed to converge to a local optimal solution of the learning problem extensive experiments demonstrate the superiority of trl ipa over related vector and tensor representation based approaches

dictionary pair learning on grassmann manifolds for image denoising image denoising is a fundamental problem in computer vision and image processing that holds considerable practical importance for real world applications the traditional patch based and sparse coding driven image denoising methods convert 2d image patches into 1d vectors for further processing thus these methods inevitably break down the inherent 2d geometric structure of natural images to overcome this limitation pertaining to the previous image denoising methods we propose a 2d image denoising model namely the dictionary pair learning dpl model and we design a corresponding algorithm called the dpl on the grassmann manifold dplg algorithm the dplg algorithm first learns an initial dictionary pair i e the left and right dictionaries by employing a subspace partition technique on the grassmann manifold wherein the refined dictionary pair is obtained through a sub dictionary pair merging the dplg obtains a sparse representation by encoding each image patch only with the selected sub dictionary pair the non zero elements of the sparse representation are further smoothed by the graph laplacian operator to remove the noise consequently the dplg algorithm not only preserves the inherent 2d geometric structure of natural images but also performs manifold smoothing in the 2d sparse coding space we demonstrate that the dplg algorithm also improves the structural similarity values of the perceptual visual quality for denoised images using the experimental evaluations on the benchmark images and berkeley segmentation data sets moreover the dplg also produces the competitive peak signal to noise ratio values from popular image denoising algorithms

stdp compatible approximation of backpropagation in an energy based model we show that langevin markov chain monte carlo inference in an energy based model with latent variables has the property that the early steps of inference starting from a stationary point correspond to propagating error gradients into internal layers similar to backpropagation the backpropagated error is with respect to output units that have received an outside driving force pushing them away from the stationary point backpropagated error gradients correspond to temporal derivatives with respect to the activation of hidden units these lead to a weight update proportional to the product of the presynaptic firing rate and the temporal rate of change of the postsynaptic firing rate simulations and a theoretical argument suggest that this rate based update rule is consistent with those associated with spike timing dependent plasticity the ideas presented in this article could be an element of a theory for explaining how brains perform credit assignment in deep hierarchies as efficiently as backpropagation does with neural computation corresponding to both approximate inference in continuous valued latent variables and error backpropagation at the same time

scale space and edge detection using anisotropic diffusion a new definition of scale space is suggested and a class of algorithms used to realize a diffusion process is introduced the diffusion coefficient is chosen to vary spatially in such a way as to encourage intraregion smoothing rather than interregion smoothing it is shown that the no new maxima should be generated at coarse scales property of conventional scale space is preserved as the region boundaries in the approach remain sharp a high quality edge detector which successfully exploits global information is obtained experimental results are shown on a number of images parallel hardware implementations are made feasible because the algorithm involves elementary local operations replicated over the image

the second main theorem for holomorphic curves into semi abelian varieties we establish the second main theorem with the best truncation level one tf r l d n1 r f d tf r for an entire holomorphic curve f c a into a semi abelian variety a and an arbitrary effective reduced divisor d on a the low truncation level is important for applications we will actually prove this for the jet lifts of f finally we give some applications including the solution of a problem posed by mark green

large margin low rank tensor analysis we present a supervised model for tensor dimensionality reduction which is called large margin low rank tensor analysis lmlrta in contrast to traditional vector representation based dimensionality reduction methods lmlrta can take any order of tensors as input and unlike previous tensor dimensionality reduction methods which can learn only the low dimensional embeddings with a priori specified dimensionality lmlrta can automatically and jointly learn the dimensionality and the low dimensional representations from data moreover lmlrta delivers low rank projection matrices while it encourages data of the same class to be close and of different classes to be separated by a large margin of distance in the low dimensional tensor space lmlrta can be optimized using an iterative fixed point continuation algorithm which is guaranteed to converge to a local optimal solution of the optimization problem we evaluate lmlrta on an object recognition application where the data are represented as 2d tensors and a face recognition application where the data are represented as 3d tensors experimental results show the superiority of lmlrta over state of the art approaches

justifying and generalizing contrastive divergence we study an expansion of the log likelihood in undirected graphical models such as the restricted boltzmann machine rbm where each term in the expansion is associated with a sample in a gibbs chain alternating between two random variables the visible vector and the hidden vector in rbms we are particularly interested in estimators of the gradient of the log likelihood obtained through this expansion we show that its residual term converges to zero justifying the use of a truncation running only a short gibbs chain which is the main idea behind the contrastive divergence cd estimator of the log likelihood gradient by truncating even more we obtain a stochastic reconstruction error related through a mean field approximation to the reconstruction error often used to train autoassociators and stacked autoassociators the derivation is not specific to the particular parametric forms used in rbms and requires only convergence of the gibbs chain we present theoretical and empirical evidence linking the number of gibbs steps k and the magnitude of the rbm parameters to the bias in the cd estimator these experiments also suggest that the sign of the cd estimator is correct most of the time even when the bias is large so that cd k is a good descent direction even for small k

graph regularized restricted boltzmann machine the restricted boltzmann machine rbm has received an increasing amount of interest in recent years it determines good mapping weights that capture useful latent features in an unsupervised manner the rbm and its generalizations have been successfully applied to a variety of image classification and speech recognition tasks however most of the existing rbm based models disregard the preservation of the data manifold structure in many real applications the data generally reside on a low dimensional manifold embedded in high dimensional ambient space in this brief we propose a novel graph regularized rbm to capture features and learning representations explicitly considering the local manifold structure of the data by imposing manifold based locality that preserves constraints on the hidden layer of the rbm the model ultimately learns sparse and discriminative representations the representations can reflect data distributions while simultaneously preserving the local manifold structure of data we test our model using several benchmark image data sets for unsupervised clustering and supervised classification problem the results demonstrate that the performance of our method exceeds the state of the art alternatives

the effects of adding noise during backpropagation training on a generalization performance we study the effects of adding noise to the inputs outputs weight connections and weight changes of multilayer feedforward neural networks during backpropagation training we rigorously derive and analyze the objective functions that are minimized by the noise affected training processes we show that input noise and weight noise encourage the neural network output to be a smooth function of the input or its weights respectively in the weak noise limit noise added to the output of the neural networks only changes the objective function by a constant hence it cannot improve generalization input noise introduces penalty terms in the objective function that are related to but distinct from those found in the regularization approaches simulations have been performed on a regression and a classification problem to further substantiate our analysis input noise is found to be effective in improving the generalization performance for both problems however weight noise is found to be effective in improving the generalization performance only for the classification problem other forms of noise have practically no effect on generalization

cycles scaling and crossover phenomenon in length of the day lod time series abstract the dynamics of the temporal fluctuations of the length of the day lod time series from january 1 1962 to november 2 2006 were investigated the power spectrum of the whole time series has revealed annual semi annual decadal and daily oscillatory behaviors correlated with oceanic atmospheric processes and interactions the scaling behavior was analyzed by using the detrended fluctuation analysis dfa which has revealed two different scaling regimes separated by a crossover timescale at approximately 23 days flicker noise process can describe the dynamics of the lod time regime involving intermediate and long timescales while brownian dynamics characterizes the lod time series for small timescales

long range dependence in tree ring width time series of austrocedrus chilensis revealed by means of the detrended fluctuation analysis by using the detrended fluctuation analysis and detrended moving average method 823 time series of tree ring widths in austrocedrus chilensis in patagonia were analyzed the tree ring widths of a chilensis have been widely used for climatological studies the results point out to the presence of significant scaling in the temporal fluctuations of tree ring which is not due to singular probability density function of the widths but due to the presence of long range correlations such results are in good agreement with those concerning the evidence of long range dependencies in weather time series

learning consensus representation for weak style classification style classification e g baroque and gothic architecture style is grabbing increasing attention in many fields such as fashion architecture and manga most existing methods focus on extracting discriminative features from local patches or patterns however the spread out phenomenon in style classification has not been recognized yet it means that visually less representative images in a style class are usually very diverse and easily getting misclassified we name them weak style images another issue when employing multiple visual features towards effective weak style classification is lack of consensus among different features that is weights for different visual features in the local patch should have been allocated similar values to address these issues we propose a consensus style centralizing auto encoder cscae for learning robust style features representation especially for weak style classification first we propose a style centralizing auto encoder scae which centralizes weak style features in a progressive way then based on scae we propose both the non linear and linear version cscae which adaptively allocate weights for different features during the progressive centralization process consensus constraints are added based on the assumption that the weights of different features of the same patch should be similar specifically the proposed linear counterpart of cscae motivated by the shared weights idea as well as group sparsity improves both efficacy and efficiency for evaluations we experiment extensively on fashion manga and architecture style classification problems in addition we collect a new dataset online shopping for fashion style classification which will be publicly available for vision based fashion style research experiments demonstrate the effectiveness of the scae and cscae on both public and newly collected datasets when compared with the most recent state of the art works

addendum to defects for ample divisors of abelian varieties schwarz lemma and hyperbolic hypersurfaces of low degrees american journal of mathematics 119 1997 1139 1172 this addendum uses bloch s original technique of ordinary differential equations to circumvent a difficulty in the proof of lemma 2 in the paper in the title this technique at the same time yields the following stronger nevanlinna s second main theorem with truncation at an order given explicitly by the chern class of the divisor for an ample divisor d in an abelian variety a of complex dimension n and for any holomorphic map f c a whose image is not contained in any translate of d the characteristic function of f for d is dominated by the counting function of f for d truncated at order k n plus an error term of logarithmic order of the characteristic function where k n is inductively given by k 0 0 k 1 1 and k l 1 k l 3 n l 1 4 k l 1 l d n for 1 l n

dual autoencoders features for imbalance classification problem many classification problems encountered in real world applications exhibit a profile of imbalanced data current methods depend on data resampling in fact if the feature set provides a clear decision boundary resampling may not be needed to solve the imbalanced classification problem therefore this work proposes a feature learning method based on the autoencoder to learn a set of features with better classification capabilities of the minority and the majority classes to address the imbalanced classification problems two sets of features are learned by two stacked autoencoders with different activation functions to capture different characteristics of the data and they are combined to form the dual autoencoding features samples are then classified in the new feature space learned in this manner instead of the original input space experimental results show that the proposed method outperforms current resampling based methods with statistical significance for imbalanced pattern classification problems highlightsto our best knowledge this paper proposes the first feature learning based method for dealing with imbalance pattern classification using stacked autoencoders we provide a new viewpoint for solving imbalanced pattern classification problems by projecting the input space onto a learned feature space with better representation using stacked autoencoders the concatenation of learned features from two different stacked autoencoders further enhances the classification accuracy by providing a better representation with different characteristics of the data to the classifier to learn more specifically the sigmoid activation function is less sensitive to input changes and provides robust representation while the tanh activation function is more sensitive and provides detailed information of the data experimental results show that the daf yields statistical significant improvement in the auc the f1 score and the g mean in comparison to representative methods of resampling based and feature projection methods for dealing with the imbalanced pattern classification problems

graph regularized auto encoders for image representation image representation has been intensively explored in the domain of computer vision for its significant influence on the relative tasks such as image clustering and classification it is valuable to learn a low dimensional representation of an image which preserves its inherent information from the original image space at the perspective of manifold learning this is implemented with the local invariant idea to capture the intrinsic low dimensional manifold embedded in the high dimensional input space inspired by the recent successes of deep architectures we propose a local invariant deep nonlinear mapping algorithm called graph regularized auto encoder gae with the graph regularization the proposed method preserves the local connectivity from the original image space to the representation space while the stacked auto encoders provide explicit encoding model for fast inference and powerful expressive capacity for complex modeling theoretical analysis shows that the graph regularizer penalizes the weighted frobenius norm of the jacobian matrix of the encoder mapping where the weight matrix captures the local property in the input space furthermore the underlying effects on the hidden representation space are revealed providing insightful explanation to the advantage of the proposed method finally the experimental results on both clustering and classification tasks demonstrate the effectiveness of our gae as well as the correctness of the proposed theoretical analysis and it also suggests that gae is a superior solution to the current deep representation learning techniques comparing with variant auto encoders and existing local invariant methods

locator checker scaler object tracking using spatially ordered and weighted patch descriptor in this paper we propose a simple yet effective object descriptor and a novel tracking algorithm to track a target object accurately for the object description we divide the bounding box of a target object into multiple patches and describe them with color and gradient histograms then we determine the foreground weight of each patch to alleviate the impacts of background information in the bounding box to this end we perform random walk with restart rwr simulation we then concatenate the weighted patch descriptors to yield the spatially ordered and weighted patch sowp descriptor for the object tracking we incorporate the proposed sowp descriptor into a novel tracking algorithm which has three components locator checker and scaler lcs the locator and the scaler estimate the center location and the size of a target respectively the checker determines whether it is safe to adjust the target scale in a current frame these three components cooperate with one another to achieve robust tracking experimental results demonstrate that the proposed lcs tracker achieves excellent performance on recent benchmarks

a fuzzy restricted boltzmann machine novel learning algorithms based on the crisp possibilistic mean value of fuzzy numbers a fuzzy restricted boltzmann machine frbm is extended from a restricted boltzmann machine rbm by replacing all the real valued parameters with fuzzy numbers a new frbm that employs the crisp possibilistic mean value of a fuzzy number to defuzzify the fuzzy free energy function is presented this approach is much clearer and easier to obtain the expression of the defuzzified free energy function and its approximation than the centroid method several theorems that discuss the error bounds of the approximation to ensure the rationality and validity are also investigated learning algorithms are given for the designed frbm with symmetric triangular fuzzy numbers stfns asymmetric triangular fuzzy numbers and gaussian fuzzy numbers by appropriately choosing the parameters a theorem is concluded that all frbms with symmetric fuzzy numbers will have identical learning algorithm to that of frbms with stfns this is illustrated by a case of frbm with gaussian fuzzy numbers two experiments including the mnist handwriting recognition and the bars and stripes benchmark are carried out the results show that the proposed frbms significantly outperform rbms in learning accuracy and generalization ability especially when encountering unlearned samples and recovering incomplete images

probabilistic low rank multitask learning in this paper we consider the problem of learning multiple related tasks simultaneously with the goal of improving the generalization performance of individual tasks the key challenge is to effectively exploit the shared information across multiple tasks as well as preserve the discriminative information for each individual task to address this we propose a novel probabilistic model for multitask learning mtl that can automatically balance between low rank and sparsity constraints the former assumes a low rank structure of the underlying predictive hypothesis space to explicitly capture the relationship of different tasks and the latter learns the incoherent sparse patterns private to each task we derive and perform inference via variational bayesian methods experimental results on both regression and classification tasks on real world applications demonstrate the effectiveness of the proposed method in dealing with the mtl problems

subspace clustering using a low rank constrained autoencoder the performance of subspace clustering is affected by data representation data representation for subspace clustering maps data from the original space into another space with the property of better separability many data representation methods have been developed in recent years typical among them are low rank representation lrr and an autoencoder lrr is a linear representation method that captures the global structure of data with low rank constraint alternatively an autoencoder nonlinearly maps data into a latent space using a neural network by minimizing the difference between the reconstruction and input to combine the advantages of an lrr globality and autoencoder self supervision based locality we propose a novel data representation method for subspace clustering the proposed method called low rank constrained autoencoder lrae forces the latent representation of the neural network to be of low rank and the low rank constraint is computed as a prior from the input space one major advantage of the lrae is that the learned data representation not only maintains the local features of the data but also preserves the underlying low rank global structure extensive experiments on several datasets for subspace clustering were conducted they demonstrated that the proposed lrae substantially outperformed state of the art subspace clustering methods

use of a tracer specific deep artificial neural net to denoise dynamic pet images application of kinetic modeling km on a voxel level in dynamic pet images frequently suffers from high levels of noise drastically reducing the precision of parametric image analysis in this paper we investigate the use of machine learning and artificial neural networks to denoise dynamic pet images we train a deep denoising autoencoder dae using noisy and noise free spatiotemporal image patches extracted from the simulated images of 11c raclopride a dopamine d2 receptor agonist the dae processed dynamic and corresponding parametric images simulated and acquired are compared with those obtained with conventional denoising techniques including temporal and spatial gaussian smoothing iterative spatiotemporal smoothing deconvolution and the highly constrained backprojection processing hypr the simulated acquired parametric image non uniformity was 7 75 19 49 with temporal and 5 90 14 50 with spatial smoothing 5 82 16 21 with smoothing deconvolution 5 49 13 38 with hypr and 3 52 11 41 with dae the dae also produced the best results in terms of the coefficient of variation of voxel values and structural similarity index denoising induced bias in the regional mean binding potential was 7 8 with temporal and 26 31 with spatial smoothing 28 61 with smoothing deconvolution 27 63 with hypr and 14 8 with dae when the test data did not match the training data erroneous outcomes were obtained our results demonstrate that a deep dae can provide a substantial reduction in the voxel level noise compared with the conventional spatiotemporal denoising methods while introducing a similar or lower amount of bias the better dae performance comes at the cost of lower generality and requiring appropriate training data

a fast learning algorithm for deep belief nets we show how to use complementary priors to eliminate the explaining away effects that make inference difficult in densely connected belief nets that have many hidden layers using complementary priors we derive a fast greedy algorithm that can learn deep directed belief networks one layer at a time provided the top two layers form an undirected associative memory the fast greedy algorithm is used to initialize a slower learning procedure that fine tunes the weights using a contrastive version of the wake sleep algorithm after fine tuning a network with three hidden layers forms a very good generative model of the joint distribution of handwritten digit images and their labels this generative model gives better digit classification than the best discriminative learning algorithms the low dimensional manifolds on which the digits lie are modeled by long ravines in the free energy landscape of the top level associative memory and it is easy to explore these ravines by using the directed connections to display what the associative memory has in mind

training with noise is equivalent to tikhonov regularization it is well known that the addition of noise to the input data of a neural network during training can in some circumstances lead to significant improvements in generalization performance previous work has shown that such training with noise is equivalent to a form of regularization in which an extra term is added to the error function however the regularization term which involves second derivatives of the error function is not bounded below and so can lead to difficulties if used directly in a learning algorithm based on error minimization in this paper we show that for the purposes of network training the regularization term can be reduced to a positive semi definite form that involves only first derivatives of the network mapping for a sum of squares error function the regularization term belongs to the class of generalized tikhonov regularizers direct minimization of the regularized error function provides a practical alternative to training with noise

nonlinear autoassociation is not equivalent to pca a common misperception within the neural network community is that even with nonlinearities in their hidden layer autoassociators trained with backpropagation are equivalent to linear methods such as principal component analysis pca our purpose is to demonstrate that nonlinear autoassociators actually behave differently from linear methods and that they can outperform these methods when used for latent extraction projection and classification while linear autoassociators emulate pca and thus exhibit a flat or unimodal reconstruction error surface autoassociators with nonlinearities in their hidden layer learn domains by building error reconstruction surfaces that depending on the task contain multiple local valleys this interpolation bias allows nonlinear autoassociators to represent appropriate classifications of nonlinear multimodal domains in contrast to linear autoassociators which are inappropriate for such tasks in fact autoassociators with hidden unit nonlinearities can be shown to perform nonlinear classification and nonlinear recognition

dimensionality reduction using similarity induced embeddings the vast majority of dimensionality reduction dr techniques rely on the second order statistics to define their optimization objective even though this provides adequate results in most cases it comes with several shortcomings the methods require carefully designed regularizers and they are usually prone to outliers in this paper a new dr framework that can directly model the target distribution using the notion of similarity instead of distance is introduced the proposed framework called similarity embedding framework sef can overcome the aforementioned limitations and provides a conceptually simpler way to express optimization targets similar to existing dr techniques deriving a new dr technique using the sef becomes simply a matter of choosing an appropriate target similarity matrix a variety of classical tasks such as performing supervised dr and providing out of sample extensions as well as new novel techniques such as providing fast linear embeddings for complex techniques are demonstrated in this paper using the proposed framework six data sets from a diverse range of domains are used to evaluate the proposed method and it is demonstrated that it can outperform many existing dr techniques

defects for ample divisors of abelian varieties schwarz lemma and hyperbolic hypersurfaces of low degrees we prove that the defect vanishes for a holomorphic map f from the affine complex line to an abelian variety a and for an ample divisor d in a the proof uses the translational invariance of the zariski closure of the k jet space of the image of f and the theorem of riemann roch to construct a nonidentically zero meromorphic k jet differential whose pole divisor is dominated by a divisor equivalent to pd and which vanishes along the k jet space of d to order q with p q smaller than a prescribed small positive number then estimates involving the theta function with divisor d and the logarithmic derivative lemma are used we also prove a pointwise schwarz lemma which gives the vanishing of the pullback by a holomorphic map from the affine complex line to a compact complex manifold of a holomorphic jet differential vanishing on an ample divisor this pointwise schwarz lemma is a slight modification of a statement whose proof green and griffiths sketched in their alternative treatment of bloch s theorem on entire curves in abelian varieties the log pole case of the pointwise schwarz lemma is also given we construct examples of hyperbolic hypersurface whose degree is only 16 times the square of its dimension

using additive noise in back propagation training the possibility of improving the generalization capability of a neural network by introducing additive noise to the training samples is discussed the network considered is a feedforward layered neural network trained with the back propagation algorithm back propagation training is viewed as nonlinear least squares regression and the additive noise is interpreted as generating a kernel estimate of the probability density that describes the training vector distribution two specific application types are considered pattern classifier networks and estimation of a nonstochastic mapping from data corrupted by measurement errors it is not proved that the introduction of additive noise to the training vectors always improves network generalization however the analysis suggests mathematically justified rules for choosing the characteristics of noise if additive noise is used in training results of mathematical statistics are used to establish various asymptotic consistency results for the proposed method numerical simulations support the applicability of the training method

noise injection theoretical prospects noise injection consists of adding noise to the inputs during neural network training experimental results suggest that it might improve the generalization ability of the resulting neural network a justification of this improvement remains elusive describing analytically the average perturbed cost function is difficult and controlling the fluctuations of the random perturbed cost function is hard hence recent papers suggest replacing the random perturbed cost by a deterministic taylor approximation of the average perturbed cost function this article takes a different stance when the injected noise is gaussian noise injection is naturally connected to the action of the heat kernel this provides indications on the relevance domain of traditional taylor expansions and shows the dependence of the quality of taylor approximations on global smoothness properties of neural networks under consideration the connection between noise injection and heat kernel also enables controlling the fluctuations of

deep convolutional neural network for inverse problems in imaging in this paper we propose a novel deep convolutional neural network cnn based algorithm for solving ill posed inverse problems regularized iterative algorithms have emerged as the standard approach to ill posed inverse problems in the past few decades these methods produce excellent results but can be challenging to deploy in practice due to factors including the high computational cost of the forward and adjoint operators and the difficulty of hyperparameter selection the starting point of this paper is the observation that unrolled iterative methods have the form of a cnn filtering followed by pointwise non linearity when the normal operator h h where h is the adjoint of the forward imaging operator h of the forward model is a convolution based on this observation we propose using direct inversion followed by a cnn to solve normal convolutional inverse problems the direct inversion encapsulates the physical model of the system but leads to artifacts when the problem is ill posed the cnn combines multiresolution decomposition and residual learning in order to learn to remove these artifacts while preserving image structure we demonstrate the performance of the proposed network in sparse view reconstruction down to 50 views on parallel beam x ray computed tomography in synthetic phantoms as well as in real experimental sinograms the proposed network outperforms total variation regularized iterative reconstruction for the more realistic phantoms and requires less than a second to reconstruct a 512 times 512 image on the gpu

stacked denoising autoencoders learning useful representations in a deep network with a local denoising criterion we explore an original strategy for building deep networks based on stacking layers of denoising autoencoders which are trained locally to denoise corrupted versions of their inputs the resulting algorithm is a straightforward variation on the stacking of ordinary autoencoders it is however shown on a benchmark of classification problems to yield significantly lower classification error thus bridging the performance gap with deep belief networks dbn and in several cases surpassing it higher level representations learnt in this purely unsupervised fashion also help boost the performance of subsequent svm classifiers qualitative experiments show that contrary to ordinary autoencoders denoising autoencoders are able to learn gabor like edge detectors from natural image patches and larger stroke detectors from digit images this work clearly establishes the value of using a denoising criterion as a tractable unsupervised objective to guide the learning of useful higher level representations

training products of experts by minimizing contrastive divergence it is possible to combine multiple latent variable models of the same data by multiplying their probability distributions together and then renormalizing this way of combining individual expert models makes it hard to generate samples from the combined model but easy to infer the values of the latent variables of each expert because the combination rule ensures that the latent variables of different experts are conditionally independent when given the data a product of experts poe is therefore an interesting candidate for a perceptual system in which rapid inference is vital and generation is unnecessary training a poe by maximizing the likelihood of the data is difficult because it is hard even to approximate the derivatives of the renormalization term in the combination rule fortunately a poe can be trained using a different objective function called contrastive divergence whose derivatives with regard to the parameters can be approximated accurately and efficiently examples are presented of contrastive divergence learning using several types of expert on several types of data

detecting long range correlations with detrended fluctuation analysis abstract we examine the detrended fluctuation analysis dfa which is a well established method for the detection of long range correlations in time series we show that deviations from scaling which appear at small time scales become stronger in higher orders of dfa and suggest a modified dfa method to remove them the improvement is necessary especially for short records that are affected by non stationarities furthermore we describe how crossovers in the correlation behavior can be detected reliably and determined quantitatively and show how several types of trends in the data affect the different orders of dfa

robust transfer metric learning for image classification metric learning has attracted increasing attention due to its critical role in image analysis and classification conventional metric learning always assumes that the training and test data are sampled from the same or similar distribution however to build an effective distance metric we need abundant supervised knowledge i e side label information which is generally inaccessible in practice because of the expensive labeling cost in this paper we develop a robust transfer metric learning rtml framework to effectively assist the unlabeled target learning by transferring the knowledge from the well labeled source domain specifically rtml exploits knowledge transfer to mitigate the domain shift in two directions i e sample space and feature space in the sample space domain wise and class wise adaption schemes are adopted to bridge the gap of marginal and conditional distribution disparities across two domains in the feature space our metric is built in a marginalized denoising fashion and low rank constraint which make it more robust to tackle noisy data in reality furthermore we design an explicit rank constraint regularizer to replace the rank minimization np hard problem to guide the low rank metric learning experimental results on several standard benchmarks demonstrate the effectiveness of our proposed rtml by comparing it with the state of the art transfer learning and metric learning algorithms

holomorphic flexibility properties of complex manifolds we obtain results on approximation of holomorphic maps by algebraic maps the jet transversality theorem for holomorphic and algebraic maps between certain classes of manifolds and the homotopy principle for holomorphic submersions of stein manifolds to certain algebraic manifolds

an information maximization approach to blind separation and blind deconvolution we derive a new self organizing learning algorithm that maximizes the information transferred in a network of nonlinear units the algorithm does not assume any knowledge of the input distributions and is defined here for the zero noise limit under these conditions information maximization has extra properties not found in the linear case linsker 1989 the nonlinearities in the transfer function are able to pick up higher order moments of the input distributions and perform something akin to true redundancy reduction between units in the output representation this enables the network to separate statistically independent components in the inputs a higher order generalization of principal components analysis we apply the network to the source separation or cocktail party problem successfully separating unknown mixtures of up to 10 speakers we also show that a variant on the network architecture is able to perform blind deconvolution cancellation of unknown echoes and reverberation in a speech signal finally we derive dependencies of information transfer on time delays we suggest that information maximization provides a unifying framework for problems in blind signal processing

clustering of cell populations in flow cytometry data using a combination of gaussian mixtures we propose a supervised learning approach to automatic quantification of cell populations in flow cytometric samples one sample contains up to millions of measurement vectors with a dimensionality between 10 and 20 normally each measurement vector corresponds to a single cell in the biological sample identifying biologically meaningful cell populations is essentially a clustering problem however standard clustering methods are impractical because size shape and location of corresponding clusters may vary strongly between samples mainly due to phenotypic differences and inter laboratory variations in our holistic approach we implicitly employ the structural information such as relative locations and shape of sub populations a new input sample is reconstructed by a linear combination of artificial reference samples each represented by a gaussian mixture model gmm in which for each gaussian component the class label of the corresponding cluster of observations is known the reference samples are calculated from a larger set of training samples by non negative matrix factorization and can be regarded as the basis of a lower dimensional feature space in which input samples are reconstructed we show a method for calculating the feature space transformation based on minimization the l2 distance defined between two gmm the feature space representation of the sample is then used to assign each observation to one of the specified sub populations by a bayes decision we present classification results on a database of about 170 patients with acute lymphoblastic leukemia all where high accuracy in the prediction of relatively small leukemic populations is crucial the approach is not limited to our application it can be employed wherever analysis of large multi dimensional numerical data of a specific class of samples with related structure has to be performed highlightsa density model based on the interpolation of gaussian mixture models is presented non negative matrix factorization is used to compress the model affine registration of gaussian mixture models using the l2 distance is performed the applicability of the method is demonstrated on flow cytometry data

a connection between score matching and denoising autoencoders denoising autoencoders have been previously shown to be competitive alternatives to restricted boltzmann machines for unsupervised pretraining of each layer of a deep architecture we show that a simple denoising autoencoder training criterion is equivalent to matching the score with respect to the data of a specific energy based model to that of a nonparametric parzen density estimator of the data this yields several useful insights it defines a proper probabilistic model for the denoising autoencoder technique which makes it in principle possible to sample from them or rank examples by their energy it suggests a different way to apply score matching that is related to learning to denoise and does not require computing second derivatives it justifies the use of tied weights between the encoder and decoder and suggests ways to extend the success of denoising autoencoders to a larger family of energy based models