design of capacity approaching irregular low density parity check codes we design low density parity check ldpc codes that perform at rates extremely close to the shannon capacity the codes are built from highly irregular bipartite graphs with carefully chosen degree patterns on both sides our theoretical analysis of the codes is based on the work of richardson and urbanke see ibid vol 47 no 2 p 599 618 2000 assuming that the underlying communication channel is symmetric we prove that the probability densities at the message nodes of the graph possess a certain symmetry using this symmetry property we then show that under the assumption of no cycles the message densities always converge as the number of iterations tends to infinity furthermore we prove a stability condition which implies an upper bound on the fraction of errors that a belief propagation decoder can correct when applied to a code induced from a bipartite graph with a given degree distribution our codes are found by optimizing the degree structure of the underlying graphs we develop several strategies to perform this optimization we also present some simulation results for the codes found which show that the performance of the codes is very close to the asymptotic theoretical bounds

hierarchical multi label classification using fully associative ensemble learning developing a local hierarchical ensemble framework for hierarchical multi label classification hmc in which all the structural relationships in the class hierarchy are used to obtain global prediction introducing empirical loss minimization into hmc so that the learned model can capture the most useful information from historical data proposing sparse kernel and binary constraint hmc models traditional flat classification methods e g binary or multi class classification neglect the structural information between different classes in contrast hierarchical multi label classification hmc considers the structural information embedded in the class hierarchy and uses it to improve classification performance in this paper we propose a local hierarchical ensemble framework for hmc fully associative ensemble learning fael we model the relationship between each class nodes global prediction and the local predictions of all the class nodes as a multi variable regression problem with frobenius norm or l1 norm regularization it can be extended using the kernel trick which explores the complex correlation between global and local prediction in addition we introduce a binary constraint model to restrict the optimal weight matrix learning the proposed models have been applied to image annotation and gene function prediction datasets with tree structured class hierarchy and large scale visual recognition dataset with direct acyclic graph dag structured class hierarchy the experimental results indicate that our models achieve better performance when compared with other baseline methods

connected component model for multi object tracking in multi object tracking it is critical to explore the data associations by exploiting the temporal information from a sequence of frames rather than the information from the adjacent two frames since straightforwardly obtaining data associations from multi frames is an np hard multi dimensional assignment mda problem most existing methods solve this mda problem by either developing complicated approximate algorithms or simplifying mda as a 2d assignment problem based upon the information extracted only from adjacent frames in this paper we show that the relation between associations of two observations is the equivalence relation in the data association problem based on the spatial temporal constraint that the trajectories of different objects must be disjoint therefore the mda problem can be equivalently divided into independent subproblems by equivalence partitioning in contrast to existing works for solving the mda problem we develop a connected component model ccm by exploiting the constraints of the data association and the equivalence relation on the constraints based upon ccm we can efficiently obtain the global solution of the mda problem for multi object tracking by optimizing a sequence of independent data association subproblems experiments on challenging public data sets demonstrate that our algorithm outperforms the state of the art approaches

a confidence limit for the empirical mode decomposition and hilbert spectral analysis the confidence limit is a standard measure of the accuracy of the result in any statistical analysis most of the confidence limits are derived as follows the data are first divided into subsections and then under the ergodic assumption the temporal mean is substituted for the ensemble mean next the confidence limit is defined as a range of standard deviations from this mean however such a confidence limit is valid only for linear and stationary processes furthermore in order for the ergodic assumption to be valid the subsections have to be statistically independent for non stationary and nonlinear processes such an analysis is no longer valid the confidence limit of the method here termed emd hsa for empirical mode decomposition hilbert spectral analysis is introduced by using various adjustable stopping criteria in the sifting processes of the emd step to generate a sample set of intrinsic mode functions imfs the emd technique acts as a pre processor for hsa on the original data producing a set of components imfs from the original data that equal the original data when added back together each imf represents a scale in the data from smallest to largest the ensemble mean and standard deviation of the imf sample sets obtained with different stopping criteria are calculated and these form a simple random sample set the confidence limit for emd hsa is then defined as a range of standard deviations from the ensemble mean without evoking the ergodic assumption subdivision of the data stream into short sections is unnecessary hence the results and the confidence limit retain the full frequency resolution of the full dataset this new confidence limit can be applied to the analysis of nonlinear and non stationary processes by these new techniques data from length of day measurements and a particularly violent recent earthquake are used to demonstrate how the confidence limit is obtained and applied by providing a confidence limit for this new approach a stable range of stopping criteria for the decomposition or sifting phase emd has been established making the results of the final processing with hsa and the entire emd hsa method more definitive

local rademacher complexity for multi label learning we analyze the local rademacher complexity of empirical risk minimization based multi label learning algorithms and in doing so propose a new algorithm for multi label learning rather than using the trace norm to regularize the multi label predictor we instead minimize the tail sum of the singular values of the predictor in multi label learning benefiting from the use of the local rademacher complexity our algorithm therefore has a sharper generalization error bound compared with methods that minimize over all singular values concentrating on the tail singular values results in better recovery of the low rank structure of the multi label predictor which plays an important role in exploiting label correlations we propose a new conditional singular value thresholding algorithm to solve the resulting objective function moreover a variance control strategy is employed to reduce the variance of variables in optimization empirical studies on real world data sets validate our theoretical results and demonstrate the effectiveness of the proposed algorithm for multi label learning

multilabel classification with principal label space transformation we consider a hypercube view to perceive the label space of multilabel classification problems geometrically the view allows us not only to unify many existing multilabel classification approaches but also design a novel algorithm principal label space transformation plst that captures key correlations between labels before learning the simple and efficient plst relies on only singular value decomposition as the key step we derive the theoretical guarantee of plst and evaluate its empirical performance using real world data sets experimental results demonstrate that plst is faster than the traditional binary relevance approach and is superior to the modern compressive sensing approach in terms of both accuracy and efficiency

design of serially concatenated systems depending on the block length based on extrinsic information transfer exit charts the convergence behavior of iterative decoding is studied for a number of serially concatenated systems such as a serially concatenated code coded data transmission over an intersymbol interference channel bit interleaved coded modulation or trellis coded modulation efficient optimization algorithms based on simplified exit chart construction are devised to find irregular codes improving the convergence of iterative decoding one optimization criterion is to find concatenated systems exhibiting thresholds of successful decoding convergence which are close to information theoretic limits however these thresholds are approached only for very long block lengths to overcome this problem the decoding convergence after a fixed finite number of iterations is optimized which yields systems performing very well for short block lengths too as an example optimal system configurations for communication over an additive white gaussian noise channel are presented

extrinsic information transfer functions model and erasure channel properties extrinsic information transfer exit charts are a tool for predicting the convergence behavior of iterative processors for a variety of communication problems a model is introduced that applies to decoding problems including the iterative decoding of parallel concatenated turbo codes serially concatenated codes low density parity check ldpc codes and repeat accumulate ra codes exit functions are defined using the model and several properties of such functions are proved for erasure channels one property expresses the area under an exit function in terms of a conditional entropy a useful consequence of this result is that the design of capacity approaching codes reduces to a curve fitting problem for all the aforementioned codes a second property relates the exit function of a code to its helleseth klove levenshtein information functions and thereby to the support weights of its subcodes the relation is via a refinement of information functions called split information functions and via a refinement of support weights called split support weights split information functions are used to prove a third property that relates the exit function of a linear code to the exit function of its dual

a fast optimization method for general binary code learning hashing or binary code learning has been recognized to accomplish efficient near neighbor search and has thus attracted broad interests in recent retrieval vision and learning studies one main challenge of learning to hash arises from the involvement of discrete variables in binary code optimization while the widely used continuous relaxation may achieve high learning efficiency the pursued codes are typically less effective due to accumulated quantization error in this paper we propose a novel binary code optimization method dubbed discrete proximal linearized minimization dplm which directly handles the discrete constraints during the learning process specifically the discrete thus nonsmooth nonconvex problem is reformulated as minimizing the sum of a smooth loss term with a nonsmooth indicator function the obtained problem is then efficiently solved by an iterative procedure with each iteration admitting an analytical discrete solution which is thus shown to converge very fast in addition the proposed method supports a large family of empirical loss functions which is particularly instantiated in this paper by both a supervised and an unsupervised hashing losses together with the bits uncorrelation and balance constraints in particular the proposed dplm with a supervised ell 2 loss encodes the whole nus wide database into 64 b binary codes within 10 s on a standard desktop computer the proposed approach is extensively evaluated on several large scale data sets and the generated binary codes are shown to achieve very promising results on both retrieval and classification tasks

on the consistency of multi label learning multi label learning has attracted much attention during the past few years many multi label approaches have been developed mostly working with surrogate loss functions because multi label loss functions are usually difficult to optimize directly owing to their non convexity and discontinuity these approaches are effective empirically however little effort has been devoted to the understanding of their consistency i e the convergence of the risk of learned functions to the bayes risk in this paper we present a theoretical analysis on this important issue we first prove a necessary and sufficient condition for the consistency of multi label learning based on surrogate loss functions then we study the consistency of two well known multi label loss functions i e ranking loss and hamming loss for ranking loss our results disclose that surprisingly none of convex surrogate loss is consistent we present the partial ranking loss with which some surrogate losses are proven to be consistent we also discuss on the consistency of univariate surrogate losses for hamming loss we show that two multi label learning methods i e one vs all and pairwise comparison which can be regarded as direct extensions from multi class learning are inconsistent in general cases yet consistent under the dominating setting and similar results also hold for some recent multi label approaches that are variations of one vs all in addition we discuss on the consistency of learning approaches that address multi label learning by decomposing into a set of binary classification problems

the empirical mode decomposition and the hilbert spectrum for nonlinear and non stationary time series analysis a new method for analysing nonlinear and non stationary data has been developed the key part of the method is the empirical mode decomposition method with which any complicated data set can be dec

reduced rank regression via adaptive nuclear norm penalization we propose an adaptive nuclear norm penalization approach for low rank matrix approximation and use it to develop a new reduced rank estimation method for high dimensional multivariate regression the adaptive nuclear norm is defined as the weighted sum of the singular values of the matrix and it is generally non convex under the natural restriction that the weight decreases with the singular value however we show that the proposed non convex penalized regression method has a global optimal solution obtained from an adaptively soft thresholded singular value decomposition the method is computationally efficient and the resulting solution path is continuous the rank consistency of and prediction estimation performance bounds for the estimator are established for a high dimensional asymptotic regime simulation studies and an application in genetics demonstrate its efficacy

bilinear optimized product quantization for scalable visual content analysis product quantization pq has been recognized as a useful technique to encode visual feature vectors into compact codes to reduce both the storage and computation cost recent advances in retrieval and vision tasks indicate that high dimensional descriptors are critical to ensuring high accuracy on large scale data sets however optimizing pq codes with high dimensional data is extremely time consuming and memory consuming to solve this problem in this paper we present a novel pq method based on bilinear projection which can well exploit the natural data structure and reduce the computational complexity specifically we learn a global bilinear projection for pq where we provide both non parametric and parametric solutions the non parametric solution does not need any data distribution assumption the parametric solution can avoid the problem of local optima caused by random initialization and enjoys a theoretical error bound besides we further extend this approach by learning locally bilinear projections to fit underlying data distributions we show by extensive experiments that our proposed method dubbed bilinear optimization product quantization achieves competitive retrieval and classification accuracies while having significant lower time and space complexities

instantaneous frequency time analysis of physiology signals the application of pregnant women s radial artery pulse signals abstract this study used the hilbert huang transform a recently developed instantaneous frequency time analysis to analyze radial artery pulse signals taken from women in their 36th week of pregnancy and after pregnancy the acquired instantaneous frequency time spectrum hilbert spectrum is further compared with the morlet wavelet spectrum results indicate that the hilbert spectrum is especially suitable for analyzing the time series of non stationary radial artery pulse signals since in the hilbert huang transform signals are decomposed into different mode functions in accordance with signal s local time scale therefore the hilbert spectrum contains more detailed information than the morlet wavelet spectrum from the hilbert spectrum we can see that radial artery pulse signals taken from women in their 36th week of pregnancy and after pregnancy have different patterns this approach could be applied to facilitate non invasive diagnosis of fetus physiological signals in the future

coded sampling bound how much training is needed for iterative semi blind channel estimation coherent detection is commonly facilitated by means of pilot aided channel estimation while this approach offers a high estimation accuracy the spectral and power efficiencies are reduced due to the pilot overhead in the case that redundancy due to channel coding is not used for the purpose of channel estimation the maximum pilot spacing for which a time varying channel can be reconstructed without distortion is given by the nyquist shannon sampling theorem in this paper it is shown that if channel coding is taken into account for channel estimation in an iterative fashion the maximum spacing can be significantly extended which results in an increased spectral efficiency towards this goal a so called coded sampling bound is derived this semi analytical bound is compared with the nyquist shannon sampling theorem the presented results indicate that the maximum pilot spacing can be arbitrarily large given a suitable channel code and code rate

an adaptive fixed point proximity algorithm for solving total variation denoising models abstract we study an adaptive fixed point proximity algorithm to solve the total variation denoising model the objective function is a sum of two convex functions and one of them is composed by an affine transformation which is usually a regularization term by decoupling and splitting the problem is changed into two subproblems due to the nonsmooth and nondifferentiability of the subproblem we solve its proximity minimization problem instead of the original one to overcome the staircase effect during the process of denoising an adaptive criterion on proximity parameters is put forward at last we apply the improved algorithm to solve the isotropic total variation denoising model the numerical results are given to illustrate the efficiency of the algorithm

ica based on asymmetry we build a new approach to ica which is based on the data asymmetry instead of densities with heavy tails we use asymmetric ones split gaussians we verified our approach on images sound and eeg data in the case of source signal reconstructing our approach gives better results independent component analysis ica one of the basic tools in data analysis aims to find a coordinate system in which the components of the data are independent most of existing methods are based on the minimization of the function of fourth order moment kurtosis skewness third order moment has received much less attention in this paper we present a competitive approach to ica based on the split gaussian distribution which is well adapted to asymmetric data consequently we obtain a method which works better than the classical approaches especially in the case when the underlying density is not symmetric which is a typical situation in the color distribution in images

supervised matrix factorization hashing for cross modal retrieval the target of cross modal hashing is to embed heterogeneous multimedia data into a common low dimensional hamming space which plays a pivotal part in multimedia retrieval due to the emergence of big multimodal data recently matrix factorization has achieved great success in cross modal hashing however how to effectively use label information and local geometric structure is still a challenging problem for these approaches to address this issue we propose a cross modal hashing method based on collective matrix factorization which considers both the label consistency across different modalities and the local geometric consistency in each modality these two elements are formulated as a graph laplacian term in the objective function leading to a substantial improvement on the discriminative power of latent semantic features obtained by collective matrix factorization moreover the proposed method learns unified hash codes for different modalities of an instance to facilitate cross modal search and the objective function is solved using an iterative strategy the experimental results on two benchmark data sets show the effectiveness of the proposed method and its superiority over state of the art cross modal hashing methods

multi label image categorization with sparse factor representation the goal of multilabel classification is to reveal the underlying label correlations to boost the accuracy of classification tasks most of the existing multilabel classifiers attempt to exhaustively explore dependency between correlated labels it increases the risk of involving unnecessary label dependencies which are detrimental to classification performance actually not all the label correlations are indispensable to multilabel model negligible or fragile label correlations cannot be generalized well to the testing data especially if there exists label correlation discrepancy between training and testing sets to minimize such negative effect in the multilabel model we propose to learn a sparse structure of label dependency the underlying philosophy is that as long as the multilabel dependency cannot be well explained the principle of parsimony should be applied to the modeling process of the label correlations the obtained sparse label dependency structure discards the outlying correlations between labels which makes the learned model more generalizable to future samples experiments on real world data sets show the competitive results compared with existing algorithms

dual diversified dynamical gaussian process latent variable model for video repairing in this paper we propose a dual diversified dynamical gaussian process latent variable model text d 3 gplvm to tackle the video repairing issue for preservation purposes videos have to be conserved on media however storing on media such as films and hard disks can suffer from unexpected data loss for instance physical damage so repairing of missing or damaged pixels is essential for better video maintenance most methods seek to fill in missing holes by synthesizing similar textures from local patches the neighboring pixels consecutive frames or the whole video however these can introduce incorrect contexts especially when the missing hole or number of damaged frames is large furthermore simple texture synthesis can introduce artifacts in undamaged and recovered areas to address aforementioned problems we introduce two diversity encouraging priors to both of inducing points and latent variables for considering the variety in existing videos in text d 3 gplvm the inducing points constitute a smaller subset of observed data while latent variables are a low dimensional representation of observed data since they have a strong correlation with the observed data it is essential that both of them can capture distinct aspects of and fully represent the observed data the dual diversity encouraging priors ensure that the trained inducing points and latent variables are more diverse and resistant for context aware and artifacts free based video repairing the defined objective function in our proposed model is initially not analytically tractable and must be solved by variational inference finally experimental testing results illustrate the robustness and effectiveness of our method for damaged video repairing

neighborhood discriminant hashing for large scale image retrieval with the proliferation of large scale community contributed images hashing based approximate nearest neighbor search in huge databases has aroused considerable interest from the fields of computer vision and multimedia in recent years because of its computational and memory efficiency in this paper we propose a novel hashing method named neighborhood discriminant hashing ndh for short to implement approximate similarity search different from the previous work we propose to learn a discriminant hashing function by exploiting local discriminative information i e the labels of a sample can be inherited from the neighbor samples it selects the hashing function is expected to be orthogonal to avoid redundancy in the learned hashing bits as much as possible while an information theoretic regularization is jointly exploited using maximum entropy principle as a consequence the learned hashing function is compact and nonredundant among bits while each bit is highly informative extensive experiments are carried out on four publicly available data sets and the comparison results demonstrate the outperforming performance of the proposed ndh method over state of the art hashing techniques

probability estimates for multi class classification by pairwise coupling pairwise coupling is a popular multi class classification method that combines all comparisons for each pair of classes this paper presents two approaches for obtaining class probabilities both methods can be reduced to linear systems and are easy to implement we show conceptually and experimentally that the proposed approaches are more stable than the two existing popular methods voting and the method by hastie and tibshirani 1998

nonlinear indicial response of complex nonstationary oscillations as pulmonary hypertension responding to step hypoxia this paper is devoted to the quantization of the degree of nonlinearity of the relationship between two biological variables when one of the variables is a complex nonstationary oscillatory signal an example of the situation is the indicial responses of pulmonary blood pressure p to step changes of oxygen tension po2 in the breathing gas for a step change of po2 beginning at time t1 the pulmonary blood pressure is a nonlinear function of time and po2 which can be written as p t t1 po2 an effective method does not exist to examine the nonlinear function p t t1 po2 a systematic approach is proposed here the definitions of mean trends and oscillations about the means are the keys with these keys a practical method of calculation is devised we fit the mean trends of blood pressure with analytic functions of time whose nonlinearity with respect to the oxygen level is clarified here the associated oscillations about the mean can be transformed into hilbert spectrum an integration of the square of the hilbert spectrum over frequency yields a measure of oscillatory energy which is also a function of time whose mean trends can be expressed by analytic functions the degree of nonlinearity of the oscillatory energy with respect to the oxygen level also is clarified here theoretical extension of the experimental nonlinear indicial functions to arbitrary history of hypoxia is proposed application of the results to tissue remodeling and tissue engineering of blood vessels is discussed

correlated logistic model with elastic net regularization for multilabel image classification in this paper we present correlated logistic corrlog model for multilabel image classification corrlog extends conventional logistic regression model into multilabel cases via explicitly modeling the pairwise correlation between labels in addition we propose to learn the model parameters of corrlog with elastic net regularization which helps exploit the sparsity in feature selection and label correlations and thus further boost the performance of multilabel classification corrlog can be efficiently learned though approximately by regularized maximum pseudo likelihood estimation and it enjoys a satisfying generalization bound that is independent of the number of labels corrlog performs competitively for multilabel image classification on benchmark data sets mulan scene mit outdoor scene pascal voc 2007 and pascal voc 2012 compared with the state of the art multilabel classification algorithms

sce a manifold regularized set covering method for data partitioning cluster analysis plays a very important role in data analysis in these years cluster ensemble as a cluster analysis tool has drawn much attention for its robustness stability and accuracy many efforts have been done to combine different initial clustering results into a single clustering solution with better performance however they neglect the structure information of the raw data in performing the cluster ensemble in this paper we propose a structural cluster ensemble sce algorithm for data partitioning formulated as a set covering problem in particular we construct a laplacian regularized objective function to capture the structure information among clusters moreover considering the importance of the discriminative information underlying in the initial clustering results we add a discriminative constraint into our proposed objective function finally we verify the performance of the sce algorithm on both synthetic and real data sets the experimental results show the effectiveness of our proposed method sce algorithm

a constrained least squares regression model abstract least squares regression lsr is a widely used regression technique for multicategory classification conventional lsr model assumes that during the learning phase the labeled samples can be exactly transformed into a discrete label matrix which is too strict to learn a regression matrix for fitting the labels to overcome this drawback lots of lsr s variants utilize the soft target label which contains the continuous values to replace this discrete label to improve the learning performance since the regression matrix can be learnt from these soft target labels it is reasonable to assume that the samples in the same class have similar soft target labels nevertheless most of existing lsr based models don t adequately consider this similarity assumption in this paper we propose a constrained least squares regression clsr model for multicategory classification the main motivation of clsr is to force the samples in the same class to obtain the similar soft target labels to effectively optimize clsr we propose a novel alternating algorithm which can converge to the globally optimal solution extensive experiments results on face and digit data sets confirm the effectiveness of our proposed model

local rademacher complexities we propose new bounds on the error of learning algorithms in terms of a data dependent notion of complexity the estimates we establish give optimal rates and are based on a local and empirical version of rademacher averages in the sense that the rademacher averages are computed from the data on a subset of functions with small empirical error we present some applications to classification and prediction with convex function classes and with kernel classes in particular

optimal placement of training for frequency selective block fading channels the problem of placing training symbols optimally for orthogonal frequency division multiplexing ofdm and single carrier systems is considered the channel is assumed to be quasi static with a finite impulse response of length l 1 samples under the assumptions that neither the transmitter nor the receiver knows the channel and that the receiver forms a minimum mean square error mmse channel estimate based on training symbols only training is optimized by maximizing a tight lower bound on the ergodic training based independent and identically distributed i i d capacity for ofdm systems it is shown that the lower bound is maximized by placing the known symbols periodically in frequency for single carrier systems under the assumption that the training symbols are placed in clusters of length spl alpha spl ges 2l 1 it is shown that the lower bound is maximized by a family of placement schemes called qpp spl alpha where qpp stands for quasi periodic placement these placement schemes are formed by grouping the known symbols into as many clusters as possible and then placing these clusters periodically in the packet for both ofdm and single carrier systems the optimum energy tradeoff between training and data is also obtained