In this paper, we study the problem of semi-supervised structured output prediction, which aims to learn predictors for structured outputs, such as sequences, tree nodes, vectors, etc., from a set of data points of both inputoutput pairs and single inputs without outputs. The traditional methods to solve this problem usually learns one single predictor for all the data points, and ignores the variety of the different data points. Different parts of the data set may have different local distributions, and requires different optimal local predictors. To overcome this disadvantage of existing methods, we propose to learn different local predictors for neighborhoods of different data points, and the missing structured outputs simultaneously. In the neighborhood of each data point, we proposed to learn a linear predictor by minimizing both the complexity of the predictor and the upper bound of the structured prediction loss. The minimization is conducted by gradient descent algorithms. Experiments over four benchmark data sets, including DDSM mammography medical images, SUN natural image data set, Cora research paper data set, and Spanish news wire article sentence data set, show the advantages of the proposed method.
@cite_14 proposed to regularize the structured outputs by the manifold constructed from the input space directly. This method constructs a nearest neighbor graph from the input features, and use it to represent the manifold. Then the manifold is used to regularize the learning of the missing outputs of the unlabeled data points. The outputs and the predictor are learned simultaneously, and they regularize each other in the learning process.
@cite_14
1904566509
Structured output prediction aims to learn a predictor to predict a structured output from a input data vector . The structured outputs include vector, tree, sequence, etc. We usually assume that we have a training set of input---output pairs to train the predictor. However, in many real-world applications, it is difficult to obtain the output for a input, and thus for many training input data points, the structured outputs are missing. In this paper, we discuss how to learn from a training set composed of some input---output pairs and some input data points without outputs. This problem is called semi-supervised structured output prediction. We propose a novel method for this problem by constructing a nearest neighbor graph from the input space to present the manifold structure and use it to regularize the structured output space directly. We define a slack structured output for each training data point and propose to predict it by learning a structured output predictor. The learning of both slack structured outputs and the predictor are unified within one single minimization problem. In this problem, we propose to minimize the structured loss between the slack structured outputs of neighboring data points and the prediction error measured by the structured loss. The problem is optimized by an iterative algorithm. Experiment results over three benchmark data sets show its advantage. In this letter, we propose a novel low-rank-represen-tation (LRR) based manifold-regularization approach for zero-shot learning (ZSL). Most existing regularization-based ZSL approaches perform the alignment between visual feature space and semantic space based on the affinity matrix constructed from the test instances. The affinity matrix plays a significant role in exploiting the manifold structures of visual feature space, hence we propose to use the LRR to guide the affinity-matrix construction by exploring the subspace structures of data. Considering the locality and similarity information among data, we incorporate a Laplacian regularization term to the LRR framework to ensure that the learned affinity matrix can capture the local geometric structures in data. We also explicitly impose the nonnegative sparse constraint on the affinity matrix to facilitate the learning of local manifold structures. Moreover, we use an effective manifold-regularization methodology to learn discriminative semantic representations of test instances, leading to significant improvements in classification performance over the unseen classes. Extensive experiments on three benchmark datasets demonstrate that the proposed approach outperforms the state of the arts. Manifold regularization (, 2006) is a geometrically motivated framework for machine learning within which several semi-supervised algorithms have been constructed. Here we try to provide some theoretical understanding of this approach. Our main result is to expose the natural structure of a class of problems on which manifold regularization methods are helpful. We show that for such problems, no supervised learner can learn effectively. On the other hand, a manifold based learner (that knows the manifold or "learns" it from unlabeled examples) can learn with relatively few labeled examples. Our analysis follows a minimax style with an emphasis on finite sample results (in terms of n: the number of labeled examples). These results allow us to properly interpret manifold regularization and related spectral and geometric algorithms in terms of their potential use in semi-supervised learning. a b s t r a c t Constructing an informative and discriminative graph plays an important role in various pattern recogni- tion tasks such as clustering and classification. Among the existing graph-based learning models, low-rank representation (LRR) is a very competitive one, which has been extensively employed in spectral cluster- ing and semi-supervised learning (SSL). In SSL, the graph is composed of both labeled and unlabeled sam- ples, where the edge weights are calculated based on the LRR coefficients. However, most of existing LRR related approaches fail to consider the geometrical structure of data, which has been shown beneficial for discriminative tasks. In this paper, we propose an enhanced LRR via sparse manifold adaption, termed manifold low-rank representation (MLRR), to learn low-rank data representation. MLRR can explicitly take the data local manifold structure into consideration, which can be identified by the geometric sparsity idea; specifically, the local tangent space of each data point was sought by solving a sparse representation objective. Therefore, the graph to depict the relationship of data points can be built once the manifold information is obtained. We incorporate a regularizer into LRR to make the learned coefficients preserve the geometric constraints revealed in the data space. As a result, MLRR combines both the global informa- tion emphasized by low-rank property and the local information emphasized by the identified manifold structure. Extensive experimental results on semi-supervised classification tasks demonstrate that MLRR is an excellent method in comparison with several state-of-the-art graph construction approaches.