nils-herrmann/clean_scirepeval huggingface dataset:
Patch-Set-Based Representation for Alignment-Free Image Set Classification

This paper presents a patch-set-based sparse representation for image set classification. Compared with image-based image set representation, our patch-set-based representation is alignment free and thus has an advantage for tasks like video-based face recognition, image-set-based object recognition, and video-based hand gesture recognition, where precious alignment is usually difficult or even impossible due to large variance in view angle or pose. Specifically, to bypass the alignment issue, we propose to adopt the patch-based image set representation by dividing each image within each set into patches, then we cluster all the training patches into multiple clusters and classify the test patches based on the cluster centers of training patches. The labels of test patches within each cluster are inferred from a patch-set-based sparse representation for classification, and the labels of all test patches from all the clusters are then aggregated to predict a single label for the test set. Experimental results on video-based face recognition data sets (CMU-MoBo and YouTube Celebrities), image-set-based object recognition data set (ETH-80), and video-based hand gesture recognition data set (Kinect Hand Gestures) demonstrate that our proposed method consistently outperforms all existing ones, and the improvement is very significant on the YouTube Celebrities and Kinect Hand Gesture data sets. Moreover, we also quantitatively show the robustness of our method to misalignment on the Mutli-PIE data set.
Learning discriminative supplementary features to attributes for novel-category classification

Semantic attributes have been introduced as an effective representation for image classification especially in zero-shot learning. However, most of the existing semantic attributes are previously defined by people, thus the size of the attribute is restricted in practice and these attributes are not necessarily discriminative. Therefore, the classification accuracy is often relatively low using a fixed incomplete semantic attribute set for image representation. One intuitive solution is to expand the semantic attribute representation with some non-semantic features. However, how to make the supplementary features more effective and discriminative is still an open problem. In this paper, we propose a Discriminative Supplementary Feature Learning (DSFL) method to implement semantic attribute augmentation. In DSFL, the non-semantic supplementary features are learned simultaneously with the classifiers for the novel-categories. Extensive experiments are conducted on two public datasets and the results show that our approach achieves encouraging performance.
Semantic Neighbor Graph Hashing for Multimodal Retrieval.

Hashing methods have been widely used for approximate nearest neighbor search in recent years due to its computational and storage effectiveness. Most existing multimodal hashing methods try to preserve the similarity relationship based on either metric distances or semantic labels in a procrustean way, while ignoring the intra-class and inter-class variations inherent in the metric space. In this paper, we propose a novel multimodal hashing method, termed as semantic neighbor graph hashing (SNGH), which aims to preserve the fine-grained similarity metric based on the semantic graph that is constructed by jointly pursuing the semantic supervision and the local neighborhood structure. Specifically, the semantic graph is constructed to capture the local similarity structure for the image modality and the text modality, respectively. Furthermore, we define a function based on the local similarity in particular to adaptively calculate multi-level similarities by encoding the intra-class and inter-class variations. After obtaining the unified hash codes, the logistic regression with kernel trick is employed to learn view-specific hash functions independently for each modality. Extensive experiments are conducted on four widely used multimodal data sets. The experimental results demonstrate the superiority of the proposed SNGH method compared with the state-of-the-art multimodal hashing methods.
Multi-Grained Random Fields for Mitosis Identification in Time-Lapse Phase Contrast Microscopy Image Sequences

This paper proposes a multi-grained random fields (MGRFs) model for mitosis identification. To deal with the difficulty in hidden state discovery and sequential structure modeling in mitosis sequences only containing gradual visual pattern changes, we design the graphical structure to transform individual sequence into a set of coarse-to-fine grained sequencesconveying diverse temporal dynamics. Furthermore, we propose the corresponding probabilistic model for joint temporal learning and feature learning. To deal with the non-convex formulation of MGRF, we decomposemodel training into two sub-tasks, layer-wise sequential learning of both temporal dynamics and visual feature and new layer generation by graph-based sequential grouping, and optimize the model by alternating between them iteratively. The proposed method is validated on very challenging mitosis data set of C3H10T1/2 and C2C12 stem cells. Extensive comparison experiments demonstrate its superiority to the state of the arts.
Image Classification With Tailored Fine-Grained Dictionaries

In this paper, we propose a novel fine-grained dictionary learning method for image classification. To learn a high-quality discriminative dictionary, three types of multispecific subdictionaries, i.e., class-specific dictionaries (CSDs), universal dictionary (UD), and family-specific dictionaries (FSDs), are simultaneously uncovered. Here, CSDs and UD, respectively, model the patterns for each class and the patterns irrespective of any class. FSDs can help reveal the shared patterns between multiple image classes, by filling the gap between the patterns in CSDs and UD. The dependence among image classes is revealed by the shared FSDs, and a common FSD can be assigned to several classes to represent their residual. Finally, the most discriminative FSD for each class is identified by minimizing the sparse reconstruction error. Extensive experiments are conducted on different widely used data sets for image classification. The results demonstrate the superior performance of the proposed method over some state-of-the-art methods.
Beyond local image features: Scene calssification using supervised semantic representation

The use of local features for image representation has been proven very effective for a variety of visual tasks such as object localization and scene classification. However, local image features carry little semantic information which is potentially not enough for high level visual tasks. To solve this problem, in this paper, we propose to use a supervised semantic image representation for scene classification, where an image is represented as a response histogram. This response histogram is a combination of the prediction of pre-trained generic object classifiers and classifiers generated by supervised learning. Besides, the use of sparsity constraints makes the proposed representation more efficient and effective to compute. Performances on the UIUC-Sports dataset, the MIT Indoor scene dataset and the Scene-15 dataset demonstrate the effectiveness of the proposed method.
Discriminative Deep Quantization Hashing for Face Image Retrieval

This paper proposes a new discriminative deep quantization hashing (DDQH) approach for large-scale face image retrieval by learning discriminative and compact binary codes. It jointly explores the discrete code learning, batch normalization quantization (BNQ) module, and end-to-end learning in one unified framework, which can guarantee the optimal compatibility of hash coding and feature learning. To learn multiscale and robust facial features, a deep network properly stacking several convolution-pooling layers and pooling layers is designed, and the facial features are obtained by fusing the outputs of the last convolutional layer and the last pooling layer. Besides, the prediction errors of the learned binary codes are minimized to learn discriminative binary codes of images. To obtain higher retrieval accuracies, a BNQ module is utilized to control quantization at a moderate level. Experiments are conducted on two widely used data sets, and the proposed DDQH method achieves encouraging improvements over some state-of-the-art hashing approaches.
Modeling Multimodal Clues in a Hybrid Deep Learning Framework for Video Classification

Videos are inherently multimodal. This paper studies the problem of exploiting the abundant multimodal clues for improved video classification performance. We introduce a novel hybrid deep learning framework that integrates useful clues from multiple modalities, including static spatial appearance information, motion patterns within a short time window, audio information, as well as long-range temporal dynamics. More specifically, we utilize three Convolutional Neural Networks (CNNs) operating on appearance, motion, and audio signals to extract their corresponding features. We then employ a feature fusion network to derive a unified representation with an aim to capture the relationships among features. Furthermore, to exploit the long-range temporal dynamics in videos, we apply two long short-term memory (LSTM) networks with extracted appearance and motion features as inputs. Finally, we also propose refining the prediction scores by leveraging contextual relationships among video semantics. The hybrid deep learning framework is able to exploit a comprehensive set of multimodal features for video classification. Through an extensive set of experiments, we demonstrate that: 1) LSTM networks that model sequences in an explicitly recurrent manner are highly complementary to the CNN models; 2) the feature fusion network that produces a fused representation through modeling feature relationships outperforms a large set of alternative fusion strategies; and 3) the semantic context of video classes can help further refine the predictions for improved performance. Experimental results on two challenging benchmarks—the UCF-101 and the Columbia Consumer Videos (CCV)—provide strong quantitative evidence that our framework can produce promising results: ${\text{93.1}\%}$  on the UCF-101 and ${\text{84.5}\%}$  on the CCV, outperforming several competing methods with clear margins.
RGB-D Object Recognition via Incorporating Latent Data Structure and Prior Knowledge

For the task of RGB-D object recognition, it is important to identify suitable representations of images, which can boost the performance of object recognition. In this work, we propose a novel representation learning method for RGB-D images by jointly incorporating the underlying data structure and the prior knowledge of the data. Specifically, the convolutional neural networks (CNN) are employed to learn image representation by exploiting the underlying data structure. To handle the problem of the limited RGB and depth images for object recognition, the multi-level hierarchies of features trained on ImageNet from the CNN are transferred to learn rich generic feature representation for RGB and depth images while the labeled images are leveraged. On the other hand, we propose a novel deep auto-encoders (DAE) to exploit the prior knowledge, which can overcome the expensive computational cost of optimization in feature encoding. The expected representations of images are obtained by integrating the two types of image representations. To verify the effectiveness of the proposed method, we thoroughly conduct extensive experiments on two publicly available RGB-D datasets. The encouraging experimental results compared with the state-of-the-art approaches demonstrate the advantages of the proposed method.
Robust Structured Nonnegative Matrix Factorization for Image Representation

Dimensionality reduction has attracted increasing attention, because high-dimensional data have arisen naturally in numerous domains in recent years. As one popular dimensionality reduction method, nonnegative matrix factorization (NMF), whose goal is to learn parts-based representations, has been widely studied and applied to various applications. In contrast to the previous approaches, this paper proposes a novel semisupervised NMF learning framework, called robust structured NMF, that learns a robust discriminative representation by leveraging the block-diagonal structure and the <inline-formula> <tex-math notation="LaTeX">$\ell _{2,p}$ </tex-math></inline-formula>-norm (especially when <inline-formula> <tex-math notation="LaTeX">$0<p\leq 1$ </tex-math></inline-formula>) loss function. Specifically, the problems of noise and outliers are well addressed by the <inline-formula> <tex-math notation="LaTeX">$\ell _{2,p}$ </tex-math></inline-formula>-norm (<inline-formula> <tex-math notation="LaTeX">$0<p\leq 1$ </tex-math></inline-formula>) loss function, while the discriminative representations of both the labeled and unlabeled data are simultaneously learned by explicitly exploring the block-diagonal structure. The proposed problem is formulated as an optimization problem with a well-defined objective function solved by the proposed iterative algorithm. The convergence of the proposed optimization algorithm is analyzed both theoretically and empirically. In addition, we also discuss the relationships between the proposed method and some previous methods. Extensive experiments on both the synthetic and real-world data sets are conducted, and the experimental results demonstrate the effectiveness of the proposed method in comparison to the state-of-the-art methods.
Image Denoising Based on Edge Recovery

Common smoothing algorithms can be considered as different kinds of diffusion, which try to make a trade off between denoising and edge information reservation. In this paper, however, we deal with the denoising problem from a novel thought-employing an inverse diffusion to recover edge information after image smoothing, combining adaptive scale filtering and inverse diffusion function for image denoising. This method adopts minimal description length (MDL) rule to adaptively choose the optimal scale for each pixel to smooth the image and improves inverse diffusion function to recover the edges in the degraded image. Experiments show that this method is more effective than classical filters and anisotropic diffusion
Image Registration Based on Fitted Straight Lines of Edges

Rotation problem is a difficulty in common image registration methods for computational complexity. Aiming at this, a novel method based on fitted straight lines of edges is proposed. It first estimates the rotating angle by angle difference histogram of fitted straight lines, and then estimates the translation by Hausdorff distance. After compensating the rotation and translation, mutual information based on Renyi entropy is used to find the accurate registration result in a small domain. Experiment results demonstrate that this method is effective in solving rotation problem
Circulating-current analysis and power sharing control of parallel inverters with different capacities

This paper proposes control methods to achieve accurate active and reactive power sharing of parallel multi-inverters in proportion to their capacities. Firstly, according to physical properties of parallel inverters' output power, the interaction mechanism between circulating-current and load-sharing is elaborated. Conditions for a control strategy to result in specified power sharing and minimum circulating-current are discussed based on the assumption that transmission lines are inductive. Limited effectiveness of traditional droop control is deeply analysed. An improved droop control strategy is proposed to achieve accurate proportional power sharing results unaffected by the impedance of lines. Moreover, since lines in microgrid are not inductive in practice, coupling of active and reactive power is observed on lines. Based on improved droop control, a new control strategy with added virtual reactance is furtherly proposed to enable decoupling control of active and reactive power sharing. The effectiveness of theoretical analyses and proposed control strategies in this paper is verified by PSCAD/EMTDC simulation results.
A Deterministic Analysis for LRR

The recently proposed low-rank representation (LRR) method has been empirically shown to be useful in various tasks such as motion segmentation, image segmentation, saliency detection and face recognition. While potentially powerful, LRR depends heavily on the configuration of its key parameter, λ. In realistic environments where the prior knowledge about data is lacking, however, it is still unknown how to choose λ in a suitable way. Even more, there is a lack of rigorous analysis about the success conditions of the method, and thus the significance of LRR is a little bit vague. In this paper we therefore establish a theoretical analysis for LRR, striving for figuring out under which conditions LRR can be successful, and deriving a moderately good estimate to the key parameter λ as well. Simulations on synthetic data points and experiments on real motion sequences verify our claims.
Generalized Nonconvex Nonsmooth Low-Rank Minimization

As surrogate functions of L0-norm, many nonconvex penalty functions have been proposed to enhance the sparse vector recovery. It is easy to extend these nonconvex penalty functions on singular values of a matrix to enhance low-rank matrix recovery. However, different from convex optimization, solving the nonconvex low-rank minimization problem is much more challenging than the nonconvex sparse minimization problem. We observe that all the existing nonconvex penalty functions are concave and monotonically increasing on [0, ∞). Thus their gradients are decreasing functions. Based on this property, we propose an Iteratively Reweighted Nuclear Norm (IRNN) algorithm to solve the nonconvex nonsmooth low-rank minimization problem. IRNN iteratively solves a Weighted Singular Value Thresholding (WSVT) problem. By setting the weight vector as the gradient of the concave penalty function, the WSVT problem has a closed form solution. In theory, we prove that IRNN decreases the objective function value monotonically, and any limit point is a stationary point. Extensive experiments on both synthetic data and real images demonstrate that IRNN enhances the low-rank matrix recovery compared with state-of-the-art convex algorithms.
Latent Dirichlet Truth Discovery: Separating Trustworthy and Untrustworthy Components in Data Sources

The discovery of truth is a critical step toward effective information and knowledge utilization, especially in Web services, social media networks, and sensor networks. Typically, a set of sources with varying reliability claim observations about a set of objects and the goal is to jointly discover the true fact for each object and the trustworthy degree of each source. In this paper, we propose a latent Dirichlet truth (LDT) discovery model to approach this problem. It defines a random field over all the possible configurations of the trustworthy degrees of sources and facts, and the most probable configuration is inferred by a maximum a posteriori criterion over the observed claims. We note that a typical source is usually made of mixed trustworthy and untrustworthy components, since it can make true or false claims on different objects. While most of the existing algorithms do not attempt separate the untrustworthy component from the trustworthy one in each source, the proposed model explicitly identifies untrustworthy component in each source. This makes the LDT model more capable of separating the trustworthy and untrustworthy components, and in turn improves the accuracy of truth discovery. Experiments on real data sets show competitive results compared with existing algorithms.
Weakly Supervised Deep Matrix Factorization for Social Image Understanding

The number of images associated with weakly supervised user-provided tags has increased dramatically in recent years. User-provided tags are incomplete, subjective and noisy. In this paper, we focus on the problem of social image understanding, i.e., tag refinement, tag assignment, and image retrieval. Different from previous work, we propose a novel weakly supervised deep matrix factorization algorithm, which uncovers the latent image representations and tag representations embedded in the latent subspace by collaboratively exploring the weakly supervised tagging information, the visual structure, and the semantic structure. Due to the well-known semantic gap, the hidden representations of images are learned by a hierarchical model, which are progressively transformed from the visual feature space. It can naturally embed new images into the subspace using the learned deep architecture. The semantic and visual structures are jointly incorporated to learn a semantic subspace without overfitting the noisy, incomplete, or subjective tags. Besides, to remove the noisy or redundant visual features, a sparse model is imposed on the transformation matrix of the first layer in the deep architecture. Finally, a unified optimization problem with a well-defined objective function is developed to formulate the proposed problem and solved by a gradient descent procedure with curvilinear search. Extensive experiments on real-world social image databases are conducted on the tasks of image understanding: image tag refinement, assignment, and retrieval. Encouraging results are achieved with comparison with the state-of-the-art algorithms, which demonstrates the effectiveness of the proposed method.
Joint Video Frame Set Division and Low-Rank Decomposition for Background Subtraction

The recently proposed robust principle component analysis (RPCA) has been successfully applied in background subtraction. However, low-rank decomposition makes sense on the condition that the foreground pixels (sparsity patterns) are uniformly located at the scene, which is not realistic in real-world applications. To overcome this limitation, we reconstruct the input video frames and aim to make the foreground pixels not only sparse in space but also sparse in time. Therefore, we propose a joint video frame set division and RPCA-based method for background subtraction. In addition, we use the motion as a priori knowledge which has not been considered in the current subspace-based methods. The proposed method consists of two phases. In the first phase, we propose a lower bound-based within-class maximum division method to divide the video frame set into several subsets. In this way, the successive frames are assigned to different subsets in which the foregrounds are located at the scene randomly. In the second phase, we augment each subset using the frames with a small quantity of motion. To evaluate the proposed method, the experiments are conducted on real-world and public datasets. The comparisons with the state-of-the-art background subtraction methods validate the superiority of our method.
Semantic-Gap-Oriented Active Learning for Multilabel Image Annotation

User interaction is an effective way to handle the semantic gap problem in image annotation. To minimize user effort in the interactions, many active learning methods were proposed. These methods treat the semantic concepts individually or correlatively. However, they still neglect the key motivation of user feedback: to tackle the semantic gap. The size of the semantic gap of each concept is an important factor that affects the performance of user feedback. User should pay more efforts to the concepts with large semantic gaps, and vice versa. In this paper, we propose a semantic-gap-oriented active learning method, which incorporates the semantic gap measure into the information-minimization-based sample selection strategy. The basic learning model used in the active learning framework is an extended multilabel version of the sparse-graph-based semisupervised learning method that incorporates the semantic correlation. Extensive experiments conducted on two benchmark image data sets demonstrated the importance of bringing the semantic gap measure into the active learning process.