nils-herrmann/clean_scirepeval huggingface dataset:
Advertise gently - in-image advertising with low intrusiveness

The new trend of online advertisement is in-image advertising, which is facing the risk of being intrusive. Several works have been done to reduce the intrusiveness. However, intrusiveness is a subjective concept and is difficult to be measured objectively. In this paper, by considering the fact that gentle advertising will not disturb audiences' attention too much but the intrusive ones will, we investigate the relationship between intrusiveness and audience attention. By experiment, we find that two aspects of attention will affect intrusiveness. Firstly, if the inserted advertisement covers the Region of Interest (ROI), it is truly very intrusive. Secondly, if the advertisement distracts audience attention from the original attending point, it is also very intrusive. We measure intrusiveness from the above two aspects. Using this measurement, we insert advertisements into online image collections gently. Given a pair of an image and an advertisement, we detect the suitable place, using attention analysis and visual consistency, to reduce intrusiveness. Given an image set and an advertisement set, we minimize the intrusiveness by searching for an optimal match. Experimental results verify the effectiveness of the proposed measurement of intrusiveness and of the advertising approach.
Reliable Video Clock Time Recognition

We propose a novel approach to read the video clock in real time by recognizing the clock digits using a few techniques relative to the transition patterns of the clock. With these techniques, the clock digits are located without recognizing all the text characters overlaid together with the clock on the video, the sample digit patterns are automatically extracted and labeled in real time. The clock digits are recognized while the video is playing back without any offline training. The recognition result is further verified against the digit transition sequence to improve the performance. Experimental results show that this approach is reliable and the result is accurate regardless of the variation of font, size and color of the digit characters in different videos
Event based news video people classification and ranking using multimodality features

Existing research on news video analysis mainly concentrates on structure analysis, semantic concept detection, annotation and search. However, little work has been contributed to news video people community analysis, which is helpful for users to understand the event. In this paper, we propose a novel approach to classify the people appearing in the news video into different communities. In our approach, the people appearing in the news video are first identified by associating their faces with names. The faces are detected from the video frames, and the names are obtained from the text. Then, the people belonging to the same organization are clustered. After that, the relationships between these organizations are determined using sentiment analysis. The sentiment words are diverse in each news story and contain both positive and negative ones. However, we have news title, which is the summary of the story and the sentiment of which is clear, to help us to mine the relationships between the organizations. At last, social networks are built to classify those people/organizations into different classes, and the people/organizations are ranked in each community according to their influence. The main contributions of the paper are two folds: 1) we propose a novel approach to present the news video event according to communities; 2) we propose to use the sentiment analysis and social network to classify the news people/organizations. The experimental results on the selected news topics demonstrate that the proposed approach is effective.
Compact Codebook Generation Towards Scale-Invariance

In this paper, we present a novel visual codebook learning approach towards compactness and scale-invariance for dense patch image encoding. Firstly, each image is described as a bag of orderless gridding local patches, each of which is expressed in three scales. Then a unified objective function is proposed to simultaneously enforce the codebook compactness and select the optimal scale for each local patch, and a convergency provable iterative procedure is utilized for optimization. A direct advantage of the new codebook is that each local patch is essentially described by its best scale, and thus shares certain characteristic of SIFT yet not constrained to any salient point detectors. The experiments on PASCAL 07 dataset validate the effectiveness and efficiency of our proposed method for image classification task.
Spectral Hashing With Semantically Consistent Graph for Image Indexing

The ability of fast similarity search in a large-scale dataset is of great importance to many multimedia applications. Semantic hashing is a promising way to accelerate similarity search, which designs compact binary codes for a large number of images so that semantically similar images are mapped to close codes. Retrieving similar neighbors is then simply accomplished by retrieving images that have codes within a small Hamming distance of the code of the query. Among various hashing approaches, spectral hashing (SH) has shown promising performance by learning the binary codes with a spectral graph partitioning method. However, the Euclidean distance is usually used to construct the graph Laplacian in SH, which may not reflect the inherent distribution of the data. Therefore, in this paper, we propose a method to directly optimize the graph Laplacian. The learned graph, which can better represent similarity between samples, is then applied to SH for effective binary code learning. Meanwhile, our approach, unlike metric learning, can automatically determine the scale factor during the optimization. Extensive experiments are conducted on publicly available datasets and the comparison results demonstrate the effectiveness of our approach.
Knowing Verb From Object: Retagging With Transfer Learning on Verb-Object Concept Images

Image retagging is significant and essential for tag-based applications, such as search and browsing. However, most existing image retagging approaches are typically based on enriching-and-removing and/or reranking strategies, which lead to two drawbacks: 1) since the object and/or human appeared in the images are tagged as individuals, the meanings represented by the mutual context of object and human are ignored and not tagged, and 2) some images which are visually dissimilar but semantically similar could be filtered incorrectly, as they are conflict with the content consistency rule. These two defects are distinct especially when images with human-object interactions are retagged. To tackle these defects, in this paper we propose a Bayesian approach to jointly consider the human and object in an image and retag it properly. In our approach, human and objects in images are detected and their interrelationships are taken into account. Tags which represent the mutual context of human and objects are then mapped to those interrelationships by a probabilistic graphical model. For a new image which lacks the tag representing the interaction between human and object, our model can correctly retag it for the interaction. In this paper, those images involving human-object interactions are called verb-object concept images, and experiments on a 60-class dataset demonstrate the capacity of our Bayesian retagging approach of verb-object concept images (BRVOI).
Fully and Semi-Automatic Music Sports Video Composition

Video composition is important for music video production. In this paper we propose an automatic method to assist the music sports video composition operation. Our approach is based on dynamic programming algorithm which finds a set of video shots that best matches the music. The method by default is fully-automatic, and users specification could be inserted to control the composition process, making it a semiautomatic system. This research has obvious importance to reduce manual processing, and enables the generation of high quality personalized music sports video. The proposed method is generic and fast. The experimental results are satisfactory
Tag-aware image classification via Nested Deep Belief nets

With the rising of internet photos-sharing web sites, the rich aware text information surrounding images on the sites are proved helpful to improve the image classification. This paper presents a novel nested deep learning model called Nested Deep Belief Network(NDBN) for tag-aware image classification. A multi-layer structure of Deep Belief Network(DBN) is established to learn a unified representation of visual feature and tag feature for an image, and an additional Gaussian Restricted Boltzmann Machine is built to capture the tag-tag dependency. Compared with conventional methods, the proposed model can not only find correlations across modalities, but mine the importance for different tags, and also bring about low-rank tag feature representation. We conduct experiments over the MIR Flickr dataset and the results show that the proposed NDBN model outperforms the existing image classification techniques.
Inductive Robust Principal Component Analysis

In this paper, we address the error correction problem, that is, to uncover the low-dimensional subspace structure from high-dimensional observations, which are possibly corrupted by errors. When the errors are of Gaussian distribution, principal component analysis (PCA) can find the optimal (in terms of least-square error) low-rank approximation to high-dimensional data. However, the canonical PCA method is known to be extremely fragile to the presence of gross corruptions. Recently, Wright established a so-called robust principal component analysis (RPCA) method, which can well handle the grossly corrupted data. However, RPCA is a transductive method and does not handle well the new samples, which are not involved in the training procedure. Given a new datum, RPCA essentially needs to recalculate over all the data, resulting in high computational cost. So, RPCA is inappropriate for the applications that require fast online computation. To overcome this limitation, in this paper, we propose an inductive robust principal component analysis (IRPCA) method. Given a set of training data, unlike RPCA that targets on recovering the original data matrix, IRPCA aims at learning the underlying projection matrix, which can be used to efficiently remove the possible corruptions in any datum. The learning is done by solving a nuclear-norm regularized minimization problem, which is convex and can be solved in polynomial time. Extensive experiments on a benchmark human face dataset and two video surveillance datasets show that IRPCA cannot only be robust to gross corruptions, but also handle the new data well and in an efficient way.
Cross-Domain Collaborative Learning via Discriminative Nonparametric Bayesian Model

Cross-domain data analysis has been becoming more and more important, and can be effectively adopted for many applications. However, it is difficult to propose a unified cross-domain collaborative learning framework for cross-domain analysis in social multimedia, because cross-domain data have multidomain, multimodal, sparse, and supervised properties. In this paper, we propose a generic cross-domain collaborative learning (CDCL) framework via a discriminative nonparametric Bayesian dictionary learning model for cross-domain data analysis. Compared with existing cross-domain learning methods, our proposed model mainly has four advantages: First, to address the domain discrepancy, we utilize the shared domain priors among multiple domains to make them share a common feature space. Second, to exploit the multimodal property, we use the shared modality priors to model the relationship between different modalities. Third, to deal with the sparse property of media data in one domain, our goal is to learn a shared dictionary to bridge different domains and complement each other. Finally, to make use of the supervised property, we exploit class label information to learn the shared discriminative dictionary, and utilize a latent probability vector to select different dictionary elements for representation of each class. Therefore, the proposed model can investigate the superiorities of different sources to supplement and improve each other effectively. In experiments, we have evaluated our model for two important applications including cross-platform event recognition and cross-network video recommendation. The experimental results have showed the effectiveness of our CDCL model for cross-domain analysis.
Weakly Supervised Graph Propagation Towards Collective Image Parsing

In this work, we propose a weakly supervised graph propagation method to automatically assign the annotated labels at image level to those contextually derived semantic regions. The graph is constructed with the over-segmented patches of the image collection as nodes. Image-level labels are imposed on the graph as weak supervision information over subgraphs, each of which corresponds to all patches of one image, and the contextual information across different images at patch level are then mined to assist the process of label propagation from images to their descendent regions. The ultimate optimization problem is efficiently solved by Convex Concave Programming (CCCP). Extensive experiments on four benchmark datasets clearly demonstrate the effectiveness of our proposed method for the task of collective image parsing. Two extensions including image annotation and concept map based image retrieval demonstrate the proposed image parsing algorithm can effectively aid other vision tasks.
On Analyzing the 'Variety' of Big Social Multimedia

Social media contributes significantly to the arrival of the Big Data era. It is interesting to re-examine "multimedia" in the context of social media. This positioning paper proposes to analyzing into Variety of big social multimedia from the perspective of various sources, which is the study of heterogeneous data generated and consumed in various social media networks that receives little attention in the literature. Since social multimedia is essentially "user-centric", which is generated from user and customized for user, a research schema is introduced to exploit the overlapped users as bridges to address challenges and exploit social multimedia data from various sources.
Partial Occlusion Handling for Visual Tracking via Robust Part Matching

Part-based visual tracking is advantageous due to its robustness against partial occlusion. However, how to effectively exploit the confidence scores of individual parts to construct a robust tracker is still a challenging problem. In this paper, we address this problem by simultaneously matching parts in each of multiple frames, which is realized by a locality-constrained low-rank sparse learning method that establishes multi-frame part correspondences through optimization of partial permutation matrices. The proposed part matching tracker (PMT) has a number of attractive properties. (1) It exploits the spatial-temporal locality-constrained property for robust part matching. (2) It matches local parts from multiple frames jointly by considering their low-rank and sparse structure information, which can effectively handle part appearance variations due to occlusion or noise. (3) The proposed PMT model has the inbuilt mechanism of leveraging multi-mode target templates, so that the dilemma of template updating when encountering occlusion in tracking can be better handled. This contrasts with existing methods that only do part matching between a pair of frames. We evaluate PMT and compare with 10 popular state-of-the-art methods on challenging benchmarks. Experimental results show that PMT consistently outperform these existing trackers.
Robust movie character identification and the sensitivity analysis

Automatic face identification of characters in movies has drawn significant research interests and led to various applications. It is a challenging problem due to the huge variation in the appearance of each character. Although existing methods demonstrate promising results in clean environment, the performances are limited in complex movie scenes due to the noises generated during the face tracking and face clustering process. In this paper we present a robust character identification approach by incorporating a noise insensitive relationship representation and a graph matching algorithm. Beyond existing character identification approaches, we further perform explicit sensitivity analysis on character identification by introducing two types of simulated noises. Experiments validate the advantage of the proposed method.
Effective Annotation and Search for Video Blogs with Integration of Context and Content Analysis

In recent years, weblogs (or blogs) have received great popularity worldwide, among which video blogs (or vlogs) are playing an increasingly important role. However, research on vlog analysis is still in the early stage, and how to manage vlogs effectively so that they can be more easily accessible is a challenging problem. In this paper, we propose a novel vlog management model which is comprised of automatic vlog annotation and user-oriented vlog search. For vlog annotation, we extract informative keywords from both the target vlog itself and relevant external resources; besides semantic annotation, we perform sentiment analysis on comments to obtain the overall evaluation. For vlog search, we present saliency-based matching to simulate human perception of similarity, and organize the results by personalized ranking and category-based clustering. An evaluation criterion is also proposed for vlog annotation, which assigns a score to an annotation according to its accuracy and completeness in representing the vlog's semantics. Experimental results demonstrate the effectiveness of the proposed management model for vlogs.
Personalized video recommendation based on cross-platform user modeling

Online propagation of videos has surged up to an unparalleled level. Most personalized video recommendation methods are based on single-platform user modeling, which suffer from data sparsity and cold-start issues. In this paper, we introduce cross-platform user modeling as a solution by smartly aggregating user information from different platforms. Unlike traditional recommendation methods where sufficient user information is assumed available in the target platform, this proposed method works well when there is little knowledge about users' interests in the target platform. While considering the difference of user behaviors in different platforms, on one hand, we enrich user profile in the target platform with related information in the auxiliary platform. On the other hand, we transfer the collaborative relationship defined in behaviors from the auxiliary platform to the target platform. Carefully designed experiments have demonstrated the effectiveness of the proposed method.
Design of the lifting mechanism with virtual prototyping technology

In the traditional design process, the design method of port machines mainly adopts the static design and has no effective dynamic technique. The technique of virtual prototype can factually simulate all kinds of running states of each mechanism under various working conditions which vary from different loads and in all kinds of processes. The results of the emulation are used for estimating and judging the data of actual movement. It also offers certain value for reference for the dynamic analysis and the calculation of port machinery, and provides technical support for the creative design and the optimization of crane, and plays an important part in expediting the development cycle and reducing the cost. ADAMS is a famous virtual prototyping modeling and analysis software. This paper introduces the application of the virtual prototyping technology based on ADAMS software which is used in the research process of the lifting mechanism of the gantry crane. The related dynamics models were established and the dynamics loads were conformed under different working conditions according to crane dynamics theories. Interrelated data, which is got in the simulation analyses, offers effective methods on dynamic design of integrated system of the lifting mechanism. It will shorten the design period of the gantry crane and will improve the efficiency and the quality of the design. The virtual prototype study on integrating system of the lifting mechanism leads to an efficient tool and a method for designing gantry crane which is economic, credible, stable and high-powered. This design method will be extensively applied in mechanism design.
Cross-Space Affinity Learning with Its Application to Movie Recommendation

In this paper, we propose a novel cross-space affinity learning algorithm over different spaces with heterogeneous structures. Unlike most of affinity learning algorithms on the homogeneous space, we construct a cross-space tensor model to learn the affinity measures on heterogeneous spaces subject to a set of order constraints from the training pool. We further enhance the model with a factorization form which greatly reduces the number of parameters of the model with a controlled complexity. Moreover, from the practical perspective, we show the proposed factorized cross-space tensor model can be efficiently optimized by a series of simple quadratic optimization problems in an iterative manner. The proposed cross-space affinity learning algorithm can be applied to many real-world problems, which involve multiple heterogeneous data objects defined over different spaces. In this paper, we apply it into the recommendation system to measure the affinity between users and the product items, where a higher affinity means a higher rating of the user on the product. For an empirical evaluation, a widely used benchmark movie recommendation data set-MovieLens-is used to compare the proposed algorithm with other state-of-the-art recommendation algorithms and we show that very competitive results can be obtained.
User-Aware Image Tag Refinement via Ternary Semantic Analysis

Large-scale user contributed images with tags are easily available on photo sharing websites. However, the noisy or incomplete correspondence between the images and tags prohibits them from being leveraged for precise image retrieval and effective management. To tackle the problem of tag refinement, we propose a method of Ranking based Multi-correlation Tensor Factorization (RMTF), to jointly model the ternary relations among user, image, and tag, and further to precisely reconstruct the user-aware image-tag associations as a result. Since the user interest or background can be explored to eliminate the ambiguity of image tags, the proposed RMTF is believed to be superior to the traditional solutions, which only focus on the binary image-tag relations. During the model estimation, we employ a ranking based optimization scheme to interpret the tagging data, in which the pair-wise qualitative difference between positive and negative examples is used, instead of the point-wise 0/1 confidence. Specifically, the positive examples are directly decided by the observed user-image-tag interrelations, while the negative ones are collected with respect to the most semantically and contextually irrelevant tags. Extensive experiments on a benchmark Flickr dataset demonstrate the effectiveness of the proposed solution for tag refinement. We also show attractive performances on two potential applications as the by-products of the ternary relation analysis.