nils-herrmann/clean_scirepeval huggingface dataset:
Meta-Path based Nonnegative Matrix Factorization for clustering on multi-type relational data

Clustering on multi-type relational data has attracted increasing interest due to its great practical and theoretical importance. One of the most popular solutions is nonnegative matrix factorization. However, previous work on nonnegative matrix factorization typically copes with multi-type relations individually, and ignores the underlying semantics conveyed by the relation propagation. Additionally, these approaches may suffer from data sparsity as most of the relations between object pairs are unknown. In this paper we propose a novel Meta-Path based Nonnegative Matrix Factorization (MPNMF) framework, which enriches potentially useful similarity semantics for the improved clustering performance. We begin with constructing meta-paths, i.e., paths that connects object types via a sequence of relations, which are appropriately weighted according to certain propagation decay rules. Based on the weighted meta-paths, we are promised to characterize the strength of pairwise interactions among the objects. Together with the attributes in the bag-of-word form, we cluster the objects of target type by collective nonnegative matrix factorization. Experiments on real world datasets demonstrate the effectiveness of our method.
Research on spatial non-homogenous Kalman filter and its application in cephalometric image

In the view of problems by processing the lateral cephalometric films in the automatic cephalometric analysis system, a noise-removing Kalman filter in spatial non-homogeneous images is proposed. The noise is effectively reduced and the edges in the image are preserved by adding information about the edges to the input of the filter. This approach can avoid the computational burden caused by the model parameter identification.
The security flaws in some authentication watermarking schemes

Watermarking technology was originally proposed for copyright protection. Recently it has been applied to media authentication so that a proof of authenticity is inserted into the media instead of being appended to the media as a separated attachment. However, security requirements of the authentication are overlooked in some authentication watermark schemes. In this paper we analyze three authentication watermarking schemes and point out their security flaws. The first scheme is the color authentication scheme in [S.C. Byun et al., 2002]. The scheme is not secure in the sense that as long as an attacker obtains one authenticated image, he is able to forge authentic images without the secret key. The second scheme [Ping Wah Wing, et al., 2001] is an authentication scheme but it is extended for ownership incorrectly. The third one, the robust invertible watermarking scheme [J. Friedrich et al., 2002], employs a multiple of secret random sequences to produce a watermark. However these sequences are independent of the original images, i.e., they remain invariable for different images. An adversary, having sufficient number of original images, can reconstruct the secret sequences by solving simultaneous equations. With these reconstructed sequences, the attacker can forge authentic image freely. The attack can be thwarted with content related sequences generated from both the secret key and the original image.
Relational User Attribute Inference in Social Media

Nowadays, more and more people are engaged in social media to generate multimedia information, i.e, creating text and photo profiles and posting multimedia messages. Such multimodal social networking activities reveal multiple user attributes such as age, gender, and personal interest. Inferring user attributes is important for user profiling, retrieval, and personalization. Existing work is devoted to inferring user attributes independently and ignores the dependency relations between attributes. In this work, we investigate the problem of relational user attribute inference by exploring the relations between user attributes and extracting both lexical and visual features from online user-generated content. We systematically study six types of user attributes: gender, age, relationship, occupation, interest, and emotional orientation. In view of methodology, we propose a relational latent SVM (LSVM) model to combine a rich set of user features, attribute inference, and attribute relations in a unified framework. In the model, one attribute is selected as the target attribute and others are selected as the auxiliary attributes to assist the target attribute inference. The model infers user attributes and attribute relations simultaneously. Extensive experiments conducted on a collected dataset from Google+ with full attribute annotations demonstrate the effectiveness of the proposed approach in user attribute inference and attribute-based user retrieval.
Research on adaptive filtering of colored noise and its application

In the view of the drawbacks of conventional Kalman filtering, a kind of adaptive Kalman filtering approach for colored noise based on the correlative method of the system output is proposed to solve the cephalometric images of stomatology. This approach builds a Kalman filtering model of colored noise by adopting the equivalent measurement equation in order to avoid complicated computation and the expansion of the dimension of the filter. It is also unnecessary to know the variance of measurement noise beforehand so that it is closer to the actual situation. Results of several experiments are presented to demonstrate the feasibility and good performance of this approach.
Adaptive edge detecting approach based on scale-space theory

The main problem existing in many edge detecting approaches is that they are sensitive to the noise. There is a conflict between the precision of edge detection and the effect of the noise removal. The scale-space theory on the basis of the optimal edge detecting theory is introduced in this paper. Each specific edge corresponds to an optimal filter scale (OFS) and the value of the OFS of each edge point is calculated based on the optimal edge detecting theory. An automatic threshold is introduced in the process of edge detection in order to eliminate the ragged edge further. Based on the optimal scale and the automatic threshold, a fast adaptive edge detecting algorithm is proposed. Experiment results demonstrate that this algorithm can get either the noise removing effect of low-resolution filter or the edge detecting precision of high-resolution filter and make a better compromise between the precision of edge detection and the effect of the noise removal.
A Standalone Demo for Quiz Game "Describe and Guess"

The popular quiz game "Describe and Guess" can only be played by at least two players which limits its utilization. In this paper, we propose a standalone demo with a multimodal interface which is much more convenient for a single player. To deduce the target object based on user-provided attributes and sketches, we propose attributeimproved neural networks (AINN) which are joint learned for both the single-label object classification and the multilabel attribute prediction. Finally, we build a real-world system which can be effectively played on mobile devices.
Automatic music classification and summarization

Automatic music classification and summarization are very useful to music indexing, content-based music retrieval and on-line music distribution, but it is a challenge to extract the most common and salient themes from unstructured raw music data. In this paper, we propose effective algorithms to automatically classify and summarize music content. Support vector machines are applied to classify music into pure music and vocal music by learning from training data. For pure music and vocal music, a number of features are extracted to characterize the music content, respectively. Based on calculated features, a clustering algorithm is applied to structure the music content. Finally, a music summary is created based on the clustering results and domain knowledge related to pure and vocal music. Support vector machine learning shows a better performance in music classification than traditional Euclidean distance methods and hidden Markov model methods. Listening tests are conducted to evaluate the quality of summarization. The experiments on different genres of pure and vocal music illustrate the results of summarization are significant and effective.
Cephalometric image measurement and prediction system

A practical automatic cephalometric measurement and facial prediction system is built by using adaptive filtering and edge detecting techniques and some artificial intelligent approaches. The system not only realizes the automation from landmark location to parameter measurement on the cephalogram, but also makes the preoperative diagnostic design of orthognathic surgery quantitive. Compared with manual prediction, it provides a new method to enable the diagnosis and treatment of dento-maxillofacial deformities more accurately. It is significant for clinical application and scientific research.
Robust soccer highlight generation with a novel dominant-speech feature extractor

We describe soccer highlight generation from only the audio stream in the video. A novel audio feature is used to detect parts of the commentary corresponding to dominant and excited speech. It is computed by a twice-iterated composite Fourier transform (CFT) on short-time windows, wherein the magnitude spectrum of the first transform is input to a second transform. Dominant speech portions are found to be robustly characterized by increased density in the peak profile. We verify the robustness of CFT via large scale empirical testing and explain its working based on a pulse train postulate of dominant speech signals. Our audio-only approach results in a compute-efficient system deployable on current generation set-top-boxes and digital video recording devices
The Demo: A Real-Time Score Detection and Recognition Approach in Broadcast Basketball Sports Video

For broadcast sports video, score information is an effective mid-level representation to facilitate high-level video content analysis. In this paper, we propose a real-time approach to detect score region and recognize scores in broadcast basketball sports video. The flow chart of our proposed approach has two major modules: score detection and score recognition. Using our approach, we can locate score regions automatically and reliably without detecting all the texts in the frame, and recognize the scores without training process. The running speed of our algorithm is fast.
Extracting Key Sub-trajectory Features for Supervised Tactic Detection in Sports Video

Tactic analysis is receiving more attention in sports video analysis for its assistance to coaches and players. This paper proposes an efficient key sub-trajectory feature representation of ball trajectory for tactic analysis. Ball trajectories are modeled with the generalized suffix tree where frequent sub-trajectory patterns are searched for. Key sub-trajectory patterns are extracted by further filtering these frequent sub-trajectory patterns. Instead of directly using individual sub-trajectories as features to train tactic detectors, we take key sub-trajectory patterns as a whole. Key sub-trajectory feature representation effectively removes noise, reduces the dimension of features, and improves the performance of supervised learning to detect tactics.
Low-Rank Sparse Coding for Image Classification

In this paper, we propose a low-rank sparse coding (LRSC) method that exploits local structure information among features in an image for the purpose of image-level classification. LRSC represents densely sampled SIFT descriptors, in a spatial neighborhood, collectively as low-rank, sparse linear combinations of code words. As such, it casts the feature coding problem as a low-rank matrix learning problem, which is different from previous methods that encode features independently. This LRSC has a number of attractive properties. (1) It encourages sparsity in feature codes, locality in codebook construction, and low-rankness for spatial consistency. (2) LRSC encodes local features jointly by considering their low-rank structure information, and is computationally attractive. We evaluate the LRSC by comparing its performance on a set of challenging benchmarks with that of 7 popular coding and other state-of-the-art methods. Our experiments show that by representing local features jointly, LRSC not only outperforms the state-of-the-art in classification accuracy but also improves the time complexity of methods that use a similar sparse linear representation model for feature coding.
Efficient multimodal features for automatic soccer highlight generation

We describe efficient audio/visual features and their multimodal combination to detect highlights in soccer video. A novel audio feature first detects dominant speech portions in the commentary coincident with segments of high excitement in the game. Verification is then performed in the visual domain by detecting the presence of goal-mouth in the current shot and a high frequency of camera shot change in the subsequent shots. The cascaded process filters spurious candidate highlights from the noisy audio. The impressive results obtained on a large video test-set belie the technical simplicity in the system, which may now enable rapid generation of highlights on low-cost devices such as household set-top-boxes.
Character Identification in Feature-Length Films Using Global Face-Name Matching

Identification of characters in films, although very intuitive to humans, still poses a significant challenge to computer methods. In this paper, we investigate the problem of identifying characters in feature-length films using video and film script. Different from the state-of-the-art methods on naming faces in the videos, most of which used the local matching between a visible face and one of the names extracted from the temporally local video transcript, we attempt to do a global matching between names and clustered face tracks under the circumstances that there are not enough local name cues that can be found. The contributions of our work include: 1) A graph matching method is utilized to build face-name association between a face affinity network and a name affinity network which are, respectively, derived from their own domains (video and script). 2) An effective measure of face track distance is presented for face track clustering. 3) As an application, the relationship between characters is mined using social network analysis. The proposed framework is able to create a new experience on character-centered film browsing. Experiments are conducted on ten feature-length films and give encouraging results.
Mining Semantic Context Information for Intelligent Video Surveillance of Traffic Scenes

Automated visual surveillance systems are attracting extensive interest due to public security. In this paper, we attempt to mine semantic context information including object-specific context information and scene-specific context information (learned from object-specific context information) to build an intelligent system with robust object detection, tracking, and classification and abnormal event detection. By means of object-specific context information, a cotrained classifier, which takes advantage of the multiview information of objects and reduces the number of labeling training samples, is learned to classify objects into pedestrians or vehicles with high object classification performance. For each kind of object, we learn its corresponding semantic scene-specific context information: motion pattern, width distribution, paths, and entry/exist points. Based on this information, it is efficient to improve object detection and tracking and abnormal event detection. Experimental results demonstrate the effectiveness of our semantic context features for multiple real-world traffic scenes.
Correlation Particle Filter for Visual Tracking

In this paper, we propose a novel correlation particle filter (CPF) for robust visual tracking. Instead of a simple combination of a correlation filter and a particle filter, we exploit and complement the strength of each one. Compared with existing tracking methods based on correlation filters and particle filters, the proposed tracker has four major advantages: 1) it is robust to partial and total occlusions, and can recover from lost tracks by maintaining multiple hypotheses; 2) it can effectively handle large-scale variation via a particle sampling strategy; 3) it can efficiently maintain multiple modes in the posterior density using fewer particles than conventional particle filters, resulting in low computational cost; and 4) it can shepherd the sampled particles toward the modes of the target state distribution using a mixture of correlation filters, resulting in robust tracking performance. Extensive experimental results on challenging benchmark data sets demonstrate that the proposed CPF tracking algorithm performs favorably against the state-of-the-art methods.
Cross-Domain Feature Learning in Multimedia

In the Web 2.0 era, a huge number of media data, such as text, image/video, and social interaction information, have been generated on the social media sites (e.g., Facebook, Google, Flickr, and YouTube). These media data can be effectively adopted for many applications (e.g., image/video annotation, image/video retrieval, and event classification) in multimedia. However, it is difficult to design an effective feature representation to describe these data because they have multi-modal property (e.g., text, image, video, and audio) and multi-domain property (e.g., Flickr, Google, and YouTube). To deal with these issues, we propose a novel cross-domain feature learning (CDFL) algorithm based on stacked denoising auto-encoders. By introducing the modal correlation constraint and the cross-domain constraint in conventional auto-encoder, our CDFL can maximize the correlations among different modalities and extract domain invariant semantic features simultaneously. To evaluate our CDFL algorithm , we apply it to three important applications: sentiment classification, spam filtering, and event classification. Comprehensive evaluations demonstrate the encouraging performance of the proposed approach.
Attribute-Assisted Domain Transfer from Image to Sketch

Data-inefficiency greatly hinders sketch-related applications and studies. Transferring knowledge from the realimage domain to the sketch domain may eliminate this obstacle. However, a huge domain gap exists between images and sketches. To reduce the domain shift between the image and the sketch, we propose an attribute-assisted domain transfer method. By separating the shared geometric features from the private semantic features of the real images, the proposed attribute-assisted networks (ASN) can learn more effective domain-invariant features for unsupervised domain adaptation. Extensive results on the Imageto- Sketch task demonstrate the effectiveness of the proposed method.
Video based 3D reconstruction using spatio-temporal attention analysis

3D reconstruction has been widely used in many important applications. While extensive research has been done in 3D reconstruction, several key issues are still open and the precision of the recovered regions is still far from satisfaction. In this paper, we propose a novel approach to selecting regions of interest in video frames by analyzing multiple spatio-temporal characteristics and reconstructing 3D objects based on the selected regions. Firstly, the static, location and motion attention are extracted from video frames to generate saliency maps. Then, all the video frames are clustered and a candidate set of key frames is extracted based on the saliency maps, where the key frames are extracted according to the constraints in terms of geometry and visibility. Finally, the 3D structure of the attention region is recovered using the selected key frames and the generated saliency maps. The experiments on real-world indoor and outdoor scenes demonstrate that the proposed approach is both more accurate (better attention regions) and computationally more efficient.
Interaction Design for Mobile Visual Search

Mobile devices are becoming ubiquitous. People take pictures via their phone cameras to explore the world on the go. In many cases, they are concerned with the picture-related information. Understanding user intent conveyed by those pictures therefore becomes important. Existing mobile applications employ visual search to connect the captured picture with the physical world. However, they only achieve limited success due to the ambiguity nature of user intent in the picture-one picture usually contains multiple objects. By taking advantage of multitouch interactions on mobile devices, this paper presents a prototype of interactive mobile visual search, named TapTell, to help users formulate their visual intent more conveniently. This kind of search leverages limited yet natural user interactions on the phone to achieve more effective visual search while maintaining a satisfying user experience. We make three contributions in this work. First, we conduct a focus study on the usage patterns and concerned factors for mobile visual search, which in turn leads to the interactive design of expressing visual intent by gesture. Second, we introduce four modes of gesture-based interactions (crop, line, lasso, and tap) and develop a mobile prototype. Third, we perform an in-depth usability evaluation on these different modes, which demonstrates the advantage of interactions and shows that lasso is the most natural and effective interaction mode. We show that TapTell provides a natural user experience to use phone camera and gesture to explore the world. Based on the observation and conclusion, we also suggest some design principles for interactive mobile visual search in the future.
Multi-view multi-label active learning for image classification

Image classification is an important topic in multimedia analysis, among which multi-label image classification is a very challenging task with respect to the large demand for human annotation of multi-label samples. In this paper, we propose a multi-view multi-label active learning strategy, which integrates the mechanism of active learning and multi-view learning. On one hand we explore the sample and label uncertainties within each view; on the other hand we capture the uncertainty over different views based on multi-view fusion. Then the overall uncertainty along the sample, label and view dimensions are obtained to detect the most informative sample-label pairs. Experimental results demonstrate the effectiveness of the proposed scheme.
Deep-Structured Event Modeling for User-Generated Photos

Vision-based event analysis is difficult because of the following challenges. The first challenge is intraclass variation. Photos uploaded by users are sparsely sampled visual appearances of an event over time. Thus, each photo may only capture a single object or scene of a specific complex event. The second challenge is interclass confusion. Photos related to different events may contain similar objects or scenes. Third, unusual events are characterized by scarcity, and only a few samples are available for use in learning event patterns. In this paper, by considering the photo timestamp, we propose a structured event modeling (SEM) framework for event analysis that exploits the temporal information of visual features and event classes in a photo sequence. Specifically, the temporal event patterns of the photo sequence and the relationships of different photos are jointly learned using deep neural networks (convolutional neural networks and recurrent neural networks) and a conditional random field. We evaluate the proposed SEM framework in two applications: multiclass event recognition and unusual event detection in photo sequences. The results of extensive experiments performed on a public event recognition dataset and a collected unusual event dataset demonstrate the effectiveness of the proposed method.