nils-herrmann/clean_scirepeval huggingface dataset:
Scale-Adaptive Convolutions for Scene Parsing

Many existing scene parsing methods adopt Convolutional Neural Networks with fixed-size receptive fields, which frequently result in inconsistent predictions of large objects and invisibility of small objects. To tackle this issue, we propose a scale-adaptive convolution to acquire flexiblesize receptive fields during scene parsing. Through adding a new scale regression layer, we can dynamically infer the position-adaptive scale coefficients which are adopted to resize the convolutional patches. Consequently, the receptive fields can be adjusted automatically according to the various sizes of the objects in scene images. Thus, the problems of invisible small objects and inconsistent large-object predictions can be alleviated. Furthermore, our proposed scale-adaptive convolutions are not only differentiable to learn the convolutional parameters and scale coefficients in an end-to-end way, but also of high parallelizability for the convenience of GPU implementation. Additionally, since the new scale regression layers are learned implicitly, any extra training supervision of object sizes is unnecessary. Extensive experiments on Cityscapes and ADE20K datasets well demonstrate the effectiveness of the proposed scaleadaptive convolutions.
Motion region-based trajectory analysis and re-ranking for video retrieval

Event-related query is playing a more and more important role in video retrieval. However, it is still a challenge to the existing video retrieval engines for lacking the effective motion analysis. In this paper, we propose a novel re-ranking scheme for video retrieval based on motion region trajectory analysis. By focusing on the changes of the primary moving regions, we construct an intuitive motion region-based trajectory descriptor (MRTD) to depict the shot activities. In the re-ranking phase, the proposed approach takes the MRTD as a motion cue and re-ranks the baseline results by motion-related query selection and MRTD-based weighting. We evaluate our method in TRECVID2007 and 2008 datasets, and observe consistent improvement over all the baselines, leading to a greatest performance gain of 42.9%, and an average gain of 17%. The experiments also show that the motion descriptor of MRTD is fruitful for a variety of features.
Parallel deblocking filter for HEVC on many-core processor

High-efficiency video coding (HEVC) is the next generation standard of video coding. The deblocking filter (DF) constitutes a significant part of the HEVC decoder complexity. A three-step parallel framework (TPF) is proposed for the H.264/AVC DF, which is also suitable for HEVC except the third step. The third step of the TPF is replaced with a directed acyclic graph-based order. Experiments show that the proposed method dramatically accelerates more than the state-of-the-art parallel method.
News Credibility Evaluation on Microblog with a Hierarchical Propagation Model

Benefiting from its openness, collaboration and real-time features, Micro blog has become one of the most important news communication media in modern society. However, it is also filled with fake news. Without verification, such information could spread promptly through social network and result in serious consequences. To evaluate news credibility on Micro blog, we propose a hierarchical propagation model. We detect sub-events within a news event to describe its detailed aspects. Thus, for a news event, a three-layer credibility network consisting of event, sub-events and messages can represent it from different scale and reveal vital information for credibility evaluation. After linking these entities with their semantic and social associations, the credibility value of each entity is propagated on this network to achieve the final evaluation result. By formulating this propagation process as a graph optimization problem, we provide a globally optimal solution with an iterative algorithm. Experiments conducted on two real-world datasets show that the proposed model boosts the accuracy by more than 6% and the F-score by more than 16% over a baseline method.
Improved Fractional Pixel Motion Estimation Algorithm for H.264/AVC

In H.264/AVC fractional pixel motion compensation is adopted in order to achieve more accurate motion description and higher compression efficiency. It, however, remarkably increased computation load in motion estimation (ME) process for video coder. In this paper an improved fractional pixel ME algorithm is proposed to reduce its computation load, which is based on the fact that the distribution of the fractional motion vector (MV) is centered on the best integer pixel. Experiment results show that our proposed algorithm can reduce 75% computation load at most and can also keep rate distortion performance as compared with full fractional ME method used in H.264 video coder
Highly Parallel Framework for HEVC Motion Estimation on Many-Core Platform

As the next generation standard of video coding, High Efficiency Video Coding (HEVC) is expected to be more complex than H.264/AVC. Many-core platforms are good candidates for speeding up HEVC in the case that HEVC can provide sufficient parallelism. The local parallel method (LPM) is the most promising parallel proposal for HEVC motion estimation (ME), but it can't provide sufficient parallelism for many-core platforms. On the premise of keeping the data dependencies and coding efficiency the same as the LPM, we propose a highly parallel framework to exploit the implicit parallelism. Compared with the well-known LPM, experiments conducted on a 64-core system show that our proposed method achieves averagely more than 10 and 13 times speedup for 1920×1080 and 2560×1600 video sequences, respectively.
Motion adaptive deinterlacing of video data with texture detection

In this paper, we propose a motion adaptive deinterlacing method with texture detection for scan-rate conversion of video data. The basic idea of this method is to classify the missing pixel to four different regions including motion smooth region, motion texture region, static smooth region, and static texture region based on the results of motion detection and texture detection, then four different interpolation methods are used to produce the missing pixel. Extensive simulations conducted for video sequences indicate that the performance of the proposed method is superior to the previous methods based on the direct merging method, bilinear interpolation, ELA, and vertical temporal median method. Moreover, our proposed method requires lower computational complexity and provides better object and subject performance
An Innovative Tempo Model for Movie Content Analysis

The paper presents a novel tempo model for movie content analysis. We originally propose that tempo indicates the rhythm of both movie scenarios and human perception and focus on the low level features extraction to represent both aspects. By thoroughly analyzing them, we classify the factors of tempo into two sorts. The first is based on the film grammar and we use the low level features of shot length and camera motion to describe filmmaking by directors. The second is based on the human perception and the low level features of motion intensity, motion complexity, audio energy and audio pace are integrated for the formulation of information to describe the viewers' emotional changes to continuously developing storyline. With both factors, tempo is defined and tempo flow plot is derived as the clue of storyline. Then we implement the tempo model for Action scene detection and build a system, SmartMovirPlayer, for hierarchical browse and edit with action concept annotation. The large-scale experiments demonstrate the effectiveness of the tempo model for movie content analysis.
Efficient Parallel Framework for HEVC Deblocking Filter on Many-Core Platform

Many-core platforms are good candidates for speeding up High Efficiency Video Coding (HEVC) in the case that HEVC can provide sufficient parallelism. As the most promising proposal for parallelizing HEVC deblocking filter (DF), the order-changed parallel method (OCPM) changes the order of filtering and incurs considerable loss in coding efficiency. Meanwhile, the parallelism of OCPM still has some room for improvement. In this paper, we propose an efficient parallel framework for HEVC DF, which exploits the implicit parallelism and keeps the filtering order of DF unchanged. Compared with the well-known OCPM, experiments conducted on a 64-core system show that our proposed method saves averagely 37.18% and 37.93% DF time with different quantization parameters (QPs). Meanwhile, our proposed method improves coding efficiency, which achieves an average BD-rate reduction of 0.09%, 0.11% and 0.12% for Y, U and V components, respectively.
Context-Aware Architecture for Intelligent Application Services in Ubiquitous Computing

In this paper, we design a context-aware architecture for dealing with intelligent application services in ubiquitous computing. The context-aware architecture is composed of middleware, context server, and client. The middleware component of our context-aware architecture plays an important role in recognizing a moving node with mobility by using a Bluetooth wireless communication technology as well as in executing an appropriate execution module according to the context acquired from a context server. The context server functions as a manager that efficiently stores into the database server context information, such as user's current status, physical environment, and resources of a computing system. To verify the usefulness of our architecture, we finally develop a context-aware application system base on it, which provides users with a music playing service in ubiquitous computing environment.
Entropy Coding Based On Code-Value Compact Memory Structure for AVS Video Coding Standard

Entropy coding based on k-th order Exp-Golomb (EGk) codes is a key part in the new AVS video coding standard issued by Audio Video Coding Standard Workgroup of China. An efficient design based on code-value compact memory structure (CVCMS) is proposed to reduce the computational complexity and memory requirement. Only 789 byte memory is required for variable length coding (VLC) tables in CVCMS, which is just about 5.92% compared with that of the reference software. Furthermore, code-value is stored in memory, which reduced the computational complexity of EGk coding. The simulation results show that the proposed entropy coding for AVS video coding standard reduces the computational cost by 26.48%
Multi-Modality Transfer Based on Multi-Graph Optimization for Domain Adaptive Video Concept Annotation

Multi-modality, the unique and important property of video data, is typically ignored in existing video adaptation processes. To solve this problem, we propose a novel approach, named multi-modality transfer based on multi- graph optimization (MMT-MGO) in this paper, which leverages multi-modality knowledge generalized by auxiliary classifiers in the source domain to assist multi-graph optimization (a graph-based semi-supervised learning method) in the target domain for video concept annotation. To our best knowledge, it is the first time to introduce multi-modality transfer into domain adaptive video concept detection and annotation. Moreover, we propose an efficient incremental extension scheme to sequentially estimate a small batch of new emerging data without modifying the structure of multi-graph scheme. The proposed scheme can achieve a comparable accuracy with that of the brand-new round optimization which combines these data with the data corpus for the nearest round optimization, while the time for estimation has been greatly reduced. Extensive experiments over TRECVID2005 and 2007 data sets demonstrate the effectiveness of both the multi-modality transfer scheme and the incremental extension scheme.
Human skin detection in images by MSER analysis

Human skin detection in images is desirable in many practical applications, e.g., adult-content filtering. However, existing methods are mainly pixel-based and ignore that human skin is region-based. In this paper, we introduce a successful region detector, i.e., MSER, into the skin detection by regarding the skin region as the maximally stable extremal region (MSER). We extend the original MSER to both color and texture analysis to reduce the skinlike regions1. Furthermore, to be adaptive to the dynamic illumination and chrominance, face detection is used to customize the skin color model to each image. The proposed method has achieved promising performance over our dataset, which is a challenging set with a great part of hard images. Our True Positive Rate is 81.2% under False Positive Rate 8.2%, which outperforms all of eight state-of-the-art algorithms.
Effective Uyghur Language Text Detection in Complex Background Images for Traffic Prompt Identification

Text detection in complex background images is a challenging task for intelligent vehicles. Actually, almost all the widely-used systems focus on commonly used languages while for some minority languages, such as the Uyghur language, text detection is paid less attention. In this paper, we propose an effective Uyghur language text detection system in complex background images. First, a new channel-enhanced maximally stable extremal regions (MSERs) algorithm is put forward to detect component candidates. Second, a two-layer filtering mechanism is designed to remove most non-character regions. Third, the remaining component regions are connected into short chains, and the short chains are extended by a novel extension algorithm to connect the missed MSERs. Finally, a two-layer chain elimination filter is proposed to prune the non-text chains. To evaluate the system, we build a new data set by various Uyghur texts with complex backgrounds. Extensive experimental comparisons show that our system is obviously effective for Uyghur language text detection in complex background images. The F-measure is 85%, which is much better than the state-of-the-art performance of 75.5%.
Improvements on Rate-Distortion Performance of H.264 Rate Control in Low Bit Rate Video Coding

This paper points out some defects in the techniques used in H.264 rate control and presents two new techniques to improve them. The improved scheme has the following main features: 1) the bits allocated to each P-frame is proportional to the local motion in it, i.e., more bits are allocated to a frame if the local motion in it is stronger; 2) the quantization parameter (QP) calculation is based on a simple encoding complexity prediction scheme, which is more robust and simple than the quadratic model used by H.264 in low bit rate video coding. Simulation results show that compared to rate control scheme in H.264, the improved scheme has significantly improved R-D performance (up to 1.29 dB)
Efficient Parallel Framework for H.264/AVC Deblocking Filter on Many-Core Platform

The H.264/AVC deblocking filter is becoming the performance bottleneck of H.264/AVC parallelization on many-core platform. Efficient parallelization of the deblocking filter on a many-core platform is challenging, because the deblocking filter has complicated data dependencies, which provide insufficient parallelism for so many cores. Furthermore, parallelization may have significant synchronization and load imbalance overhead. At present, research on the parallelizing deblocking filter on a many-core platform is rare and focuses on data-level parallelization. In this paper, we propose a three-step framework considering task-level segmentation and data-level parallelization to efficiently parallelize the deblocking filter. First, we review the entire deblocking filter process in 4 × 4 block edge-level and divide it into two parts: 1) boundary strength computation (BSC) and 2) edge discrimination and filtering (EDF), which increases the parallelism. Then, we apply the Markov empirical transition probability matrix and Huffman tree (METPMHT) to the BSC, which alleviate the load imbalance problem. Finally, we use an independent pixel connected area parallelization (IPCAP) for the EDF, which increases the parallelism and reduces the synchronization. In experiments, we apply our parallel method to the deblocking filter of the H.264/AVC reference software JM15.1 on the Tile64 platform without any Tile64 platform-based optimizations. Compared to the well-known 2D-wavefront method, the proposed method achieves on average 14.85, 17.83, and 10.60 times speed-up for QCIF, CIF, and HD videos using 62 cores, respectively.
Web Image Search Re-Ranking With Click-Based Similarity and Typicality

In image search re-ranking, besides the well-known semantic gap, intent gap, which is the gap between the representation of users' query/demand and the real intent of the users, is becoming a major problem restricting the development of image retrieval. To reduce human effects, in this paper, we use image click-through data, which can be viewed as the implicit feedback from users, to help overcome the intention gap, and further improve the image search performance. Generally, the hypothesis-visually similar images should be close in a ranking list-and the strategy-images with higher relevance should be ranked higher than others-are widely accepted. To obtain satisfying search results, thus, image similarity and the level of relevance typicality are determinate factors correspondingly. However, when measuring image similarity and typicality, conventional re-ranking approaches only consider visual information and initial ranks of images, while overlooking the influence of click-through data. This paper presents a novel re-ranking approach, named spectral clustering re-ranking with click-based similarity and typicality. First, to learn an appropriate similarity measurement, we propose click-based multi-feature similarity learning algorithm, which conducts metric learning based on click-based triplets selection, and integrates multiple features into a unified similarity space via multiple kernel learning. Then, based on the learnt click-based image similarity measure, we conduct spectral clustering to group visually and semantically similar images into same clusters, and get the final re-rank list by calculating click-based clusters typicality and within-clusters click-based image typicality in descending order. Our experiments conducted on two real-world query-image data sets with diverse representative queries show that our proposed re-ranking approach can significantly improve initial search results, and outperform several existing re-ranking approaches.
Efficient Parallel Framework for HEVC Motion Estimation on Many-Core Processors

High Efficiency Video Coding (HEVC) provides superior coding efficiency than previous video coding standards at the cost of increasing encoding complexity. The complexity increase of motion estimation (ME) procedure is rather significant, especially when considering the complicated partitioning structure of HEVC. To fully exploit the coding efficiency brought by HEVC requires a huge amount of computations. In this paper, we analyze the ME structure in HEVC and propose a parallel framework to decouple ME for different partitions on many-core processors. Based on local parallel method (LPM), we first use the directed acyclic graph (DAG)-based order to parallelize coding tree units (CTUs) and adopt improved LPM (ILPM) within each CTU (DAGILPM), which exploits the CTU-level and prediction unit (PU)-level parallelism. Then, we find that there exist completely independent PUs (CIPUs) and partially independent PUs (PIPUs). When the degree of parallelism (DP) is smaller than the maximum DP of DAGILPM, we process the CIPUs and PIPUs, which further increases the DP. The data dependencies and coding efficiency stay the same as LPM. Experiments show that on a 64-core system, compared with serial execution, our proposed scheme achieves more than 30 and 40 times speedup for 1920 × 1080 and 2560 × 1600 video sequences, respectively.
Scalable logo recognition based on compact sparse dictionary for mobile devices

In this paper, we present a novel scalable logo recognition system which can recognize a large number of logo categories locally on mobile devices. The system is unsupervised without any supervised training procedure, and very time efficient at low memory cost. It is also robust against challenging conditions such as noise addition, different image scale, rotation, etc. To achieve this goal, we propose an efficient segmental quantization approach for generation of large visual words over one million size with a very compact vocabulary. The vocabulary consists of two small dictionaries learned through sparse non-negative matrix factorization (NMF) of local SIFT descriptors. With an inverted index structure built through the large visual words, query images containing logos can be recognized through efficient retrieval of K-nearest neighbors (K-NN) of logo instances in the dataset. Our vocabulary size is very small, only one thousandth of that of traditional Approximate K-Means (AKM) method, which is of great importance for mobile devices with limited memory. Furthermore, based on the compact dictionary, we present a promising verification way of filtering false positives via sparse reconstruction of SIFT descriptors with a very few number of sparse codes due to the sparsity's property of lowest reconstruction error. Experiments on our dataset with 400 logo classes show that our system is very efficient and effective.
Automatic Detection and Analysis of Player Action in Moving Background Sports Video Sequences

This paper presents a system for automatically detecting and analyzing complex player actions in moving background sports video sequences, aiming at action-based sports videos indexing and providing kinematic measurements for coach assistance and performance improvement. The system works in a coarse-to-fine fashion. For an input video, in the coarse granularity level, we automatically segment the highlights, that is, the video clips containing the desired action as summaries for general user viewing purposes; in the middle granularity level, we recognize the action types to support action-based video indexing and retrieval; and finally in the fine granularity level, the critical kinematic parameters of player action are obtained for sports professionals' training purposes. However, the complex and dynamic background of sports videos and the complexity of player actions bring considerable difficulty to the automatic analysis. To fulfill such a challenging task, robust algorithms including global motion estimation with adaptive outliers filtering, object segmentation based on adaptive background construction, and automatic human body tracking are proposed in this paper. Two visual analyzing tools: motion panorama and overlay composition, are also introduced. Real diving and jump game videos are used to test the proposed system and algorithms, and the extensive and encouraging experimental results show their effectiveness.