nils-herrmann/clean_scirepeval huggingface dataset:
Parallel spatial matching for object retrieval implemented on GPU

Spatial matching for object retrieval is often time-consuming and susceptible to viewpoint changes. To address this problem, we propose a novel spatial matching method and implement it on modern GPU in parallel. Unlike previous spatial matching methods, in which the affine transformation estimation is based on the gravity vector assumption, our method abandons this strong assumption by matching the ACNs (affine covariant neighbors) of corresponding local regions and estimating affine transformation from a single pair of corresponding local regions. To speed up the process, we implement the method on modern GPU in parallel. Computations are distributed evenly to threads with load balancing, and the memory accesses are optimized and bitmap based parallel scan is exploited. Experimental results demonstrate that our method is more robust and more efficient than previous methods especially when the viewpoints are changed, and the parallel implementation on GPU obtains ten times speedup.
Multiview Spectral Embedding

In computer vision and multimedia search, it is common to use multiple features from different views to represent an object. For example, to well characterize a natural scene image, it is essential to find a set of visual features to represent its color, texture, and shape information and encode each feature into a vector. Therefore, we have a set of vectors in different spaces to represent the image. Conventional spectral-embedding algorithms cannot deal with such datum directly, so we have to concatenate these vectors together as a new vector. This concatenation is not physically meaningful because each feature has a specific statistical property. Therefore, we develop a new spectral-embedding algorithm, namely, multiview spectral embedding (MSE), which can encode different features in different ways, to achieve a physically meaningful embedding. In particular, MSE finds a low-dimensional embedding wherein the distribution of each view is sufficiently smooth, and MSE explores the complementary property of different views. Because there is no closed-form solution for MSE, we derive an alternating optimization-based iterative algorithm to obtain the low-dimensional embedding. Empirical evaluations based on the applications of image retrieval, video annotation, and document clustering demonstrate the effectiveness of the proposed approach.
A Novel Image Text Extraction Method Based on K-Means Clustering

Texts in web pages, images and videos contain important clues for information indexing and retrieval. Most existing text extraction methods depend on the language type and text appearance. In this paper, a novel and universal method of image text extraction is proposed. A coarse-to-fine text location method is implemented. Firstly, a multi-scale approach is adopted to locate texts with different font sizes. Secondly, projection profiles are used in location refinement step. Color-based k-means clustering is adopted in text segmentation. Compared to grayscale image which is used in most existing methods, color image is more suitable for segmentation based on clustering. It treats corner-points, edge-points and other points equally so that it solves the problem of handling multilingual text. It is demonstrated in experimental results that best performance is obtained when k is 3. Comparative experimental results on a large number of images show that our method is accurate and robust in various conditions.
Efficient quantization step selection scheme for I-frame in rate-constrained video coding

Quantization step (QS) selection for I-frame has an obvious impact on rate-distortion performance in rate-constrained video coding. An efficient QS selection scheme for I-frame is presented which is based on a balanced bits allocation scheme between I and P frames and a new experimental rate-quantization step (R-Q) model. In the balanced bits allocation scheme, firstly a model on the ratio of bits allocation between I and P frame is derived under the constraint of certain target bit rate; then based on the model, a feasible bits allocation algorithm is proposed by experimental analysis among different target bit rates. The analysis and final simulation results show that compared to the existing methods, the proposed algorithm improves PSNR performance significantly (up to 1.2 dB).
Enhancing Video Event Recognition Using Automatically Constructed Semantic-Visual Knowledge Base

The task of recognizing events from video has attracted a lot of attention in recent years. However, due to the complex nature of user-defined events, the use of purely audio- visual content analysis without domain knowledge has been found to be grossly inadequate. In this paper, we propose to construct a semantic-visual knowledge base to encode the rich event-centric concepts and their relationships from the well- established lexical databases, including FrameNet, as well as the concept-specific visual knowledge from ImageNet. Based on this semantic-visual knowledge bases, we design an effective system for video event recognition. Specifically, in order to narrow the semantic gap between the high-level complex events and low-level visual representations, we utilize the event-centric semantic concepts encoded in the knowledge base as the intermediate-level event representation, which offers both human-perceivable and machine-interpretable semantic clues for event recognition. In addition, in order to leverage the abundant ImageNet images, we propose a robust transfer learning model to learn the noise- resistant concept classifiers for videos. Extensive experiments on various real-world video datasets demonstrate the superiority of our proposed system as compared to the state-of-the-art approaches.
Object Localization Based on Proposal Fusion

Traditional regression framework of object locali-zation such as Overfeat often suffers from the problem of inaccurate scoring due to the separate scoring of classification network and regression network upon inconsistent regions. To tackle this problem, in this paper, we propose a novel object localization framework based on multiple complementary region proposal methods from the view of classification rather than regression. On top of our framework, we first combine multiple complementary region proposals during both training and testing as a means of data augmentation to generate more dense and reliable proposals for fusion, then achieve optimal compromise between complexity and efficiency through category clustering for bounding box sharing among similar categories, and finally propose a dense proposal fusion approach to merge dense region proposals near true object for fine-tuning of the final bounding box's coordinates and updating the confidence of fused proposals for final decision. Extensive experiments on the well-known large scale ILSVRC 2015 LOC dataset verify the effectiveness of our object localization framework.
Implicit Negative Sub-Categorization and Sink Diversion for Object Detection

In this paper, we focus on improving the proposal classification stage in the object detection task and present implicit negative sub-categorization and sink diversion to lift the performance by strengthening loss function in this stage. First, based on the observation that the “background” class is generally very diverse and thus challenging to be handled as a single indiscriminative class in existing state-of-the-art methods, we propose to divide the background category into multiple implicit sub-categories to explicitly differentiate diverse patterns within it. Second, since the ground truth class inevitably has low-value probability scores for certain images, we propose to add a “sink” class and divert the probabilities of wrong classes to this class when necessary, such that the ground truth label will still have a higher probability than other wrong classes even though it has low probability output. Additionally, we propose to use dilated convolution, which is widely used in the semantic segmentation task, for efficient and valuable context information extraction. Extensive experiments on PASCAL VOC 2007 and 2012 data sets show that our proposed methods based on faster R-CNN implementation can achieve state-of-the-art mAPs, i.e., 84.1%, 82.6%, respectively, and obtain 2.5% improvement on ILSVRC DET compared with that of ResNet.
A Simple and Efficient Re-Scrambling Scheme for DTV Programs

In order to guarantee pay-TV services, data of digital television (DTV) programs are scrambled by conditional access systems (CAS). In practical applications, some DTV transmitting nodes need descramble the scrambled DTV programs for editing purposes. After editing, how to re-scramble these edited programs is a challenging task since building CAS at transmitting nodes is expensive and insecure. In this paper, we proposed a novel scheme to solve this problem. Together with the recommended scheme, we also proposed techniques regarding video key data selection and extraction, synchronization of scrambled and descrambled transport stream (TS) packets, and scrambled/unscrambled interleaving multiplexer. The proposed scheme requires much lower complexity than existing methods while maintaining enough security for practical applications. Neither professional CAS equipment nor real-time common key (CK) transmitting is required by our re-scrambling scheme. Various experimental results demonstrated that the proposed re-scrambling scheme achieved superior performance in practical DTV systems, and it also obtained good compatibility with different CAS algorithms.
Motion adaptive deinterlacing with accurate motion detection and anti-aliasing interpolation filter

Both the motion-detection and infra-field interpolation filter are important factors affect the efficiency of motion adaptive de-interlacing. New accurate motion detection (AMD) algorithm is proposed to improve the accuracy of motion detection, which reduces the possibility of error motion detection with a median filter. To improve the efficiency of intra-field interpolation deinterlacing in moving regions, an anti-aliasing interpolation filter (AAIF) is proposed, which is better than the typical windowed sine function. The simulation results show that peak signal noise ratio (PSNR) of our proposed deinterlacing method is 0.5-7.5 dB higher than that of previous studies and attains the best quality of subjective view.
Logo detection based on spatial-spectral saliency and partial spatial context

Logo detection is important for brand advertising and surveillance applications. The central issues of this technology are fast localization and accurate matching. Based on key traits analysis of common logos, this paper presents a two-stage detection scheme based on spatialspectral saliency (SSS) and partial spatial context (PSC). SSS speeds up logo location and avoid the impact of cluttered background. PSC filters false matching using spatial consistency of local invariant points. The integration of SSS and PSC result in faster localization and increased accuracy. Experiments on a dataset of nearly 10,000 web images containing several popular logo types are presented. The results indicate that our method is applicable and precise for different logo detection scenarios.
AP-Based Adaboost in High Level Feature Extraction at TRECVID

We propose an improved fusion method used in high level feature extraction at TRECVID - average precision based Adaboost (AP-based Adaboost). The AP-based weighting scheme makes use of both the weight and the rank of each sample that all have contribution to the final average precision. The weighting scheme along with the more adaptive formulae modified in our method makes it outperform the standard Adaboost algorithm as well as many other fusion methods. Experimental results on TRECVID-2005 development set show that our method is an effective and relatively robust fusion method.
Transcoing-based Data Transmission of Sphere Panoramic Videos

Omnidirectional video is a kind of new emerging media. The filed of view of an omnidirectional video can be 360 degrees in vertical and horizontal. The users can navigate interactively through the scene and change their view angel. The scene of an omnidirectional video is often projected on the surface of a sphere or a cylinder to store, which is a panoramic video. Panoramic videos are often high-resolution and consume a significant amount of bandwidth for transmission. To resolve the problem, tiles-based data transmission is applied in some systems but it is not efficient for sphere panoramic video and transmits a mass of redundant bits to the users. In this paper, we proposed an efficient transcoding-based data transmission technique for panoramic videos, which reduced the amount of data transmission at most
High throughput and low memory access sub-pixel interpolation architecture for H.264/AVC HDTV decoder

In this paper, we proposed parallel and pipeline architecture for the sub-pixel interpolation filter in H.264/AVC conformed HDTV decoder. To efficiently use the bus bandwidth, we bring forward three memory access optimization strategies to avoid redundant data transfer and improve data bus utilization. To improve the processing throughput, we use parallel and multi-stage pipeline architecture for conducting data transmission and interpolation filtering in parallel. Moreover, to balance the tradeoff between memory accessing scheme and sub-pixel interpolation processing granularity we devise a dedicated buffer organization to convert tree-structured block size reading to fixable and sequential processing. As compared to the traditional designs, our scheme offers 60% reduced memory data transfer. While clocking at 66 MHz, our design can support 1280 /spl times/ 720 @30 Hz processing throughput. The proposed design is suitable for low cost and real-time applications. Moreover, it can easily be applied in system-on-chip design.
Real-Time Scene Text Detection Based on Stroke Model

In this paper we bring forth a novel stroke-based method which is simple and effective to detect texts in natural scenes. We first introduce a general mathematical model to describe character strokes from the perspective of the scale space along with difference of Gaussian filters. Then we detail a text line aggregation approach utilizing the inherent text layout. Afterwards, we set up the whole scheme with three main steps, i.e. stroke extraction, text line aggregation and verification. Finally, experiments show the advantage of our method. As strokes are considered to be the fundamental component of characters, compared to edge- or other connected-component-based methods, our method is much more reasonable.
Parallel deblocking filter for H.264/AVC implemented on Tile64 platform

For the purpose of accelerating deblocking filter, which accounts for a significant percentage of H.264/AVC decoding time, some researchers use multi-core platforms to achieve the required performance. We study the problem under the context of many-core systems. Parallelization of deblocking filter on many-core platform is challenging not only because deblocking filter has complicated data dependencies which provides insufficient parallelism for so many cores but also because parallelization may have significant synchronization overhead. We present a new method to exploit the implicit parallelism and reduce the synchronization overhead. We apply our implementation to the deblocking filter of the H.264/AVC reference software JM15.1 on Tile64 platform. The proposed method achieves up to 817%, 604% and 532% speedup for CIF, SD and HD videos compared to the well-known wavefront method using 62 cores, respectively.
Efficient Feature Detection and Effective Post-Verification for Large Scale Near-Duplicate Image Search

State-of-the-art near-duplicate image search systems mostly build on the bag-of-local features (BOF) representation. While favorable for simplicity and scalability, these systems have three shortcomings: 1) high time complexity of the local feature detection; 2) discriminability reduction of local descriptors due to BOF quantization; and 3) neglect of the geometric relationships among local features after BOF representation. To overcome these shortcomings, we propose a novel framework by using graphics processing units (GPU). The main contributions of our method are: 1) a new fast local feature detector coined Harris-Hessian (H-H) is designed according to the characteristics of GPU to accelerate the local feature detection; 2) the spatial information around each local feature is incorporated to improve its discriminability, supplying semi-local spatial coherent verification (LSC); and 3) a new pairwise weak geometric consistency constraint (P-WGC) algorithm is proposed to refine the search result. Additionally, part of the system is implemented on GPU to improve efficiency. Experiments conducted on reference datasets and a dataset of one million images demonstrate the effectiveness and efficiency of H-H, LSC, and P-WGC.
Query Range Sensitive Probability Guided Multi-probe Locality Sensitive Hashing

Locality Sensitive Hashing (LSH) is proposed to construct indexes for high-dimensional approximate similarity search. Multi-Probe LSH (MPLSH) is a variation of LSH which can reduce the number of hash tables. Based on the idea of MPLSH, this paper proposes a novel probability model and a query-adaptive algorithm to generate the optimal multi-probe sequence for range queries. Our probability model takes the query range into account to generate the probe sequence which is optimal for range queries. Furthermore, our algorithm does not use a fixed number of probe steps but a query-adaptive threshold to control the search quality. We do the experiments on an open dataset to evaluate our method. The experimental results show that our method can probe fewer points than MPLSH for getting the same recall. As a result, our method can get an average acceleration of 10% compared to MPLSH.
Estimation Of Epipolar Geometry Via The Radon Transform

One of the key problems in computer vision is the recovery of epipolar geometry constraints between different camera views. The majority of existing techniques rely on point correspondences, which are typically perturbed by mismatches and noise, hence limiting the accuracy of these techniques. To overcome these limitations, we propose a novel approach that estimates epipolar geometry constraints based on a statistical model in the Radon domain. The method requires no correspondences, explicit constraints on the data or assumptions regarding the scene structure. Results are presented on both synthetic and real data that show the method's robustness to noise and outliers
A wavelet visible difference predictor

In this paper, we describe a model of the human visual system (HVS) based on the wavelet transform. This model is largely based on a previously proposed model, but has a number of modifications that make it more amenable to potential integration into a wavelet based image compression scheme. These modifications include the use of a separable wavelet transform instead of the cortex transform, the application of a wavelet contrast sensitivity function (CSF), and a simplified definition of subband contrast that allows one to predict the noise visibility directly from the wavelet coefficients. Initially, we outline the luminance, frequency, and masking sensitivities of the HVS and discuss how these can be incorporated into the wavelet transform. We then outline a number of limitations of the wavelet transform as a model of the HVS, namely the lack of translational invariance and poor orientation sensitivity. In order to investigate the efficacy of this wavelet based model, a wavelet visible difference predictor (WVDP) is described. The WVDP is then used to predict visible differences between an original and compressed (or noisy) image. Results are presented to emphasize the limitations of commonly used measures of image quality and to demonstrate the performance of the WVDP. The paper concludes with suggestions on how the WVDP can be used to determine a visually optimal quantization strategy for wavelet coefficients and produce a quantitative measure of image quality.
Correspondence-Free Determination of the Affine Fundamental Matrix

Fundamental matrix estimation is a central problem in computer vision and forms the basis of tasks such as stereo imaging and structure from motion. Existing algorithms typically analyze the relative geometries of matched feature points identified in both projected views. Automated feature matching is itself a challenging problem. Results typically have a large number of false matches. Traditional fundamental matrix estimation methods are very sensitive to matching errors, which led naturally to the application of robust statistical estimation techniques to the problem. In this work, an entirely novel approach is proposed to the fundamental matrix estimation problem. Instead of analyzing the geometry of matched feature points, the problem is recast in the frequency domain through the use of integral projection, showing how this is a reasonable model for orthographic cameras. The problem now reduces to one of identifying matching lines in the frequency domain which, most importantly, requires no feature matching or correspondence information. Experimental results on both real and synthetic data are presented that demonstrate the algorithm is a practical technique for fundamental matrix estimation. The behavior of the proposed algorithm is additionally characterized with respect to input noise, feature counts, and other parameters of interest
3D Motion Estimation from Three Orthographic Views without Matching Constraints or Brightness Gradients

An important task in computer vision is to estimate the 3D motion of rigid objects from multiple views. Conventional methods typically rely on point correspondences or image brightness gradients. The underlying matching and brightness constraints are generally not met in practice which limits the accuracy of these techniques. In this paper, we present a new approach that performs 3D motion estimation directly from trinocular point clouds under orthography. The method requires no correspondences or brightness gradients which constitutes a major advantage over existing approaches. Results are presented which highlight the estimation accuracy of the proposed method with respect to conventional correspondence-based techniques.