nils-herrmann/clean_scirepeval huggingface dataset:
Total Variation Regularized RPCA for Irregularly Moving Object Detection Under Dynamic Background

Moving object detection is one of the most fundamental tasks in computer vision. Many classic and contemporary algorithms work well under the assumption that backgrounds are stationary and movements are continuous, but degrade sharply when they are used in a real detection sreal detection systemystem, mainly due to: 1) the dynamic background (e.g., swaying trees, water ripples and fountains in real scenarios, as well as raindrops and snowflakes in bad weather) and 2) the irregular object movement (like lingering objects). This paper presents a unified framework for addressing the difficulties mentioned above, especially the one caused by irregular object movement. This framework separates dynamic background from moving objects using the spatial continuity of foreground, and detects lingering objects using the temporal continuity of foreground. The proposed framework assumes that the dynamic background is sparser than the moving foreground that has smooth boundary and trajectory. We regard the observed video as being made up of the sum of a low-rank static background, a sparse and smooth foreground, and a sparser dynamic background. To deal with this decomposition, i.e., a constrained minimization problem, the augmented Lagrangian multiplier method is employed with the help of the alternating direction minimizing strategy. Extensive experiments on both simulated and real data demonstrate that our method significantly outperforms the state-of-the-art approaches, especially for the cases with dynamic backgrounds and discontinuous movements.
Self-Calibration Using Constant Camera Motion

This paper investigates using constant inter-frame motion for self-calibration from an image sequence of an object rotating around a single axis with varying camera internal parameters. Our approach is based on the facts that in many commercial systems rotation angles are often controlled by an electromechanical system, and the inter-frame essential matrices are invariant if the rotation angles are constant but not necessarily known. It is shown that recovering camera internal parameters is possible by making use of the equivalence of essential matrices, which relate the unknown calibration matrices to the fundamental matrices computed from the point correspondences. Experimental results on both synthetic and real sequences are presented to determine the accuracy and the robustness of the proposed algorithm
Visual Word Pairs for Similar Image Search

The state-of-the-art large scale image retrieval systems have mainly relied on two seminal works: the SIFT descriptor and bag-of-features (BOF) model. However, with the growth of image dataset, the discriminative power of SIFT descriptors was weakened rapidly when mapped to visual words. In this paper, we present a new approach to generate visual word pairs for image retrieval. Two different descriptors are employed to represent the same interest region, and then a visual word pair is obtained by quantizing the descriptor pair with two independent codebooks. By encoding different types of information of the same region, our approach can effectively boost the matching accuracy of descriptors. We evaluate our approach with INRIA Holidays dataset on a 120K image database, and the experiment results suggest that our approach significantly improved the retrieval performance of BOF model.
Multimedia Analysis with Deep Learning

Recently, deep learning method has been attracting more and more researchers due to its great success in various computer vision tasks. Particularly, some researchers focus on the study of multimedia analysis by deep learning method, and the research tasks mainly include the following six aspects: classification, retrieval, segmentation, tracking, detection and recommendation. As far as we know, there is not any literature conducting on survey of these studies, and it is of great significance for the community to review this subject. In this paper, we discuss the application of deep learning method in the six multimedia analysis tasks, and also point out the future directions of deep learning in multimedia analysis.
Joint Optic Disc and Cup Segmentation Based on Multi-Label Deep Network and Polar Transformation

Glaucoma is a chronic eye disease that leads to irreversible vision loss. The cup to disc ratio (CDR) plays an important role in the screening and diagnosis of glaucoma. Thus, the accurate and automatic segmentation of optic disc (OD) and optic cup (OC) from fundus images is a fundamental task. Most existing methods segment them separately, and rely on hand-crafted visual feature from fundus images. In this paper, we propose a deep learning architecture, named M-Net, which solves the OD and OC segmentation jointly in a one-stage multi-label system. The proposed M-Net mainly consists of multi-scale input layer, U-shape convolutional network, side-output layer, and multi-label loss function. The multi-scale input layer constructs an image pyramid to achieve multiple level receptive field sizes. The U-shape convolutional network is employed as the main body network structure to learn the rich hierarchical representation, while the side-output layer acts as an early classifier that produces a companion local prediction map for different scale layers. Finally, a multi-label loss function is proposed to generate the final segmentation map. For improving the segmentation performance further, we also introduce the polar transformation, which provides the representation of the original image in the polar coordinate system. The experiments show that our M-Net system achieves state-of-the-art OD and OC segmentation result on ORIGA data set. Simultaneously, the proposed method also obtains the satisfactory glaucoma screening performances with calculated CDR value on both ORIGA and SCES datasets.
Automated visual analysis in large scale sensor networks

Modern automated video analysis systems consist of large networks of heterogeneous sensors. These systems must extract, integrate and present relevant information from the sensors in real-time. This paper addresses some of the major challenges such systems face: efficient video processing for high-resolution sensors; data fusion across multiple modalities; robustness to changing environmental conditions and video processing errors; and intuitive user interfaces for visualization and analysis. The paper discusses enabling technologies to overcome these challenges and presents a case study of a wide area video analysis system deployed at a port in the state of Florida, USA. The components of the system are also detailed and justified using quantitative and qualitative results.
Structured Saliency Fusion Based on Dempster–Shafer Theory

Visual saliency has been widely used in many applications. However, the performance of an individual saliency detection method varies with the different images. Integrating multiple methods together could compensate this shortcoming, and thus is expected to improve the performance of saliency detection. In this paper, we present an unsupervised Dempster-Shafer Theory (DST) based saliency fusion framework. DST simulates the similar reasoning logic with humans to make decision analysis, and has been proved a suitable method for data fusion. Inspired by this, our framework formalizes the saliency fusion as a statistics inference process, considering the results from several saliency methods to accomplish the fusion task. Furthermore, the proposed framework can flexibly incorporate a variety of inherent structured priors within the images (e.g., clusters and saliency voting) when leveraging the fusion rule of DST. Therefore, it is more close to the fusion mechanism. Experimental results on two benchmark datasets demonstrate the effectiveness and robustness of our framework.
Speeding uplow rank matrix recovery for foreground separation in surveillance videos

Video surveillance currently is one of the most active research topics in public safety and security, in which foreground extraction is important and fundamental for further processing, such as target tracking, activity recognition, and behavior prediction. In this paper, by assuming the background is highly correlated across different frames, we propose to separate foregrounds via speeded up low rank matrix recovery. The proposed method first shrinks the scale of data to roughly catch outliers (foregrounds) of the background. Based on the outliers, we design a sampling strategy that selects a number of frames to construct the low rank model of background. According to the constructed background model, our method further recovers both the background and the foreground for the rest frames in a reconstruction manner. Experimental results on both simulated and real data demonstrate the clear advantage of our approach compared to the state of the arts, in terms of accuracy and efficiency.
Robust Face Clustering Via Tensor Decomposition

Face clustering is a key component either in image managements or video analysis. Wild human faces vary with the poses, expressions, and illumination changes. All kinds of noises, like block occlusions, random pixel corruptions, and various disguises may also destroy the consistency of faces referring to the same person. This motivates us to develop a robust face clustering algorithm that is less sensitive to these noises. To retain the underlying structured information within facial images, we use tensors to represent faces, and then accomplish the clustering task based on the tensor data. The proposed algorithm is called robust tensor clustering (RTC), which firstly finds a lower-rank approximation of the original tensor data using a L1 norm optimization function. Because L1 norm does not exaggerate the effect of noises compared with L2 norm, the minimization of the L1 norm approximation function makes RTC robust. Then, we compute high-order singular value decomposition of this approximate tensor to obtain the final clustering results. Different from traditional algorithms solving the approximation function with a greedy strategy, we utilize a nongreedy strategy to obtain a better solution. Experiments conducted on the benchmark facial datasets and gait sequences demonstrate that RTC has better performance than the state-of-the-art clustering algorithms and is more robust to noises.
Motion saliency detection using low-rank and sparse decomposition

Motion saliency detection has an important impact on further video processing tasks, such as video segmentation, object recognition and adaptive compression. Different to image saliency, in videos, moving regions (objects) catch human beings' attention much easier than static ones. Based on this observation, we propose a novel method of motion saliency detection, which makes use of the low-rank and sparse decomposition on video slices along X-T and Y-T planes to achieve the goal, i.e. separating foreground moving objects from backgrounds. In addition, we adopt the spatial information to preserve the completeness of the detected motion objects. In virtue of adaptive threshold selection and efficient noise elimination, the proposed approach is suitable for different video scenes, and robust to low resolution and noisy cases. The experiments demonstrate the performance of our method compared with the state-of-the-art.
Simple calibration without metric information using an isoceles trapezoid

This paper addresses the problem of calibrating a pin-hole camera from images of an isoceles trapezoid. Assuming a unit aspect ratio and zero skew, we introduce a novel and simple camera calibration approach. The key features of the proposed technique are its simplicity and the lack of need for 3D coordinate information about the calibrating object - i.e. the isosceles trapezoid. By utilizing the symmetry of such trapezoid, we show that one can obtain both the internal and the external camera parameters. To demonstrate the effectiveness of the algorithm, we present the processing results on synthetic and real images, and compare our results to Zhang's flexible calibration method.
SketchNet: Sketch Classification with Web Images

In this study, we present a weakly supervised approach that discovers the discriminative structures of sketch images, given pairs of sketch images and web images. In contrast to traditional approaches that use global appearance features or relay on keypoint features, our aim is to automatically learn the shared latent structures that exist between sketch images and real images, even when there are significant appearance differences across its relevant real images. To accomplish this, we propose a deep convolutional neural network, named SketchNet. We firstly develop a triplet composed of sketch, positive and negative real image as the input of our neural network. To discover the coherent visual structures between the sketch and its positive pairs, we introduce the softmax as the loss function. Then a ranking mechanism is introduced to make the positive pairs obtain a higher score comparing over negative ones to achieve robust representation. Finally, we formalize above-mentioned constrains into the unified objective function, and create an ensemble feature representation to describe the sketch images. Experiments on the TUBerlin sketch benchmark demonstrate the effectiveness of our model and show that deep feature representation brings substantial improvements over other state-of-the-art methods on sketch classification.
Binarized Mode Seeking for Scalable Visual Pattern Discovery

This paper studies visual pattern discovery in large-scale image collections via binarized mode seeking, where images can only be represented as binary codes for efficient storage and computation. We address this problem from the perspective of binary space mode seeking. First, a binary mean shift (bMS) is proposed to discover frequent patterns via mode seeking directly in binary space. The binomial-based kernel and binary constraint are introduced for binarized analysis. Second, we further extend bMS to a more general form, namely contrastive binary mean shift (cbMS), which maximizes the contrastive density in binary space, for finding informative patterns that are both frequent and discriminative for the dataset. With the binarized algorithm and optimization, our methods demonstrate significant computation (50&#xd7;) and storage (32&#xd7;) improvement compared to standard techniques operating in Euclidean space, while the performance does not largely degenerate. Furthermore, cbMS discovers more informative patterns by suppressing low discriminative modes. We evaluate our methods on both annotated ILSVRC (1M images) and un-annotated blind Flickr (10M images) datasets with million scale images, which demonstrates both the scalability and effectiveness of our algorithms for discovering frequent and informative patterns in large scale collection.
Fashion Parsing With Video Context

In this paper, we propose a novel semi- supervised learning strategy to address human parsing. Existing human parsing datasets are relatively small due to the required tedious human labeling. We present a general, affordable and scalable solution, which harnesses the rich contexts in those easily available web videos to boost any existing human parser. First, we crawl a large number of unlabeled videos from the web. Then for each video, the cross-frame contexts are utilized for human pose co- estimation , and then video co-parsing to obtain satisfactory human parsing results for all frames. More specifically, SIFT flow and super-pixel matching are used to build correspondences across different frames, and these correspondences then contextualize the pose estimation and human parsing in individual frames. Finally these parsed video frames are used as the reference corpus for the non-parametric human parsing component of the whole solution. To further improve the accuracy of video co-parsing, we propose an active learning method to incorporate human guidance, where the labelers are required to assess the accuracies of the pose estimation results of certain selected video frames. Then we take reliable frames as the seed frames to guide the video pose co-estimation. Our human parsing framework can then easily incorporate the human feedback to train a better fashion parser. Extensive experiments on two benchmark fashion datasets as well as a newly collected challenging Fashion Icon dataset well demonstrate the encouraging performance gain from our general pipeline for human parsing.
Geometry of a Non-Overlapping Multi-Camera Network

Moving beyond single or stationary camera surveillance systems, we employ an automatically configurable network of non-overlapping cameras. These cameras need not have an overlapping Field of View (FoV) and should be able to move freely in space. In this paper, a practical framework is proposed that determines the geometry of such a dynamic camera network. Assuming each camera in the network is calibrated, it is shown that only one automatically computed vanishing point and a line lying on any plane orthogonal to the vertical direction is sufficient to infer the dynamic network configuration. Our method generalizes previous work which considers restricted camera motions. Using minimal assumptions, we are able to successfully demonstrate promising results on synthetic as well as on real data.
Weakly-supervised TV logo detection

In this paper, a TV logo detection system is proposed based on the deep learning architecture for the specific TV logo detection task. Training a robust object detector typically requires a large amount of manually annotated data, which is time-consuming. To reduce the cost, we construct a TV logo detection system in a weakly-supervised framework, which is accomplished by a TV logo localization network based on Region Proposal Network (RPN) and a classification network based on Fast RCNN. Based on observed priors of a typical TV logo in pictures and video frames, data preparation and processing are performed by carrying out keyframe extraction and data augmentation. Since we build the localization network based on RPN, only a few bounding box annotations are employed for training the localization network. Then the well-trained localization network can produce numerous positive and negative proposals. These proposals along with the logo class labels for classification network training are exploited to train the classification network. To generate reasonable anchor boxes, k-means clustering is utilized to infer the scales and aspect ratios. Besides, for efficient training and better generalization ability, hard example mining is also explored. Experimental results demonstrate that the proposed weakly-supervised TV logo detection system achieves superior performances compared to the baseline Faster RCNN approach, with a mAP as about 92% in our newly proposed dataset.
Water Reflection Detection Using a Flip Invariant Shape Detector

Water reflection detection is a tough task in computer vision, since the reflection is distorted by ripples irregularly. This paper proposes an effective method to detect water reflections. We introduce a descriptor that is not only invariant to scales, rotations and affine transformations, but also tolerant to the flip transformation and even non-rigid distortions, such as ripple effects. We analyze the structure of our descriptor and show how it outperforms the existing mirror feature descriptors in the context of water reflection. The experimental results demonstrate that our method is able to detect the water reflections.
Deep Strip-Based Network with Cascade Learning for Scene Text Localization

Scene text detection is currently a popular research topic in the computer vision community. However, it is a challenging task due to the variations of texts and clutter backgrounds. In this paper, we propose a novel framework for scene text localization. Based on the region proposal network, a Strip-based Text Detection Network (STDN) is developed with vertical anchor mechanism to predict the text/non-text strip-shaped proposals. Meanwhile, we incorporate the recurrent neural network layers in the proposed network to refine the predicted results. Specifically, hard example mining is performed to train the STDN with cascade learning, which has a remarkable improvement in precision. Besides, we exploit a clustering algorithm to generate anchor dimensions spontaneously without hand-picking, which is portable and time-saving. The text detection framework achieves the state-of-the-art performance on ICDAR2013 with 0.89 F-measure.
Forgery Authentication in Extreme Wide-Angle Lens Using Distortion Cue and Fake Saliency Map

Distortion is often considered as an unfavorable factor in most image analysis. However, it is undeniable that the distortion reflects the intrinsic property of the lens, especially, the extreme wide-angle lens, which has a significant distortion. In this paper, we discuss how explicitly employing the distortion cues can detect the forgery object in distortion image and make the following contributions: 1) a radial distortion projection model is adopted to simplify the traditional captured ray-based models, where the straight world line is projected into a great circle on the viewing sphere; 2) two bottom-up cues based on distortion constraint are provided to discriminate the authentication of the line in the image; 3) a fake saliency map is used to maximum fake detection density, and based on the fake saliency map, an energy function is provided to achieve the pixel-level forgery object via graph cut. Experimental results on simulated data and real images demonstrate the performances of our method.
Brightness perception, dynamic range and noise: a unifying model for adaptive image sensors

Many computer vision applications have to cope with large dynamic range and changing illumination conditions in the environment. Any attempt to deal with these conditions at the algorithmic level alone are inherently difficult because of the following: (1) conventional image sensors cannot completely capture wide dynamic range radiances without saturation or underexposure; (2) the quantization process destroys small signal variations especially in shadows; and (3) all possible illumination conditions cannot be completely accounted for. The paper proposes a computational model for brightness perception that deals with issues of dynamic range and noise. The model can be implemented on-chip in analog domain before the signal is saturated or destroyed through quantization. The model is "unified" because a single mathematical formulation addresses the problem of shot and thermal noise, and normalizes the signal range to simultaneously compress the dynamic range, minimize appearance variations due to changing illumination, and minimize quantization noise. The model strongly mimics brightness perception processes in early biological vision.