nils-herrmann/clean_scirepeval huggingface dataset:
Lip Segmentation under MAP-MRF Framework with Automatic Selection of Local Observation Scale and Number of Segments

This paper addresses the problem of segmenting lip region from frontal human face image. Supposing each pixel of the target image has an optimal local scale from the segmentation viewpoint, we treat the lip segmentation problem as a combination of observation scale selection and observed data classification. Accordingly, we propose a hierarchical multiscale Markov random field (MRF) model to represent the membership map of each input pixel to a specific segment and local-scale map simultaneously. Subsequently, lip segmentation can be formulated as an optimal problem in the maximum a posteriori (MAP)-MRF framework. Then, we present a rival-penalized iterative algorithm to implement the segmentation, which is independent of the number of predefined segments. The proposed method mainly features two aspects: 1) its performance is independent of the predefined number of segments, and 2) it takes into account the local optimal observation scale for each pixel. Finally, we conduct the experiments on four benchmark databases, i.e. AR, CVL, GTAV, and VidTIMIT. Experimental results show that the proposed method is robust to the segment number that changes with a speaker's appearance, and can enhance the segmentation accuracy by taking advantage of the local optimal observation scale information.
Automatic geo-registration of maritime video feeds

We propose an automatic method to geo-register maritime video feeds to satellite images. The method first detects horizon during the day time and apex during the night time for similarity rectification. It then finds water pixels on both satellite images and camera views. Finally, geo-registration is obtained by searching the best orientation where the water masks are matched. The current state-art-of-state video surveillance system benefits with this method by providing situational awareness on maps with physical data including object size, location, and velocity in the world coordinates. Experimental results and applications such as cross camera tracking are demonstrated.
IacDroid: Preventing Inter-App Communication capability leaks in Android

Inter-App Communication (IAC) plays an important role in Android platform to share data and services among applications. However, the existence of IAC capability leaks could lead to the unauthorized privileged operations. In this paper, we first investigate the usage of IAC in Android applications to show the prevalence of IAC in the Android development model. To mitigate the threat caused by IAC capability leaks in Android, we develop a real-time monitoring and control system, called IacDroid, which distinguishes and prevents the IAC capability leak accurately in both third-party and in-rom applications at runtime. IacDroid extends the Binder IPC mechanism and the system service to construct context-based component call chains between multiple applications. By leveraging the call chains, the permission system is extended to detect and prevent the IAC capability leaks. IacDroid also presents an intuitive client-side solution to help users control the IAC capability leaks. We implement the prototype in Android 4.3, and present a comprehensive assessment with 500 Google Play applications and 36 malicious applications. The experimental results demonstrate that IacDroid can effectively prevent the IAC capability leaks with a negligible performance overhead.
Holons Visual Representation for Image Retrieval

Along with the enlargement of image scale, convolutional local features, such as SIFT, are ineffective for representing or indexing and more compact visual representations are required. Due to the intrinsic mechanism, the state-of-the-art vector of locally aggregated descriptors (VLAD) has a few limits. Based on this, we propose a new descriptor named holons visual representation (HVR). The proposed HVR is a derivative mutational self-contained combination of global and local information. It exploits both global characteristics and the statistic information of local descriptors in the image dataset. It also takes advantages of local features of each image and computes their distribution with respect to the entire local descriptor space. Accordingly, the HVR is computed by a two-layer hierarchical scheme, which splits the local feature space and obtains raw partitions, as well as the corresponding refined partitions. Then, according to the distances from the centroids of partition spaces to local features and their spatial correlation, we assign the local features into their nearest raw partitions and refined partitions to obtain the global description of an image. Compared with VLAD, HVR holds critical structure information and enhances the discriminative power of individual representation with a small amount of computation cost, while using the same memory overhead. Extensive experiments on several benchmark datasets demonstrate that the proposed HVR outperforms conventional approaches in terms of scalability as well as retrieval accuracy for images with similar intra local information.
Regularity Preserved Superpixels and Supervoxels

Most existing superpixel algorithms ignore the spatial structure and regularity properties, which result in undesirable sizes and location relationships for the subsequent processing. In this paper, we introduce a new method to generate the regularity preserved superpixels. Starting from the lattice seeds, our method relocates them to the pixel with locally maximal edge magnitudes and treats them as the superpixel junctions. Then, the shortest path algorithm is employed to find the local optimal boundary connecting each adjacent junction pair. Thanks to the local constraints, our method obtains homogeneous superpixels with adjacency in lowly textured and uniform regions and simultaneously preserves the boundary adherence in the high contrast contents. Our method preserves the regularity property without significantly sacrificing the segmentation accuracy. Moreover, we extend this regular constraint for generating the supervoxels. Our method obtains the regular supervoxels, which preserves the structural relation on both spatial and temporal spaces of the video. Quantitative and qualitative experimental results on benchmark datasets demonstrate that our simple but effective method outperforms the existing regular superpixel methods.
Diversity-induced Multi-view Subspace Clustering

In this paper, we focus on how to boost the multi-view clustering by exploring the complementary information among multi-view features. A multi-view clustering framework, called Diversity-induced Multi-view Subspace Clustering (DiMSC), is proposed for this task. In our method, we extend the existing subspace clustering into the multi-view domain, and utilize the Hilbert Schmidt Independence Criterion (HSIC) as a diversity term to explore the complementarity of multi-view representations, which could be solved efficiently by using the alternating minimizing optimization. Compared to other multi-view clustering methods, the enhanced complementarity reduces the redundancy between the multi-view representations, and improves the accuracy of the clustering results. Experiments on both image and video face clustering well demonstrate that the proposed method outperforms the state-of-the-art methods.
Topology Preserved Regular Superpixel

Most existing super pixel algorithms ignore the topology and regularities, which results in undesirable sizes and location relationships for subsequent processing. In this paper, we introduce a new method to compute the regular super pixels while preserving the topology. Start from regular seeds, our method relocates them to the pixel with locally maximal edge magnitudes. Then, we find the local optimal path between each relocated seed and its four neighbors using Dijkstra algorithm. Thanks to the local constraints, our method obtains homogeneous super pixels with explicit adjacency in low-texture and uniform regions and, simultaneously, maintains the edge cues in the high contrast and salient contents. Quantitative and qualitative experimental results on Berkeley Segmentation Database Benchmark demonstrate that our proposed algorithm outperforms the existing regular super pixel methods.
Fake Colorized Image Detection

Image forensics aims to detect the manipulation of digital images. Currently, splicing detection, copy-move detection, and image retouching detection are attracting significant attention from researchers. However, image editing techniques develop over time. An emerging image editing technique is colorization, in which grayscale images are colorized with realistic colors. Unfortunately, this technique may also be intentionally applied to certain images to confound object recognition algorithms. To the best of our knowledge, no forensic technique has yet been invented to identify whether an image is colorized. We observed that, compared with natural images, colorized images, which are generated by three state-of-the-art methods, possess statistical differences for the hue and saturation channels. Besides, we also observe statistical inconsistencies in the dark and bright channels, because the colorization process will inevitably affect the dark and bright channel values. Based on our observations, i.e., potential traces in the hue, saturation, dark, and bright channels, we propose two simple yet effective detection methods for fake colorized images: Histogram-based fake colorized image detection and feature encoding-based fake colorized image detection. Experimental results demonstrate that both proposed methods exhibit a decent performance against multiple state-of-the-art colorization approaches.
Robust Target Tracking by Online Random Forests and Superpixels

This paper presents a robust joint discriminative appearance model-based tracking method using online random forests and mid-level feature (superpixels). To achieve superpixel-wise discriminative ability, we propose a joint appearance model that consists of two random forest-based models, i.e., the background-target discriminative model (BTM) and the distractor-target discriminative model (DTM). More specifically, the BTM effectively learns discriminative information between the target object and the background. In contrast, the DTM is used to suppress distracting superpixels, which significantly improves the tracker’s robustness and alleviates the drifting problem. A novel online random forest regression algorithm is proposed to build the two models. The BTM and DTM are linearly combined into a joint model to compute a confidence map. Tracking results are estimated using the confidence map, in which the position and scale of the target are estimated orderly. Furthermore, we design a model updating strategy to adapt the appearance changes over time by discarding degraded trees of the BTM and DTM and initializing new trees as replacements. We test the proposed tracking method on two large tracking benchmarks, the CVPR2013 tracking benchmark and VOT2014 tracking challenge. Experimental results show that the tracker runs at real-time speed and achieves favorable tracking performance compared with the state-of-the-art methods. The results also suggest that the DTM improves tracking performance significantly and plays an important role in robust tracking.
Camera calibration and light source estimation from images with shadows

In this paper, we describe how camera parameters and light source orientation can be recovered from two perspective views of a scene given only two vertical lines and their cast shadows. Compared to the traditional calibration methods that involve images of some precisely machined calibration pattern, our method uses new calibration objects: the vertical objects and their parallel shadow lines, which are common in natural environments. In addition to the benefit of increasing accessibility of the calibration objects, the proposed method is also especially useful in cases where only limited information is available. To demonstrate the accuracy and the applications of the proposed algorithm, we present results on both synthetic and real images.
Horizon matters: Image re-targeting using horizon cues

In this work, we propose an effective method for improving the aesthetics of a given image based on the horizon information extracted from the image. We employ the vanishing line detection method to determine the horizon and then adjust the angle of the scene to level the horizon. The image is thus retargeted to highlight the visual appearance. In addition, our method can also be applied on the task of distinguishing the image acquisition methods. For example, hand held phones typically don't have perfect level horizons. Experimental results demonstrate the performance of our proposed method.
Camera Calibration Using Symmetric Objects

This paper proposes a novel method for camera calibration using images of a mirror symmetric object. Assuming unit aspect ratio and zero skew, we show that interimage homographies can be expressed as a function of only the principal point. By minimizing symmetric transfer errors, we thus obtain an accurate solution for the camera parameters. We also extend our approach to a calibration technique using images of a 1-D object with a fixed pivoting point. Unlike existing methods that rely on orthogonality or pole-polar relationship, our approach utilizes new interimage constraints and does not require knowledge of the 3-D coordinates of feature points. To demonstrate the effectiveness of the approach, we present results for both synthetic and real images
Image Deblurring via Enhanced Low-Rank Prior.

Low-rank matrix approximation has been successfully applied to numerous vision problems in recent years. In this paper, we propose a novel low-rank prior for blind image deblurring. Our key observation is that directly applying a simple low-rank model to a blurry input image significantly reduces the blur even without using any kernel information, while preserving important edge information. The same model can be used to reduce blur in the gradient map of a blurry input. Based on these properties, we introduce an enhanced prior for image deblurring by combining the low rank prior of similar patches from both the blurry image and its gradient map. We employ a weighted nuclear norm minimization method to further enhance the effectiveness of low-rank prior for image deblurring, by retaining the dominant edges and eliminating fine texture and slight edges in intermediate images, allowing for better kernel estimation. In addition, we evaluate the proposed enhanced low-rank prior for both the uniform and the non-uniform deblurring. Quantitative and qualitative experimental evaluations demonstrate that the proposed algorithm performs favorably against the state-of-the-art deblurring methods.
A Hierarchical Distributed Processing Framework for Big Image Data

This paper introduces an effective processing framework nominated Image Cloud Processing (ICP) to powerfully cope with the data explosion in image processing field. While most previous researches focus on optimizing the image processing algorithms to gain higher efficiency, our work dedicates to providing a general framework for those image processing algorithms, which can be implemented in parallel so as to achieve a boost in time efficiency without compromising the results performance along with the increasing image scale. The proposed ICP framework consists of two mechanisms, i.e., Static ICP (SICP) and Dynamic ICP (DICP). Specifically, SICP is aimed at processing the big image data pre-stored in the distributed system, while DICP is proposed for dynamic input. To accomplish SICP, two novel data representations named P-Image and Big-Image are designed to cooperate with MapReduce to achieve more optimized configuration and higher efficiency. DICP is implemented through a parallel processing procedure working with the traditional processing mechanism of the distributed system. Representative results of comprehensive experiments on the challenging ImageNet dataset are selected to validate the capacity of our proposed ICP framework over the traditional state-of-the-art methods, both in time efficiency and quality of results.
Video Editing with Temporal, Spatial and Appearance Consistency

Given an area of interest in a video sequence, one may want to manipulate or edit the area, e.g. remove occlusions from or replace with an advertisement on it. Such a task involves three main challenges including temporal consistency, spatial pose, and visual realism. The proposed method effectively seeks an optimal solution to simultaneously deal with temporal alignment, pose rectification, as well as precise recovery of the occlusion. To make our method applicable to long video sequences, we propose a batch alignment method for automatically aligning and rectifying a small number of initial frames, and then show how to align the remaining frames incrementally to the aligned base images. From the error residual of the robust alignment process, we automatically construct a trimap of the region for each frame, which is used as the input to alpha matting methods to extract the occluding foreground. Experimental results on both simulated and real data demonstrate the accurate and robust performance of our method.
Visual Object Tracking With Partition Loss Schemes

Object tracking is a fundamental task for building vision systems of automatic transportation. Despite demonstrated success in this active research field, it is still difficult to cope with complicated appearance changes caused by background clutters, illumination change, scale variation, deformation, rotation, and occlusion, etc. Due to these challenging factors, a target bounding box tends to easily contain a disturbing context of background, which may lead to an inaccurate localization if some key parts of the foreground object share an excess of information loss. In this paper, we propose an online algorithm using the local loss features to alleviate the drift problem during tracking. An adaptive block-division appearance model is constructed to exploit patch-based loss representations by decomposing sample region sequences into a set of subblocks. The basic purpose of partition coefficients is to indicate local relevance and effectively capture spatial correlation through measuring image similarity. Namely, they can strengthen positive effects of discriminative patches within locating bounding boxes and weaken negative impacts of disturbing context possibly included in the surrounding regions. The object state estimation of a moving target is then formulated as an integrated likelihood evaluation on the ensemble loss. Experimental results on a suite of representative video sequences of realistic scenarios demonstrate the superiority of the proposed method to several state-of-the-art tracking approaches in terms of both accuracy and robustness.
Preserving privacy for video surveillance via visual cryptography

The video surveillance widely installed in public areas poses a significant threat to the privacy. This paper proposes a new privacy preserving method via the Generalized Random-Grid based Visual Cryptography Scheme (GRG-based VCS). We first separate the foreground from the background for each video frame. These foreground pixels contain the most important information that needs to be protected. Every foreground area is encrypted into two shares based on GRG-based VCS. One share is taken as the foreground, and the other one is embedded into another frame with random selection. The content of foreground can only be recovered when these two shares are got together. The performance evaluation on several surveillance scenarios demonstrates that our proposed method can effectively protect sensitive privacy information in surveillance videos.
Video Completion for Perspective Camera Under Constrained Motion

This paper presents a novel technique to fill in missing background and moving foreground of a video captured by a static or moving camera. Different from previous efforts which are typically based on processing in the 3D data volume, we slice the volume along the motion manifold of the moving object, and therefore reduce the search space from 3D to 2D, while still preserve the spatial and temporal coherence. In addition to the computational efficiency, based on geometric video analysis, the proposed approach is also able to handle real videos under perspective distortion, as well as common camera motions, such as panning, tilting, and zooming. The experimental results demonstrate that our algorithm performs comparably to 3D search based methods, and however extends the current state-of-the-art repairing techniques to videos with projective effects, as well as illumination changes
FIND: A Neat Flip Invariant Descriptor

In this paper, we introduce a novel Flip Invariant Descriptor (FIND). FIND improves the degenerated performance resulted from image flips and reduces both space and time costs. Flip invariance of FIND enables the intractable flip detection to be achieved easily, instead of duplicately implementing the procedure. To alleviate the pressure brought by the increasing scale of image and video data, FIND utilizes a concise structure with less storage space. Comparing to SIFT, FIND reduces 35.94% length for a descriptor. We compare FIND against SIFT with respect to accuracy, speed and space cost. An application to image search over a database of 3.27 million descriptors is also shown.
SYM-FISH: A Symmetry-Aware Flip Invariant Sketch Histogram Shape Descriptor

Recently, studies on sketch, such as sketch retrieval and sketch classification, have received more attention in the computer vision community. One of its most fundamental and essential problems is how to more effectively describe a sketch image. Many existing descriptors, such as shape context, have achieved great success. In this paper, we propose a new descriptor, namely Symmetric-aware Flip Invariant Sketch Histogram (SYM-FISH) to refine the shape context feature. Its extraction process includes three steps. First the Flip Invariant Sketch Histogram (FISH) descriptor is extracted on the input image, which is a flip-invariant version of the shape context feature. Then we explore the symmetry character of the image by calculating the kurtosis coefficient. Finally, the SYM-FISH is generated by constructing a symmetry table. The new SYM-FISH descriptor supplements the original shape context by encoding the symmetric information, which is a pervasive characteristic of natural scene and objects. We evaluate the efficacy of the novel descriptor in two applications, i.e., sketch retrieval and sketch classification. Extensive experiments on three datasets well demonstrate the effectiveness and robustness of the proposed SYM-FISH descriptor.