nils-herrmann/clean_scirepeval huggingface dataset:
Hallucinating Very Low-Resolution Unaligned and Noisy Face Images by Transformative Discriminative Autoencoders

Most of the conventional face hallucination methods assume the input image is sufficiently large and aligned, and all require the input image to be noise-free. Their performance degrades drastically if the input image is tiny, unaligned, and contaminated by noise. In this paper, we introduce a novel transformative discriminative autoencoder to 8X super-resolve unaligned noisy and tiny (16X16) low-resolution face images. In contrast to encoder-decoder based autoencoders, our method uses decoder-encoder-decoder networks. We first employ a transformative discriminative decoder network to upsample and denoise simultaneously. Then we use a transformative encoder network to project the intermediate HR faces to aligned and noise-free LR faces. Finally, we use the second decoder to generate hallucinated HR images. Our extensive evaluations on a very large face dataset show that our method achieves superior hallucination results and outperforms the state-of-the-art by a large margin of 1.82dB PSNR.
Visual Tracking by Sampling in Part Space

In this paper, we present a novel part-based visual tracking method from the perspective of probability sampling. Specifically, we represent the target by a part space with two online learned probabilities to capture the structure of the target. The proposal distribution memorizes the historical performance of different parts, and it is used for the first round of part selection. The acceptance probability validates the specific tracking stability of each part in a frame, and it determines whether to accept its vote or to reject it. By doing this, we transform the complex online part selection problem into a probability learning one, which is easier to tackle. The observation model of each part is constructed by an improved supervised descent method and is learned in an incremental manner. Experimental results on two benchmarks demonstrate the competitive performance of our tracker against state-of-the-art methods.
A Comprehensive Evaluation Framework and a Comparative Study for Human Detectors

We introduce a framework for evaluating human detectors that considers the practical application of a detector on a full image using multisize sliding-window scanning. We produce detection error tradeoff (DET) curves relating the miss detection rate and the false-alarm rate computed by deploying the detector on cropped windows and whole images, using, in the latter, either image resize or feature resize. Plots for cascade classifiers are generated based on confidence scores instead of on variation of the number of layers. To assess a method's overall performance on a given test, we use the average log miss rate (ALMR) as an aggregate performance score. To analyze the significance of the obtained results, we conduct 10-fold cross-validation experiments. We applied our evaluation framework to two state-of-the-art cascade-based detectors on the standard INRIA person dataset and a local dataset of near-infrared images. We used our evaluation framework to study the differences between the two detectors on the two datasets with different evaluation methods. Our results show the utility of our framework. They also suggest that the descriptors used to represent features and the training window size are more important in predicting the detection performance than the nature of the imaging process, and that the choice between resizing images or features can have serious consequences.
Automatic Refinement Strategies for Manual Initialization of Object Trackers

Tracking objects across multiple frames is a well-investigated problem in computer vision. The majority of the existing algorithms that assume an accurate initialization is readily available. However, in many real-life settings, in particular for applications where the video is streaming in real time, the initialization has to be provided by a human operator. This limitation raises an inevitable uncertainty issue. Here, we first collect a large and new data set of inputs that consists of more than 20 K human initialization clicks, by several subjects under three practical user interface scenarios for the popular TB50 tracking benchmark. We analyze the factors and mechanisms of human input, derive statistical models, and show that human input always contains deviations, which exacerbate further when the relative object-camera motion becomes large. We also design and evaluate alternative refinement schemes, and propose a strategy that refits an object window on the most probable target region after a single click. To compensate for the human initialization errors, our method generates window proposals using objectness cues extracted from color and motion attributes, accumulates them into a likelihood map that is weighted by the initial click position and visual saliency scores, and assigns the final window by the maximum likelihood estimate. Our experiments demonstrate that the presented refinement strategy effectively reduces human input errors.
A hidden Markov model framework for traffic event detection using video features

A novel approach for highway traffic event detection in video is presented. The proposed algorithm extracts event features directly from compressed video and detects traffic event using a Gaussian mixture hidden Markov model (GMHMM). First, an invariant feature vector is extracted from discrete cosine transform (DCT) domain and macro-block vectors after MPEG video stream is parsed. The extracted feature vector accurately describes the change of traffic state and is robust towards different camera setups and illumination situations, such as sunny, cloud, and night. Six traffic patterns are studied and a GMHMM is trained to model these patterns in offline stage. Then, Viterbi algorithm is used to determine the most likely traffic condition. The proposed algorithm is efficient both in terms of computational complexity and memory requirement. The experimental results prove the system has a high detection rate. The presented model based system can be easily extended for detection of similar traffic events.
Geometric Sequence (GS) imaging with Bayesian smoothing for optical and capacitive imaging sensors

In this paper, we introduce a novel technique called geometric sequence (GS) imaging, specifically for the purpose of low power and light weight tracking in human computer interface design. The imaging sensor is programmed to capture the scene with a train of packets, where each packet constitutes a few images. The delay or the baseline associated with consecutive image pairs in a packet follows a fixed ratio, as in a geometric sequence. The image pair with shorter baseline or delay captures fast motion, while the image pair with larger baseline or delay captures slow motion. Given an image packet, the motion confidence maps computed from the slow and the fast image pairs are fused into a single map. Next, we use a Bayesian update scheme to compute the motion hypotheses probability map, given the information of prior packets. We estimate the motion from this probability map. The GS imaging system reliably tracks slow movements as well as fast movements, a feature that is important in realizing applications such as a touchpad type system. Compared to continuous imaging with short delay between consecutive pairs, the GS imaging technique enjoys several advantages. The overall power consumption and the CPU load are significantly low. We present results in the domain of optical camera based human computer interface (HCI) applications, as well as for capacitive fingerprint imaging sensor based touch pad systems.
Higher Order Energies for Image Segmentation

A novel energy minimization method for general higher order binary energy functions is proposed in this paper. We first relax a discrete higher order function to a continuous one, and use the Taylor expansion to obtain an approximate lower order function, which is optimized by the quadratic pseudo-Boolean optimization or other discrete optimizers. The minimum solution of this lower order function is then used as a new local point, where we expand the original higher order energy function again. Our algorithm does not restrict to any specific form of the higher order binary function or bring in extra auxiliary variables. For concreteness, we show an application of segmentation with the appearance entropy, which is efficiently solved by our method. Experimental results demonstrate that our method outperforms the state-of-the-art methods.
Finetuning Convolutional Neural Networks for visual aesthetics

Inferring the aesthetic quality of images is a challenging computer vision task due to its subjective and conceptual nature. Most image aesthetics evaluation approaches focused on designing handcrafted features, and only a few adopted learning of relevant and imperative characteristics in a data-driven manner. In this paper, we propose to attune Convolutional Neural Networks (CNNs) for image aesthetics. Unlike previous deep learning based techniques, we employ pretrained models, namely AlexNet  and the 16-layer VGGNet [20], and calibrate them to estimate visual aesthetic quality. This enables exploiting automatically the inherent information from much larger scale and more diversified image datasets. We tested our methods on AVA and CUHKPQ image aesthetics datasets on two different training-testing partitions, and compared the performance using both local and contextual information. Experimental results suggest that our strategy is robust, effective and superior to the state-of-the-art approaches.
Shadow flow: a recursive method to learn moving cast shadows

We present a novel algorithm to detect and remove cast shadows in a video sequence by taking advantage of the statistical prevalence of the shadowed regions over the object regions. We model shadows using multivariate Gaussians. We apply a weak classifier as a pre-filter. We project shadow models into a quantized color space to update a shadow flow function. We use shadow flow, background models, and current frame to determine the shadow and object regions. This method has several advantages: It does not require a color space transformation. We pose the problem in the RGB color space, and we can carry out the same analysis in other Cartesian spaces as well. It is data-driven and adapts to the changing shadow conditions. In other words, accuracy of our method is not limited by the preset values. Furthermore, it does not assume any 3D models for the target objects or tracking of the cast shadows between frames. Our results show that the detection performance is superior than the benchmark method.
Multi-Kernel Object Tracking

In this paper, we present an object tracking algorithm for the low-frame-rate video in which objects have fast motion. The conventional mean-shift tracking fails in case the relocation of an object is large and its regions between the consecutive frames do not overlap. We provide a solution to this problem by using multiple kernels centered at the high motion areas. In addition, we improve the convergence properties of the mean-shift by integrating two likelihood terms, background and template similarities, in the iterative update mechanism. Our simulations prove the effectiveness of the proposed method
CycleStack: Inferring periodic behavior via temporal sequence visualization in ultrasound video

A range of well-known treatment methods for destroying tumor and similar harmful growth in human body utilizes the coherence between the inherently periodic movement of the affected body part and periodic respiratory signal of the patient, with the objective of minimizing damage to surrounding normal tissues. Such methods require constant monitoring by an operator who observes the 3D body motion via its 2D projection onto an ultrasound imaging plane and studies the synchronism of this motion with the respiratory signal. Keeping an attentive eye on the respiratory signal as well as the ultrasound video for the entire treatment period is often inconvenient and burdensome. In this paper, we propose a video visualization technique called CycleStack Plot which reduces this cognitive overhead by blending the video and the signal together in a stack-like layout. This visualization reveals the inherent synchronism between the target's movement and the respiratory signal, visually highlights significant phase shifts of either of the two cyclic phenomena, with the hope of arresting the operator's attention. Our proposed visualization also provides a visual overview for the post-treatment analysis which enables educated users to quickly and effectively skim through the excessively long process. This paper demonstrates the utility of CycleStack Plot with a case study using real ultrasound videos. In addition, a user study has been performed to evaluate the merits and limitations of the proposed method with respect to the conventional way of watching a video and a signal side-by-side. Even though the motivation of the proposed visualization is improvement of medical applications that use ultrasound, the core techniques discussed here have potential to be extended to other application domains requiring analysis of cyclic patterns from videos.
Accurate detection of edge orientation for color and multi-spectral imagery

The frequency domain properties of an image are used for precise detection of edge orientation in color and multi-spectral imagery. The orientation estimation is established as a minimization problem, formulated as a tensor method, and simplified by solving its dual in terms of spatial partial derivatives of the image. First, the spectral density distribution around each pixel is obtained. The edge orientation is determined by fitting a straight line to this distribution. A matching error is devised in tensor form, and minimized by rotating the frequency domain principal axes. The orientation is computed from the spatial derivatives by transposing frequency domain operations to the spatial domain. The estimated edge orientations and magnitudes for different bands are converted to vectors and summed in the vector domain. A comparison of this method with the widely used estimators shows that the adapted tensor method improves the estimation precision even in the presence of extreme noise.
KAIST Multi-Spectral Day/Night Data Set for Autonomous and Assisted Driving

We introduce the KAIST multi-spectral data set, which covers a great range of drivable regions, from urban to residential, for autonomous systems. Our data set provides the different perspectives of the world captured in coarse time slots (day and night), in addition to fine time slots (sunrise, morning, afternoon, sunset, night, and dawn). For all-day perception of autonomous systems, we propose the use of a different spectral sensor, i.e., a thermal imaging camera. Toward this goal, we develop a multi-sensor platform, which supports the use of a co-aligned RGB/Thermal camera, RGB stereo, 3-D LiDAR, and inertial sensors (GPS/IMU) and a related calibration technique. We design a wide range of visual perception tasks including the object detection, drivable region detection, localization, image enhancement, depth estimation, and colorization using a single/multi-spectral approach. In this paper, we provide a description of our benchmark with the recording platform, data format, development toolkits, and lessons about the progress of capturing data sets.
Multiview Photometric Stereo Using Planar Mesh Parameterization

We propose a method for accurate 3D shape reconstruction using uncalibrated multiview photometric stereo. A coarse mesh reconstructed using multiview stereo is first parameterized using a planar mesh parameterization technique. Subsequently, multiview photometric stereo is performed in the 2D parameter domain of the mesh, where all geometric and photometric cues from multiple images can be treated uniformly. Unlike traditional methods, there is no need for merging view-dependent surface normal maps. Our key contribution is a new photometric stereo based mesh refinement technique that can efficiently reconstruct meshes with extremely fine geometric details by directly estimating a displacement texture map in the 2D parameter domain. We demonstrate that intricate surface geometry can be reconstructed using several challenging datasets containing surfaces with specular reflections, multiple albedos and complex topologies.
High-Resolution Terrain Map from Multiple Sensor Data

The authors present 3-D vision techniques for incrementally building an accurate 3-D representation of rugged terrain using multiple sensors. They have developed the locus method to model the rugged terrain. The locus method exploits sensor geometry to efficiently build a terrain representation from multiple sensor data. The locus method is used to estimate the vehicle position in the digital elevation map (DEM) by matching a sequence of range images with the DEM. Experimental results from large-scale real and synthetic terrains demonstrate the feasibility and power of the 3-D mapping techniques for rugged terrain. In real world experiments, a composite terrain map was built by merging 125 real range images. Using synthetic range images, a composite map of 150 m was produced from 159 images. With the proposed system, mobile robots operating in rugged environments can build accurate terrain models from multiple sensor data.
Measuring conceptual relation of visual words for visual categorization

Representing image using the distribution of local features on a group of visual words is an effective method for visual categorization. Visual words can be related conceptually and the information can be incorporated to enhance the performance. However, conventional methods usually use visual words independently without considering this. This paper proposes a novel approach to measure the conceptual relation of visual words and incorporate the information into visual categorization. The conceptual relation is measured by the similarity of class distributions induced by visual words, accordingly visual words are grouped and images are represented on multiple levels. Categorization is taken using the support vector machine (SVM) with an effective kernel designed for matching multi-level representations. The proposed method is evaluated for video events categorization on the benchmark dataset and shows superior performance to conventional methods.
Artrieval: Painting retrieval without expert knowledge

As people are becoming interested in paintings, various user-interactive search systems have been presented in recent times. Many systems encourage users to search paintings by prior knowledge on paintings. We discover the limitation for existing methods on how well the query is represented by the user, and propose a simple, yet effective way to search the painting by exploiting the color to express human visual memory. To achieve our goal, we suggest color clustering based on human color perception, and hierarchical metric learning to accommodate the locality of colors. With user-interactive drawing through learned colors, the user completes the abstract image to resemble the visual memory. We show that our system is easy to use, fast to process, accurate to search and fully extensible to cover deviation among users.
Reducing ambiguity in feature point matching by preserving local geometric consistency

In this paper, feature point matching is formulated as an optimization problem in which the uniqueness condition is constrained. We propose a novel score function based on homography-induced pairwise constraints, and a novel optimization algorithm based on relaxation labeling. Homography-induced pairwise constraints are effective for image pairs with viewpoint or scale changes, unlike previous pairwise constraints. The proposed optimization algorithm searches for a uniqueness-constrained solution, while the original relaxation-labeling algorithm is appropriate for finding many- to-one correspondences. The effectiveness of the proposed method is shown by experiments involving image pairs with viewpoint or scale changes in addition to repeated textures and nonrigid deformation. The proposed method is also applied to object recognition, giving some promising results.
Disjoint Multi-task Learning Between Heterogeneous Human-Centric Tasks

Human behavior understanding is arguably one of the most important mid-level components in artificial intelligence. In order to efficiently make use of data, multi-task learning has been studied in diverse computer vision tasks including human behavior understanding. However, multitask learning relies on task specific datasets and constructing such datasets can be cumbersome. It requires huge amounts of data, labeling efforts, statistical consideration etc. In this paper, we leverage existing single-task datasets for human action classification and captioning data for efficient human behavior learning. Since the data in each dataset has respective heterogeneous annotations, traditional multi-task learning is not effective in this scenario. To this end, we propose a novel alternating directional optimization method to efficiently learn from the heterogeneous data. We demonstrate the effectiveness of our model and show performance improvements on both classification and sentence retrieval tasks in comparison to the models trained on each of the single-task datasets.
Simultaneous Classification and VisualWord Selection using Entropy-based Minimum Description Length

In this paper, we present a new entropy-based minimum description length (MDL) criterion for simultaneous classification and visual word selection. Conventional MDL criteria focus on how to minimize cluster size and maximize the likelihood of data points. We extend the MDL by replacing the likelihood term with the entropy of class posterior. This new criterion can provide optimal visual words with enough classification accuracy. We validate the entropy-based MDL to learn optimal visual words for place classification and categorization of the Caltech 101 object database
Automatic edge detection method for the mobile robot application

This paper proposes a new edge detection method using a 3/spl times/3 ideal binary pattern and lookup table (LUT) for the mobile robot localization without any parameter adjustments. We take the mean of the pixels within the 3/spl times/3 block as a threshold by which the pixels are divided into two groups. The edge magnitude and orientation are calculated by taking the difference of average intensities of the two groups and by searching directional code in the LUT, respectively. And also the input image is not only partitioned into multiple groups according to their intensity similarities by the histogram, but also the threshold of each group is determined by fuzzy reasoning automatically. Finally, the edges are determined through non-maximum suppression using edge confidence measure and edge linking. Applying this edge detection method to the mobile robot localization using projective invariance of the cross ratio, we demonstrate the robustness of the proposed method to the illumination changes in a corridor environment.
Combinatorial approach for lane detection using image and LIDAR reflectance

Recently, lane detection algorithms have played significant roles in the field of vehicle technology. While many well working algorithms have been developed, they are hard to use in complex urban environments. In this paper, we propose an efficient approach for detecting lane markings using image information and LIDAR reflectance. The proposed algorithm has three phases: ground extraction, lane detections, and combining lane information. The proposed algorithm was implemented on a real vehicle and validated in various traffic environments, including the 2014 Hyundai Autonomous Vehicle Competition (AVC).