nils-herrmann/clean_scirepeval huggingface dataset:
Self-calibration from vehicle information

In this paper, we describe a method to automatically calibrate a static camera using only information regarding the vehicles in the scene. We use standard Adaboost vehicle detection combined with feature-based tracking to determine the size and direction of travel of vehicles at various locations in the scene. We then infer the scene geometry including the height of the camera, the camera focal length, and the angle of the camera with respect to the ground. Given the calibration, we can estimate the size and speed of objects and the distances travelled in the scene. We evaluate our system on 10 cameras with manually annotated calibration based on a 3D cube with user specified locations in 2D. We show our system can be used to accurately estimate vehicle sizes to within 2 feet for a range of camera configurations.
Multimodal ranking for non-compliance detection in retail surveillance

In retail stores, cashier non-compliance activities at the Point of Sale (POS) are one of the prevalent sources of retail loss. In this paper, we propose a novel approach to reliably rank the list of detected non-compliance activities of a given retail surveillance system, thereby provide a means of significantly reducing the false alarms and improving the precision in non-compliance detection. Our approach represents each detected non-compliance activity using multi-modal features coming from video data, transaction logs (TLog) data and intermediate results of the video analytics. We then learn a binary classifier that successfully separate true positives and false positives in a labeled training set. A confidence score for each detection can then be computed using the decision value of the trained classifier, and a ranked list of detections can be formed based on this score. The benefit from having this ranked list is two-fold. First, a large number of false alarms can be avoided by simply keeping the top part of the list and discarding the rest. Second, a trade off between precision and recall can easily be performed by sliding the discarding threshold along this ranked list. Experimental results on a large scale dataset captured from real stores demonstrate that our approach achieves better precision than a state-of-the-art system at the same recall. Our approach can also reach an operating point that exceeds the retailers' expectation in terms of precision, while retaining an acceptable recall of more than 60%.
Random Laplace Feature Maps for Semigroup Kernels on Histograms

With the goal of accelerating the training and testing complexity of nonlinear kernel methods, several recent papers have proposed explicit embeddings of the input data into low-dimensional feature spaces, where fast linear methods can instead be used to generate approximate solutions. Analogous to random Fourier feature maps to approximate shift-invariant kernels, such as the Gaussian kernel, on Rd, we develop a new randomized technique called random Laplace features, to approximate a family of kernel functions adapted to the semigroup structure of R+d. This is the natural algebraic structure on the set of histograms and other non-negative data representations. We provide theoretical results on the uniform convergence of random Laplace features. Empirical analyses on image classification and surveillance event detection tasks demonstrate the attractiveness of using random Laplace features relative to several other feature maps proposed in the literature.
Temporal Modeling of Slide Change in Presentation Videos

We develop a general framework to automatically match electronic slides to the videos of the corresponding presentations. The synchronized slides support indexing and browsing of educational and corporate digital video libraries. Our approach extends previous work that matches slides based on visual features alone, and integrates multiple cues to further improve performance in more difficult cases. We model slide change in a presentation with a dynamic hidden Markov model (HMM) that captures the temporal notion of slide change and whose transition probabilities are adapted locally by using the camera events in the inference process. Our results show that combining multiple cues in a state model can greatly improve the performance in ambiguous cases.
Temporal Sequence Modeling for Video Event Detection

We present a novel approach for event detection in video by temporal sequence modeling. Exploiting temporal information has lain at the core of many approaches for video analysis (i.e., action, activity and event recognition). Unlike previous works doing temporal modeling at semantic event level, we propose to model temporal dependencies in the data at sub-event level without using event annotations. This frees our model from ground truth and addresses several limitations in previous work on temporal modeling. Based on this idea, we represent a video by a sequence of visual words learnt from the video, and apply the Sequence Memoizer [21] to capture long-range dependencies in a temporal context in the visual sequence. This data-driven temporal model is further integrated with event classification for jointly performing segmentation and classification of events in a video. We demonstrate the efficacy of our approach on two challenging datasets for visual recognition.
Practical computer vision: Example techniques and challenges

Humans, as well as many living organisms, are gifted with the power of "seeing" and Bunderstanding[ the environment around them using their eyes. The ease with which humans process and understand the visual world is very deceiving and often prompts us to underestimate the effort and methods needed to build practical, effective, and inexpensive computer vision systems. In essence, humans have a 500-million-year head start due to evolution; it is extremely difficult at this point to build a computer vision system that has the abilities of a three-year-old child. However, by confining ourselves to particular domains, we can often find shortcuts to solve particular problems. This paper illustrates a number of such solutions in various areas developed by our group at IBM. These include object finding for video surveillance, person identification via biometrics, inspection of manufactured items along railways, and scene understanding for driver assistance, as well as object recognition and motion interpretation for retail stores. We discuss the real-world constraints for each system and describe how we overcame the irksome variability inherent in each task. By further analyzing such successful systems and comparing them to each other, we can come to understand the common underlying problems and thus start to extend our initially limited areas of competence into a more general-purpose vision toolkit. This paper concludes with a set of challenging unresolved problems that if solved could spur great progress in practical computer vision.
An integer programming approach to visual compliance

Visual compliance has emerged as a new paradigm to ensure that employees comply with processes and policies in a business context . In this paper, we focus on videos from retail stores and formulate a binary integer program for detecting checkout events, which enables enforcing visual compliance in such an environment. The proposed integer program maximizes the essential quantities that characterize true events of interest, subject to an array of constraints. In particular, the binary decision variables correspond to the presence of a set of hypothesized visual events. In the objective function, the binary variables are weighted by quality measures derived from infinite Gaussian mixture modeling of the video content, such that maximizing the overall quality measure is expected to uncover the meaningful visual events. Our framework is tested and validated on videos recorded at checkout lanes, and leads to better performance than previous methods.
Modeling of temporarily static objects for robust abandoned object detection in urban surveillance

We propose a robust approach for abandoned object detection in urban surveillance with over thousands of cameras. For such a large-scale monitoring based on intelligent video analysis, it is critical that a system be designed with careful control of false alarms. Our approach is based on proactive modeling of temporally static objects (TSO) such as cars stopping at red light and still pedestrians in the street. We develop a finite state machine to track the entire life cycles of TSOs from creation to termination. The semantically meaningful object information provided by the state machine in turn allows adaptive region-level updating of the background model without using any sophisticated object classification techniques. We demonstrate that our approach significantly mitigates the problematic issue of false alarm related to people in city surveillance, using both a small publicly available data set and a large one collected from various realistic urban scenarios.
Robust image registration using phase congruency

Image registration plays an important role in many image processing applications. This paper proposed a robust image registration method that can estimate motion parameters including translation and rotation from a group of different-exposed images. Brightness change and photometric distortion, i.e., under-exposed or over-exposed, between images will result in inaccuracy of the estimation. To solve this problem, we propose to firstly transform raw image data into phase congruency that is invariant to the changes in image brightness or contrast and then apply phase cross correlation in the Fourier transform domain to estimate motion parameters. Experiment results show that the proposed algorithm is efficient and robust against photometric distortion compared with other phase correlation based registration algorithms.
Hierarchical Decomposition Based Consensus Tracking for Uncertain Interconnected Systems via Distributed Adaptive Output Feedback Control

In this note, distributed adaptive controllers are developed for output consensus tracking of multiple linear systems with unknown parameters, uncertain subsystem interconnections and external disturbances. The subsystems are allowed to have non-identical dynamics and the same yet arbitrary order. It is assumed that only part of subsystems can have direct access to the time-varying trajectory information and the subsystem states are unmeasurable for local feedback control. In our design, the directed graph representing the information transmission status among subsystems is preprocessed by splitting it into a hierarchical structure. Then local adaptive controllers of subsystems in different layers can be computed in a sequential order and the difficulty on deriving mutually dependent local controls in previous consensus works are successfully overcome. In each subsystem, additional estimates are introduced to account for the unknown parameters and states in its neighbors' dynamics. Besides, only the information of local outputs and inputs need be collected from the neighboring subsystems. Certain robust terms are added in distributed adaptive laws to mitigate the effects of uncertain subsystem interactions and disturbances. It is proved that with our scheme, all closed-loop signals can be ensured bounded when the strengths of uncertain subsystem interconnections are sufficiently weak. The tracking errors for the entire group of subsystems will converge to a compact set and the transient tracking error performance can be adjusted by appropriately choosing design parameters.
Weighted Guided Image Filtering

It is known that local filtering-based edge preserving smoothing techniques suffer from halo artifacts. In this paper, a weighted guided image filter (WGIF) is introduced by incorporating an edge-aware weighting into an existing guided image filter (GIF) to address the problem. The WGIF inherits advantages of both global and local smoothing filters in the sense that: 1) the complexity of the WGIF is O(N) for an image with N pixels, which is same as the GIF and 2) the WGIF can avoid halo artifacts like the existing global smoothing filters. The WGIF is applied for single image detail enhancement, single image haze removal, and fusion of differently exposed images. Experimental results show that the resultant algorithms produce images with better visual quality and at the same time halo artifacts can be reduced/avoided from appearing in the final images with negligible increment on running times.
Fast correction of moving regions for differently exposed images with moving objects

When a high dynamic range (HDR) image is synthesized by using a set of differently exposed low dynamic range (LDR) images, it is important to design an anti-ghost scheme to prevent ghosting artifacts from the final HDR image. An anti-ghosting process is typically comprised of detection and correction of moving regions. In this paper, we propose a new method to estimate intensity mapping functions (IMFs) between two differently exposed LDR images. The accuracy of IMFs is improved for under/over-exposed pixels. The new IMFs are then adopted to design a fast correction approach. Experimental results show that the quality of final image can be maintained while the time taken to correct the moving regions reduced.
On-chip phase shift refractometer via microfluidic liquid grating

This paper describes the design, realization and testing of a refractometer based on the principle of transmission phase grating. An integrated multichannel microfluidic chip is used as a basic optical component for building the refractometer. A laser light beam is coupled to the top of the microfluidic chip and the generated diffraction pattern is recorded by a CCD camera and the intensities of the 0th and ±1st order are further analyzed. The microfluidic chip is fabricated using standard soft lithography technology. The realized refractometer has a refractive index resolution of 1.0 × 10−5.
Fast Mode Decision Algorithm for Inter-Frame Coding in Fully Scalable Video Coding

Scalable video coding is an ongoing standard, and the current working draft (WD) is an extension of H.264/AVC. In the WD, an exhaustive search technique is employed to select the best coding mode for each macroblock. This technique achieves the highest possible coding efficiency, but it results in extremely large encoding time which obstructs it from practical use. This paper proposes a fast mode decision algorithm for inter-frame coding for spatial, coarse grain signal-to-noise ratio, and temporal scalability. It makes use of the mode-distribution correlation between the base layer and enhancement layers. Specifically, after the exhaustive search technique is performed at the base layer, the candidate modes for enhancement layers can be reduced to a small number based on the correlation. Experimental results show that the fast mode decision scheme reduces the computational complexity significantly with negligible coding loss and bit-rate increases
Spatial evaluation of crop maps by the spatial production allocation model in China

The spatial production allocation model (SPAM) is one of the broadest spatial models of crop distribution and applies a cross-entropy method to downscale the global area and yield for multiple crops in the years 2000 and 2005 with a resolution of 5 arc min. To evaluate the allocation accuracy of SPAM for three staple crops (rice, wheat, and maize) in China, we com- pared these crop maps with remote-sensed cropland data derived from national land cover data- sets. This comparison was conducted using a scheme that accounts for spatial differences at the pixel level. Overall, the map of maize has the highest area accuracy, with 64% reasonable pixels (covering 96% of the total maize area); these values were 57% (90% coverage) and 44% (81% coverage) for the wheat and rice maps, respectively. On the provincial scale, the area accuracies of crop maps in the top 10 provinces are better than those of the other provinces. Furthermore, the crop area consistency in rain-fed cropland is better than that in irrigated cropland. These evaluations provide decision makers with information regarding the strengths and weaknesses of SPAM products. This study also recommends priorities for further work to improve the reli- ability, utility, and periodic repeatability of crop distribution products.
Automatic synthetic background defocus for a single portrait image

Capturing image with defocused background by using a large aperture is a widely used technique in digital single-lens reflex (DSLR) camera photography. It is also desired to provide this function to smart phones. In this paper, a new algorithm is proposed to synthesize such an effect for a single portrait image. The foreground portrait is detected using a face prior based salient object detection algorithm. Then with an improved gradient domain guided image filter, the details in the foreground are enhanced while the background pixels are blurred. In this way, the background objects are defocused and thus the foreground objects are emphasized. The resultant image looks similar to image captured using a camera with a large aperture. The proposed algorithm can be adopted in smart phones, especially for the front cameras of smart phones.
A robust and fast anti-ghosting algorithm for high dynamic range imaging

This paper presents a robust and fast algorithm for automatically generating high dynamic range (HDR) images in presence of camera movement and moving objects. This scheme comprises five modules: 1) image alignment, 2) estimation of camera response function (CRF) in dynamic scenes, 3) moving object detection, 4) progressive image correction, and 5) construction of HDR images. The key advantage of the algorithm is the ability to generate HDR images without ghost artifact. The proposed algorithm is fast as it is a one-shot solution without iterative computation and post-processing or even manual operation. Experimental results demonstrate that the proposed method outperforms the existing commercial products.
Joint rate control and scheduling for video streaming over OFDMA networks

In this paper, we investigate joint rate control and packet scheduler for video streaming over downlink orthogonal frequency division multiple access (OFDMA) networks. We proposed algorithm which exploits rate based utility functions and the video packet's importance information which is related to its position in the group of pictures. The time scale of rate control is larger than that of packet scheduler. The importance information for scheduling is embedded in the IP packets, making it more suitable for practical purposes. We extend the Lyapunov optimization technique for single time scale network to develop a low complexity algorithm, which does not require prior statistical information on the channel and queues. The algorithm also has a bounded worst case delay, which is useful for incorporating deadline constraints for video streaming. The performance of the proposed algorithm is evaluated by simulation in a LTE network. Simulation results show that our algorithm outperforms existing techniques.
Video streaming on embedded devices through GPRS network

We introduce a PDA-based live video streaming system on GPRS network based on MPEG-4 video compression standard. Due to the limited computational resources of PDA, all the key modules of MPEG-4 codec are efficiently implemented and optimized such as multithreading, buffer design, wireless communication, encoder and decoder. Several novel techniques are developed in the coding, streaming as well as the post- processing stages of the system.
An adaptive rate control algorithm for video coding over personal digital assistants (PDA)

With the recent development of third-generation communication technologies, encoding live video using a PDA and sharing it among friends has become a reality. However, the embedded processor inside a PDA is still not powerful enough and there are two major hurdles to overcome: (1) video coding needs to meet the rigorous constraint of the available computation capacity of a PDA; (2) In a PDA the computing power allocated to video coding may vary drastically (in bursts). In this paper, a new adaptive rate control algorithm is proposed for video coding over a PDA. This adaptive rate control scheme takes into account the time constraint of a PDA, and its bit allocation depends not only on the available data bits, but more importantly, on the available coding time. Experimental results show that, compared to the existing rate control scheme, the new algorithm can always achieve the maximum frame rate, maximize the utilization of the available bandwidth and computing power, increase the average PSNR, and improve the subjective perceptual quality of the reconstructed video.
Region-of-interest based local tone mappings

This paper introduces region of interest (ROI) as a functionality to local tone mappings of high dynamic range (HDR) radiance maps. In the proposed scheme, a new quadratic regularization method is provided to decompose the luminance of an HDR radiance map into a base layer with large gradients and a detail layer with small gradients. An ROI based global tone mapping is designed to compress the base layer. Since the dynamic range of a display device is limited, more dynamic ranges are assigned to the base layer in the ROI such that the visual quality of the ROI is preserved better. The detail layer can be amplified to enhance local contrasts such that the output LDR images look sharper. Both automatic and interactive modes are provided. With our automatic mode, the generated low dynamic range (LDR) images look natural and local details are also preserved very well. In addition, a user friendly interactive mode is provided which is also able to produce dynamic LDR images in real time.
Visual-Salience-Based Tone Mapping for High Dynamic Range Images

Visual saliency aims to predict the attentional gaze of observers viewing a scene, and it is thus highly demanded for tone mapping of high dynamic range (HDR) images. In this paper, novel saliency-aware weighting and edge-aware weighting are introduced for HDR images. They are incorporated into an existing guided image filter to form a perceptually guided image filter. The saliency-aware weighting and the proposed filter are applied to design a new local tone-mapping algorithm for HDR images such that both extreme light and shadow regions can be reproduced on conventional low dynamic range displays. In particular, the proposed filter is applied to decompose the luminance of the input HDR image into a base layer and a detail layer. The saliency-aware weighting is then adopted to design a saliency-aware global tone mapping for the compression of the base layer. The proposed filter preserves sharp edges in the base layer better than the existing guided filter. Halo artifacts are thus significantly reduced in the tone-mapped image. Moreover, the visual quality of the tone-mapped image, especially attention-salient regions, is improved by the saliency-aware weighting.
Image Quality Assessment using Foveated Wavelet Error Sensitivity and Isotropic Contrast

In this paper, we propose a new image quality evaluation method, which is based on foveation spatial error sensitivity model and wavelet error sensitivity model. The image quality is measured by isotropic contrast and the standard deviation of the errors of the wavelet coefficients in subbands with the weighting factors that are determined by two models. Experiments on a test database comprising JPEG and JPEG 2000 compressed images have shown that the proposed metric can achieve very good correlation with subjective evaluation.