nils-herrmann/clean_scirepeval huggingface dataset:
A model-based sampling and sample synthesis method for auto identification in computer vision

The need for a large sample size grows exponentially with the dimensionality of the feature space ("curse of dimensionality"), which increases the labor cost during the training procedure and severely restricts the number of the practical applications. While feature selection methods can often alleviate the problems associated with the curse of dimensionality, complex large scale pattern recognition problems may not be amenable to features selection approach due to large intrinsic dimensionality. In such situations, the only effective solution to conquer the complications of the high-dimensional functions is to incorporate knowledge about the data that is correct. How to incorporate the domain knowledge with the specific machine learning system has been widely studied in the pattern classification field. In this paper, we will explore a novel method to synthesize a larger, valid training sample data set based on a smaller set of the key samples that are collected by a model based sampling theory that incorporates the domain knowledge of the computer vision. In addition to reducing the training sample size in the learning procedure, our emphasis is on providing practical advice on how to incorporate domain knowledge to design and simplify a vision based pattern classification model.
Cart Auditor: A Compliance and Training Tool for Cashiers at Checkout

Shopping carts have traditionally been used as a tool provided to the customers in retail stores to carry items from the shelf to checkout stations. These days shopping carts can also be used as a security checkpoint to prevent store losses. All the items collected in a shopping cart are supposed to be unloaded at the checkout station to be scanned and included in the bill. Any items left in the cart intentionally or by accident will not be charged and therefore cause a loss to the store. We propose a system that automatically detects shopping carts and verify their emptiness at the checkout station. We use motion segmentation, line detection, and template matching methods for the cart detection and emptiness verification. An inter-frame edge difference, cart’s path accumulator, and a finite state model are introduced for accurate cart detection. All detected carts are compared with empty cart models and the dissimilarity scores are calculated to verify the emptiness. The proposed system was evaluated on a long video clip (~12 hours) and showed promising results both in cart detection and emptiness verification.
Adaptive as-natural-as-possible image stitching

The goal of image stitching is to create natural-looking mosaics free of artifacts that may occur due to relative camera motion, illumination changes, and optical aberrations. In this paper, we propose a novel stitching method, that uses a smooth stitching field over the entire target image, while accounting for all the local transformation variations. Computing the warp is fully automated and uses a combination of local homography and global similarity transformations, both of which are estimated with respect to the target. We mitigate the perspective distortion in the non-overlapping regions by linearizing the homography and slowly changing it to the global similarity. The proposed method is easily generalized to multiple images, and allows one to automatically obtain the best perspective in the panorama. It is also more robust to parameter selection, and hence more automated compared with state-of-the-art methods. The benefits of the proposed approach are demonstrated using a variety of challenging cases.
A fast and robust intelligent headlight controller for vehicles

We describe a system that controls whether the headlights of a vehicle are in the highbeam or lowbeam state based on input from a forward looking video camera. The core of the system relies on conventional computer vision techniques, albeit with a sophisticated spot finder front-end. Despite this architecture we are able to use an automated supervised learning technique to tune the system to yield high performance. Using a customer-imposed metric we present both in-car and off-line results from our system along with several competitors, and investigate the system's performance under different weather conditions.
Intelligent headlight control using learning-based approaches

This paper describes our recent work on developing an intelligent headlight control system using machine learning-based approaches. Specifically, such a system aims to automatically control a vehicle's beam state (high beam or low beam) during a night-time drive based on the detection of oncoming/overtaking/leading traffics as well as urban areas from the videos captured by a camera. Two machine learning-based approaches, namely, support vector machine (SVM) and AdaBoost, have been applied to accomplish this task. The architect of each approach, as well as its detailed processing modules, will be elaborated in the paper. The system has been extensively tested both online and offline to validate the robustness and effectiveness of the two proposed approaches. A detailed performance study along with some comparisons between the two approaches will be reported at the end.
A filterbank-based representation for classification and matching of fingerprints

We view fingerprints as oriented texture patterns, textures which exhibit an inherent and well-defined sense of directionality. Given a fingerprint image, we demonstrate that reliable translation and rotation invariant representations can be built based entirely on the inherent properties of the underlying fingerprint texture. We also illustrate that the representations thus derived are useful for robust discrimination of the fingerprints. In particular, we present the application of our novel representation scheme for solving the problems of fingerprint classification and matching on large datasets of fingerprint images acquired in real situations. The proposed scheme of generic representation for oriented textures relies on extracting one or more invariant frames of reference of the oriented texture based on an analysis of its orientation field. The fingerprint classification is based on a two-stage classifier which uses a K-NN classifier in its first stage and a set of neural network classifiers in its second stage.
Analytics-driven asset management

Asset-intensive businesses across industries rely on physical assets to deliver services to their customers, and effective asset management is critical to the businesses. Today, businesses may make use of enterprise asset-management (EAM) solutions for many asset-related processes, ranging from the core asset-management functions to maintenance, inventory, contracts, warranties, procurement, and customer-service management. While EAM solutions have transformed the operational aspects of asset management through data capture and process automation, the decision-making process with respect to assets still heavily relies on institutional knowledge and anecdotal insights. Analytics-driven asset management is an approach that makes use of advanced analytics and optimization technologies to transform the vast amounts of data from asset management, metering, and sensor systems into actionable insight, foresight, and prescriptions that can guide decisions involving strategic and tactical assets, as well as customer and business models.
Enhanced rail component detection and consolidation for rail track inspection

For safety purposes, railroad tracks need to be inspected on a regular basis for physical defects or design noncompliances. Such track defects and non-compliances, if not detected in a timely manner, may eventually lead to grave consequences such as train derailments. In this paper, we present a real-time automatic vision-based rail inspection system, with main focus on anchors - an important rail component type, and anchor-related rail defects, or exceptions. Our system robustly detects important rail components including ties, tie plates, anchors with high accuracy and efficiency. Detected objects are then consolidated across video frames and across camera views to map to physical rail objects, by combining the video data streams from all camera views with GPS information and speed information from the distance measuring instrument (DMI). After these rail components are detected and consolidated, further data integration and analysis is followed to detect sequence-level track defects, or exceptions. Quantitative analysis performed on a real online field test conducted on different track conditions demonstrates that our system achieves very promising performance in terms of rail component detection, anchor condition assessment, and compliance-level exception detection. We also show that our system outperforms another advanced rail inspection system in anchor detection.
Biometrics: a grand challenge

Reliable person identification is an important problem in diverse businesses. Biometrics, identification based on distinctive personal traits, has the potential to become an irreplaceable part of any identification system. While successful in some niche markets, the biometrics technology has not yet delivered its promise of foolproof automatic identification. With the availability of inexpensive biometric sensors and computing power, it is becoming increasingly clear that widespread usage of biometric person identification is being stymied by our lack of understanding of three fundamental problems; (i) How to accurately and efficiently represent and recognize biometric patterns? (ii) How to guarantee that the sensed measurements are not fraudulent? and (iii) How to make sure that the application is indeed exclusively using pattern recognition for the expressed purpose (function creep (A. K. JAin et al., December 1998))? Solving these core problems will not only catapult biometrics into mainstream applications but will also stimulate adoption of other pattern recognition applications for providing effective automation of sensitive tasks without jeopardizing our individual freedoms. For these reasons, we view biometrics as a grand challenge - "a fundamental problem in science and engineering with broad economic and scientific impact".
FingerCode: a filterbank for fingerprint representation and matching

With the identity fraud in our society reaching unprecedented proportions and with an increasing emphasis on the emerging automatic positive personal identification applications, biometrics-based identification, especially fingerprint-based identification, is receiving a lot of attention. There are two major shortcomings of the traditional approaches to fingerprint representation. For a significant fraction of population, the representations based on explicit detection of complete ridge structures in the fingerprint are difficult to extract automatically. The widely used minutiae-based representation does not utilize a significant component of the rich discriminatory information, available in the fingerprints. The proposed filter-based algorithm uses a bank of Gabor filters to capture both the local and the global details in a fingerprint as a compact 640-byte fixed length FingerCode. The fingerprint matching is based on the Euclidean distance between the two corresponding FingerCodes and hence is extremely fast. Our initial results show identification accuracies comparable to the best results of minutiae-based algorithms published in the open literature. Finally, we show that the matching performance can be improved by combining the decisions of the matchers based on complementary fingerprint information.
On the Individuality of Fingerprints

Fingerprint identification is based on two basic premises: (1) persistence and (2) individuality. We address the problem of fingerprint individuality by quantifying the amount of information available in minutiae features to establish a correspondence between two fingerprint images. We derive an expression which estimates the probability of a false correspondence between minutiae-based representations from two arbitrary fingerprints belonging to different fingers. Our results show that (1) contrary to the popular belief, fingerprint matching is not infallible and leads to some false associations, (2) while there is an overwhelming amount of discriminatory information present in the fingerprints, the strength of the evidence degrades drastically with noise in the sensed fingerprint images, (3) the performance of the state-of-the-art automatic fingerprint matchers is not even close to the theoretical limit, and (4) because automatic fingerprint verification systems based on minutia use only a part of the discriminatory information present in the fingerprints, it may be desirable to explore additional complementary representations of fingerprints for automatic matching.
A touch of money [biometric authentication systems]

This paper suggests the use of a new authentication system for credit cards based on biometric sensors that could dramatically curtail identity theft. The proposed system uses fingerprint sensors, though other biometric technologies, either alone or in combination, could be incorporated. It could be economical, protect privacy, and guarantee the validity of all kinds of credit card transactions, including ones that take place at a store, over the telephone, or with an Internet-based retailer. By preventing identity thieves from entering the transaction loop, credit card companies could quickly recoup their infrastructure investments and save businesses, consumers, and themselves billions of dollars every year.
Real-Time Traffic Sign Detection: An Evaluation Study

This paper presents an experimental evaluation of three different traffic sign detection approaches, which detect or localize various types of traffic signs from real-time videos. Specifically, the first approach exploits geometric features to identify traffic signs, while the other two are developed based on SVM (Support Vector Machine) and AdaBoost learning mechanisms. We describe each of the three approaches, conduct a detailed comparison among them, and examine their pros and cons. Our conclusions should lead to useful guidelines for developing a real-time traffic sign detector.
Boosting object detection performance in crowded surveillance videos

We present a novel approach to automatically create efficient and accurate object detectors tailored to work well on specific video surveillance cameras (specific-domain detectors), using samples acquired with the help of a more expensive, general-domain detector (trained using images from multiple cameras). Our method requires no manual labels from the target domain. We automatically collect training data using tracking over short periods of time from high-confidence samples selected by the general-domain detector. In this context, a novel confidence measure is proposed for detectors based on a cascade of classifiers, which are frequently adopted for computer vision applications that require real-time processing. We demonstrate our proposed approach on the problem of vehicle detection in crowded surveillance videos, showing that an automatically generated detector significantly outperforms the original general-domain detector with much less feature computations.
On integration of vision modules

Individual cues from visual modules are fallible and often ambiguous. As a result, only integrated vision systems can be expected to give a reliable performance in practice. The design of such systems is challenging since each vision module works under different and possibly conflicting sets of assumptions. We have proposed and implemented a multiresolution system which integrates perceptual grouping, segmentation, stereo, shape from shading, and line labelling modules. The output of the integrated system is shown to be relatively insensitive to the constraints imposed by the individual modules.<<ETX>>
Resilient, UAV-embedded real-time computing

In this paper, we propose a hierarchical computational system architecture to support the target domain of realtime mobile computing in the context of unmanned aerial vehicles (UAVs). The overall architectural vision includes support for system resilience in the presence of uncertainties in the operational environment of surveillance UAVs. We report measurement-based results that are obtained from a UAV proxy demonstration apparatus. The apparatus consists of a Raspberry Pi (RPi) board that serves as an on-board UAV computer, working with support from a laptop that serves as the on-ground computing infrastructure where an operator "consumes" video information received from the UAV. We quantify the gap between the on-board UAV camera frame rate (input) and the on-ground operator-observed frame rate (output) for a specialized class of computer vision applications germane to the UAV-based aerial surveillance domain. The goal is to keep the frame rate observed by the ground operator as close (or ideally equal) to the on-board UAV camera frame rate (i.e. to preserve the real-time aspect) despite the unstable bandwidth availability in the channel connecting both ends. The proposed hierarchical approach significantly outperforms two considered baselines: one in which computation takes place entirely on the UAV computer and another in which computation takes place entirely on the ground. This improved performance is due to a more balanced resource sharing between the on-board UAV computer and UAV-to-ground communication channel. Later, we show how the observed frame rate improves when the RPi board is replaced with an NVIDIA Jetson TK1 board. Based on the observations gleaned from these "proxy" experiments, we sketch the fundamentals of our ongoing work in model-based predictive analysis of resilient "UAV swarm" computational architectures of the future.
Resilient mobile cognition: Algorithms, innovations, and architectures

The importance of the internet-of-things (IOT) is now an established reality. With that backdrop, the phenomenal emergence of cameras/sensors mounted on unmanned aerial, ground and marine vehicles (UAVs, UGVs, UMVs) and body worn cameras is a notable new development. The swarms of cameras and real-time computing thereof are at the heart of new technologies like connected cars, drone-based city-wide surveillance and precision agriculture, etc. Smart computer vision algorithms (with or without dynamic learning) that enable object recognition and tracking, supported by baseline video content summarization or 2D/3D image reconstruction of the scanned environment are at the heart of such new applications. In this article, we summarize our recent innovations in this space. We focus primarily on algorithms and architectural design considerations for video summarization systems.
Initialization-independent spectral clustering with applications to automatic video analysis

Popular clustering algorithms, such as K-means (KM) and expectation maximization (EM), are sensitive to the initialization of cluster centers. In contrast, recently proposed K-harmonic means (KHM) algorithm is more robust to the randomness of the initialization. However, KHM works best when the dimensionality of the data (N) is small (usually less than 8). Because the dimensionality of features that are used for many clustering problems in image/video and speech processing is large, the benefits of KHM cannot be exploited. Based upon this observation, this paper proposes a novel method to employ KHM for high-dimensional data so as to realize initialization-independent clustering. The proposed method employs efficient spectral clustering techniques whereby the affinity matrix of the data is decomposed into its eigenvectors and k (total number of clusters) eigenvectors corresponding to the k largest eigenvalues are retained. That is, we represent N-D data by k-D transformed data when k < N and propose to employ KHM over this k-D transformed data. The assumption of k < N indeed encompasses a large number of significant video processing and computer vision problems where the use of KHM was not beneficial before. We demonstrate the effectiveness and the efficiency of the proposed algorithm for face clustering in the domains where the number of persons (k) is not large, such as anchorperson grouping, video-based speaker clustering in videoconferencing, and identity-based tracking in small office environments.
A performance study of an intelligent headlight control system

In this paper, we first present the architecture of an intelligent headlight control (IHC) system that we developed in our earlier work. This IHC system aims to automatically control a vehicle's beam state (high beam or low beam) during a night-time drive. A three-level decision framework built around a support vector machine (SVM) learning engine is then briefly discussed. Next, we switch our focus to the study of system performance by varying the SVM feature set, as well as by exploiting various SVM training options and adjustments through a set of experiments. We believe that what we learned from this performance study can provide readers useful guidelines on extracting effective SVM features within the IHC problem domain, as well as on training an effective SVM learning engine for more generalized applications.
A comparison of FFS+LAC with AdaBoost for training a vehicle localizer

This paper describes our recent work on identifying leading vehicles in the context of Forward Collision Warning (FCW) application. Specifically, we aim at detecting and localizing leading vehicles in videos that are captured by a forward-facing camera mounted in a moving host vehicle. To achieve that goal, we propose to seamlessly extend the AdaBoost-based object detection framework beyond Haar features, by integrating in the HOG (Histograms of Oriented Gradients) features. Our experimental results show that we can effectively optimize the training of the vehicle detector, by using a large bank of HOG plus Haar features within the AdaBoost framework. Our approach can also significantly reduce the number of features required for achieving a given accuracy, while the cost of such detector with more complex training can still remain tractable by using an FFS+LAC training scheme.
Automatic Video Content Summarization Using Geospatial Mosaics of Aerial Imagery

It is estimated that less than five percent of videos are currently analyzed to any degree. In addition to petabyte-sized multimedia archives, continuing innovations in optics, imaging sensors, camera arrays, (aerial) platforms, and storage technologies indicates that for the foreseeable future existing and new applications will continue to generate enormous volumes of video imagery. Contextual video summarizations and activity maps offers one innovative direction to tackling this Big Data problem in computer vision. The goal of this work is to develop semi-automatic exploitation algorithms and tools to increase utility, dissemination and usage potential by providing quick dynamic overview geospatial mosaics and motion maps. We present a framework to summarize (multiple) video streams from unmanned aerial vehicles (UAV) or drones which have very different characteristics compared to structured commercial and consumer videos that have been analyzed in the past. Using both metadata geospatial characteristics of the video combined with fast low-level image-based algorithms, the proposed method first generates mini-mosaics that can then be combined into geo-referenced meta-mosaics imagery. These geospatial maps enable rapid assessment of hours long videos with arbitrary spatial coverage from multiple sensors by generating quick look imagery, composed of multiple mini-mosaics, summarizing spatiotemporal dynamics such as coverage, dwell time, activity, etc. The overall summarization pipeline was tested on several DARPA Video and Image Retrieval and Analysis Tool (VIRAT) datasets. We evaluate the effectiveness of the proposed video summarization framework using metrics such as compression and hours of viewing time.