nils-herrmann/clean_scirepeval huggingface dataset:
What is the space of spectral sensitivity functions for digital color cameras?

Camera spectral sensitivity functions relate scene radiance with captured RGB triplets. They are important for many computer vision tasks that use color information, such as multispectral imaging, color rendering, and color constancy. In this paper, we aim to explore the space of spectral sensitivity functions for digital color cameras. After collecting a database of 28 cameras covering a variety of types, we find this space convex and two-dimensional. Based on this statistical model, we propose two methods to recover camera spectral sensitivities using regular reflective color targets (e.g., color checker) from a single image with and without knowing the illumination. We show the proposed model is more accurate and robust for estimating camera spectral sensitivities than other basis functions. We also show two applications for the recovery of camera spectral sensitivities - simulation of color rendering for cameras and computational color constancy.
Automatic and Accurate Shadow Detection Using Near-Infrared Information

We present a method to automatically detect shadows in a fast and accurate manner by taking advantage of the inherent sensitivity of digital camera sensors to the near-infrared (NIR) part of the spectrum. Dark objects, which confound many shadow detection algorithms, often have much higher reflectance in the NIR. We can thus build an accurate shadow candidate map based on image pixels that are dark both in the visible and NIR representations. We further refine the shadow map by incorporating ratios of the visible to the NIR image, based on the observation that commonly encountered light sources have very distinct spectra in the NIR band. The results are validated on a new database, which contains visible/NIR images for a large variety of real-world shadow creating illuminant conditions, as well as manually labeled shadow ground truth. Both quantitative and qualitative evaluations show that our method outperforms current state-of-the-art shadow detection algorithms in terms of accuracy and computational efficiency.
A New In-Camera Imaging Model for Color Computer Vision and Its Application

We present a study of in-camera image processing through an extensive analysis of more than 10,000 images from over 30 cameras. The goal of this work is to investigate if image values can be transformed to physically meaningful values, and if so, when and how this can be done. From our analysis, we found a major limitation of the imaging model employed in conventional radiometric calibration methods and propose a new in-camera imaging model that fits well with today's cameras. With the new model, we present associated calibration procedures that allow us to convert sRGB images back to their original CCD RAW responses in a manner that is significantly more accurate than any existing methods. Additionally, we show how this new imaging model can be used to build an image correction application that converts an sRGB input image captured with the wrong camera settings to an sRGB output image that would have been recorded under the correct settings of a specific camera.
Deep Learning for Logic Optimization Algorithms

The slowing down of Moore's law and the emergence of new technologies puts an increasing pressure on the field of EDA. There is a constant need to improve optimization algorithms. However, finding and implementing such algorithms is a difficult task, especially with the novel logic primitives and potentially unconventional requirements of emerging technologies. In this paper, we cast logic optimization as a deterministic Markov decision process (MDP). We then take advantage of recent advances in deep reinforcement learning to build a system that learns how to navigate this process. Our design has a number of desirable properties. It is autonomous because it learns automatically and does not require human intervention. It generalizes to large functions after training on small examples. Additionally, it intrinsically supports both single- and multi-output functions, without the need to handle special cases. Finally, it is generic because the same algorithm can be used to achieve different optimization objectives, e.g., size and depth.
How connections matter: factors affecting student performance in stem disciplines

The weakening academic performance of U.S. high school students and the declining influence of the U.S. economy have been studied extensively. Several studies have pointed to the declining interest of U.S. high school students in Science, Technology, Engineering, or Math (STEM) disciplines. This paper investigates the connectivity aspects that affect a high school student¿¿s ability to excel academically. Using data from the National Center for Education Statistics (NCES), this paper analyzes the connections students have with their families, schools, friends, and community. Using these connections, this paper introduces an optimal prediction model named the Kalman-based Student Connectivity Model. By asking a student specific questions, this model can predict his/her cognitive reading and math performance on standardized exams. Further, this model can be used as a predictor on how modifying lifestyle choices or environmental factors may affect a student's performance in the STEM disciplines.
Unsupervised change detection using Spatial Transformer Networks

Change detection in images or video sequences is used to detect anomalies or important variations in the scene. Although much research has focused on change detection in surveillance videos, there is little work related to change detection on aerial or satellite imagery. This paper presents a deep learning approach for change detection in satellite images. The proposed approach utilizes Spatial Transformer Networks (STNs) which learn to perform a coordinate transformation to localize potential change regions. With unsupervised training the STN system examines differences between images for significant change activity. Our initial results illustrate the merit of this method.
A spatiotemporal descriptor based on radial distances and 3D joint tracking for action classification

Action recognition is an important research area that is particularly challenging when dealing with view independent and unconstrained human motion. While progress has been made in developing pose-dependent action classification systems, the introduction of affordable 3D sensors has opened up opportunities for action classification with depth data. In this paper, we propose an efficient 3D descriptor combining radial distance measures on 2D video sequences with 3D joint tracking on depth data for action classification through Manifold Learning using supervised Locality Preserving Projections (sLPP). We find that the application of radial distances on depth data is effective at classifying actions and when combined with 3D joint tracking the action classification performance improves. We applied our method on the Microsoft Research 3D Dataset (MSR3D) and obtained good classification accuracy on all 20 unique 3D actions. Activity recognition rates were as high as 98.95% on subsets of 3D actions.
Gesture Control Using Active Difference Signatures and Sparse Learning

With the introduction of low cost depth sensors, gesture recognition systems for computer interface control are becoming a reality and require accurate recognition in real time. In this paper, we introduce a salient feature for gesture recognition called active difference signature, obtained by robust processing of depth maps and kinematic joint information. This feature is classified with variants of semi-supervised Linear extension of Graph Embedding and sparse representations to demonstrate advanced gesture control using depth cameras. Our active difference signature technique delivers highly accurate gesture recognition that is invariant to subject distance to camera, subject size, subject location within the camera field of view, and speed at which the subject performs the gestures.
Online Distance Metric Learning for Object Tracking

Tracking an object without any prior information regarding its appearance is a challenging problem. Modern tracking algorithms treat tracking as a binary classification problem between the object class and the background class. The binary classifier can be learned offline, if a specific object model is available, or online, if there is no prior information about the object's appearance. In this paper, we propose the use of online distance metric learning in combination with nearest neighbor classification for object tracking. We assume that the previous appearances of the object and the background are clustered so that a nearest neighbor classifier can be used to distinguish between the new appearance of the object and the appearance of the background. In order to support the classification, we employ a distance metric learning (DML) algorithm that learns to separate the object from the background. We utilize the first few frames to build an initial model of the object and the background and subsequently update the model at every frame during the course of tracking, so that changes in the appearance of the object and the background are incorporated into the model. Furthermore, instead of using only the previous frame as the object's model, we utilize a collection of previous appearances encoded in a template library to estimate the similarity under variations in appearance. In addition to the utilization of the online DML algorithm for learning the object/background model, we propose a novel feature representation of image patches. This representation is based on the extraction of scale invariant features over a regular grid coupled with dimensionality reduction using random projections. This type of representation is both robust, capitalizing on the reproducibility of the scale invariant features, and fast, performing the tracking on a reduced dimensional space. The proposed tracking algorithm was tested under challenging conditions and achieved state-of-the art performance.
A random projections model for object tracking under variable pose and multi-camera views

Embedded vision systems, such as smart cameras, provide a new framework for computer vision algorithms in resource constrained environments. In this paper, we present a new object tracking methodology based on random projections, which offers the benefits of fast, low-complexity transformation of the input data into accurate and computationally attractive representations. Random projections are used for the generation of a template library that describes the object's appearance and achieves robustness under pose variations. Furthermore, the random projections model is used for reliable handoff between different cameras with partially overlapping fields of view. The proposed object tracking algorithm is tailored to the limited processing capabilities of smart cameras by requiring reduced network bandwidth during camera handoff and low memory requirements for the template library maintenance. Experimental results indicate that the proposed algorithm can maintain robust tracking under varying object pose and across camera views while using limited resources, a key benefit for embedded vision systems.
Fusion of static and temporal predictors for unconstrained facial expression recognition

Facial expression research on unconstrained spontaneous expressions has benefited from recent advances in feature extraction, dimensionality reduction, and classification techniques. While the facial action coding relies on temporal predictors, state-of-the-art facial expression recognition techniques have been slow to adapt to temporal methods. Further, despite strong evidence of sparsity in the visual cortex, few approaches to facial understanding utilize sparse representations. This paper proposes using a temporal detector based upon facial dynamics, a static expression detector based on sparse representations, and an advanced temporal-sparse fused emotion estimator. Our approach leverages techniques from both computer vision and human brain research to produce a state-of-the-art emotion estimator on unconstrained faces in natural conditions.
Learning a perceptual manifold for image set classification

We present a biologically motivated manifold learning framework for image set classification inspired by Independent Component Analysis for Grassmann manifolds. A Grassmann manifold is a collection of linear subspaces, such that each subspace is mapped on a single point on the manifold. We propose constructing Grassmann subspaces using Independent Component Analysis for robustness and improved class separation. The independent components capture spatially local information similar to Gabor-like filters within each subspace resulting in better classification accuracy. We further utilize linear discriminant analysis or sparse representation classification on the Grassmann manifold to achieve robust classification performance. We demonstrate the efficacy of our approach for image set classification on face and object recognition datasets.
Multidisciplinary Undergraduate Nano-Science, Engineering and Technology Course

Using basic fundamentals, engineering and science encompass continuously evolving technologies. In response to these changes and emerging opportunities, engineering and science curricula evolve revisiting program objectives, goals and outcomes. By integrating various disciplines and tools, nanotechnology-centered engineering and science provides a multidisciplinary approach to these needed curricula changes needed to meet societal challenges and industry needs. Extensive advances in biotechnology, electronics, energy sources, information technology and nanosystems, have brought new challenges to academia. As a result, many engineering and science schools have revised their curricula to offer relevant courses. At the RIT, a cross-listed (Electrical Engineering and Physics) multidisciplinary sophomore-level Nano-Science, Engineering and Technology (NanoSET) course has been developed and offered with support from the National Science Foundation. This course is offered as a restricted science elective within the Electrical Engineering curriculum, while students from various science and engineering departments can take the course as a science or free elective. This paper reports the course goals, objectives, emphasis, coverage, accomplishments, dissemination and assessment. Strategies for interactive team-teaching, material delivery and coverage are reported. We articulate our innovative practice and strategies for teaching nanotechnology inside and outside of the classroom through lectures, workshops and laboratories. We emphasize the need for large-scale coherent efforts in defining and developing nanotechnology at the college, institutional and multi-institutional levels. To pursue the nanotechnology-centered developments and educational innovations, a number of obstacles and impediments should need to be overcome, and serious long-term commitments are needed.
Manifold Guided Label Transfer for Deep Domain Adaptation

We propose a novel domain adaptation method for deep learning that combines adaptive batch normalization to produce a common feature-space between domains and label transfer with subspace alignment on deep features. The first step of our method automatically conditions the features from the source/target domain to have similar statistical distributions by normalizing the activations in each layer of our network using adaptive batch normalization. We then examine the clustering properties of the normalized features on a manifold to determine if the target features are well suited for the second of our algorithm, label-transfer. The second step of our method performs subspace alignment and k-means clustering on the feature manifold to transfer labels from the closest source cluster to each target cluster. The proposed manifold guided label transfer methods produce state of the art results for deep adaptation on several standard digit recognition datasets.
Flowdometry: An Optical Flow and Deep Learning Based Approach to Visual Odometry

Visual odometry is a challenging task related to simultaneous localization and mapping that aims to generate a map traveled from a visual data stream. Based on one or two cameras, motion is estimated from features and pixel differences between frames. Because of the frame rate of the cameras, there are generally small, incremental changes between subsequent frames where optical flow can be assumed to be proportional to the physical distance moved by an egocentric reference, such as a camera on a vehicle. In this paper, a visual odometry system called Flowdometry is proposed based on optical flow and deep learning. Optical flow images are used as input to a convolutional neural network, which calculates a rotation and displacement for each image pixel. The displacements and rotations are applied incrementally to construct a map of where the camera has traveled. The proposed system is trained and tested on the KITTI visual odometry dataset, and accuracy is measured by the difference in distances between ground truth and predicted driving trajectories. Different convolutional neural network architecture configurations are tested for accuracy, and then results are compared to other state-of-the-art monocular odometry systems using the same dataset. The average translation error from the Flowdometry system is 10.77% and the average rotation error is 0.0623 degrees per meter. The total execution time of the system per optical flow frame is 0.633 seconds, which offers a 23.796x speedup over state-of-the-art methods using deep learning.
Bayesian network structure learning and inference in indoor vs. outdoor image classification

Bayesian network model selection techniques may be used to learn and elucidate conditional relationships between features in pattern recognition tasks. The learned Bayesian network may then be used to infer unknown node-states, which may correspond to semantic tasks. One such application of this framework is scene categorization. In this paper, we employ low-level classification based on color and texture, semantic features, such as sky and grass detection, along with indoor vs. outdoor ground truth information, to create a feature set for Bayesian network structure learning. Indoor vs. outdoor inference may then be performed on a set of features derived from a testing set where node states are unknown. Experimental results show that this technique provides classification rates of 97% correct, which is a significant improvement over previous work, where a Bayesian network was constructed based on expert opinion.
Automated event clustering and quality screening of consumer pictures for digital albuming

In this paper, algorithms for automatic albuming of consumer photographs are described. Specifically, two core algorithms namely event clustering and screening of low-quality images, are introduced and their performance is evaluated. Event clustering and image quality screening have many applications including albuming services, image management and organization, and digital photofinishing. These are difficult tasks because there is, in general, none (or very limited) contextual information about picture content, and the final interpretation could be subjective. A novel event-clustering algorithm is created to automatically segment pictures into events and subevents for albuming, based on date/time metadata information, as well as color content of the pictures. A block-based color histogram correlation technique is developed for image content comparison of general consumer pictures. A new quality-screening algorithm is developed based on object quality measures, to detect problematic images caused by underexposure, low contrast, and camera defocus or movement.
Restorations of real defocused images using blur models based on geometrical and diffraction optics

Out-of-focus blur models that have traditionally been used in image restoration are based on geometrical optics. Continuous-focus blurs based on the principles of diffraction optics are reviewed, and their discrete representation is discussed. Restorations are carried out by using both types of blur models for defocused images blurred in a CCD (charge-coupled-device) camera with pixel resolution equal to 6.8 mu . The differences between the restorations performed by using both types of PSF (point-spread-function) models are not significant. The accuracy of the representation of the PSF generated by using diffraction optics and the resulting restoration quality are linked to the pixel resolution.<<ETX>>
A novel time-frequency feature extraction algorithm based on dictionary learning

Time-frequency (TF) representations have been widely used over the past decade to characterize the non-stationary content of signals in the joint time and frequency domain. Although a number of effective TF analysis methods based on wavelet or Gabor transform have been developed, these methods use pre-determined basis functions and still require feature extraction methods to reduce redundancy and preserve important TF information related to the application of interest. This paper explores a novel TF feature extraction algorithm using a modified dictionary learning approach. The proposed algorithm is developed to modify learned dictionaries and derive TF features unique to each class. It emulates the way joint dictionary learning algorithms use common dictionaries to promote discrimination between the data from different classes, thereby allowing for an improved analysis of complex and multitask data. The proposed method indicated a significant performance in identification of the discriminant vs. common structures of the TF data.
Random Projections for face detection under resource constraints

Face detection is a key component in numerous computer vision applications. Most face detection algorithms achieve real-time performance by some form of dimensionality reduction of the input data, such as Principal Component Analysis. In this paper, we are exploring the emerging method of Random Projections (RP), a data independent linear projection method, for dimensionality reduction in the context of face detection. The benefits of using random projections include computational efficiency that can be obtained by implementing matrix multiplications with a small number of integer additions or subtractions. The computational savings are of great significance in resource constrained environments, such as wireless video sensor networks. Experimental results suggest that RP can achieve performance that is comparable to that obtained with traditional dimensionality reduction techniques for face detection using support vector machines.
Two-stage texture segmentation using complementary features

A two-stage texture segmentation approach is proposed where an initial segmentation map is obtained through unsupervised clustering of multiresolution simultaneous autoregressive (MRSAR) features and is followed by self-supervised or bootstrapped classification of wavelet features. The self-supervised stage is based on a segmentation confidence map, where the regions of "high confidence" and "low confidence" are identified on the MRSAR segmentation result using multilevel morphological erosion. The second-stage wavelet classifier is trained from the "high-confidence" samples and is used to reclassify only the "low-confidence" pixels. The final reclassification is based on rules that combine minimum distance and spatial constraints. Additionally, an improved coefficient feature normalization procedure is used during the classification process of both stages. The proposed two-stage approach leverages on the advantages of both MRSAR and wavelet features, and incorporates an adaptive neighborhood-based spatial constraint. Experimental results show that the misclassification error can be significantly reduced compared to morphological cleaning operations alone.