nils-herrmann/clean_scirepeval huggingface dataset:
Robotic gaze control using reinforcement learning

This work examines how adaptive control can learn to point a camera at the active speaker in a conversation by using a Reinforcement Learning approach with audio and video data. A motivating scenario for this problem is a robotic platform that interacts with people around its environment. Using Reinforcement Learning, the task is specified with an observable objective referred to as the reward signal. Specifying this task with a reward signal enables an adaptive controller to improve its performance with experience. The reward for this task is generated by a visual feedback from the conversation participants that is detected by the robot's camera system. Multiple experiments have been performed on a robot system with audiovisual data to examine the feasibility and potential of this approach. Our experimental results demonstrate that the system learns very fast to identify the active speakers. Furthermore, our approach inherently learns how to deal with egonoise that originates from the robot's motor or background noise from the environment.
Calibrating an Automatic Zoom Camera With Moving Least Squares

The application of zoom camera lenses in machine vision has gained a lot of attention lately. The main difficulty in their employment lies in the accurate estimation of their intrinsic parameters. In this paper, we propose novel approaches to determine these parameters by estimating continuous models of their variations as the focus and the zoom change. The first method is based on the moving least squares (MLS) multiple regression scheme which determines from a predefined number of samples, the complete model of the intrinsic parameters. MLS fits a polynomial function at each focus and zoom setting by using the measured neighboring points. In order to reduce the computational complexity of MLS, we propose another algorithm in which the MLS generated curves are clustered. Then, each cluster is modeled with a single polynomial function. This decreases the complexity of computations for the applications where delay is critical, e.g., telepresence, to the evaluation of simple polynomials. Compared to previous techniques, the proposed algorithms lead to a noticeable increase in the estimation accuracy of the intrinsic parameters. In addition, they are able to generate accurate models of these parameters with only a few measurement points.
A fast solution to the approximation of 3D scattered point data from stereo images using triangular meshes

Meshing the 3D scattered point data is an important task in machine vision since it allows the rendering of the 3D structures on a commodity graphics hardware. In this paper, we develop a fast and adaptive method to approximate an organized set of 3D scattered data points from stereo images by a triangular mesh and which can be applied to systems that require low delay response such as telepresence. The method starts by reducing the number of points through the employment of image adaptive sampling techniques. Then, an initial triangular mesh is constructed using the reduced set of points and later refined to minimize a predefined approximation error. Compared to some state of the art schemes, the proposed method results in a significant reduction in the processing time while preserving the quality of the mesh.
Cryptanalysis of Substitution Cipher Chaining Mode (SCC)

In this paper, we cryptanalyze the Substitution Cipher Chaining mode (SCC-128) , which uses three keys. The first key is the encryption key, which we were able to recover with about 240 cipher executions and 5 × 28 chosen plaintexts. The second key is responsible of generation two layers of masks, we recovered the first layer with 213 chosen plaintext and 221 cipher executions and our attack to recover the second layer costs only one known sector plaintext and 64 cipher executions. The third key is used to generate the encrypted sector ID, we were able to recover the encrypted sector ID of a sector with 1 known plaintext and 2 cipher executions for each sector.
Modeling the Variation of the Intrinsic Parameters of an Automatic Zoom Camera System using Moving Least-Squares

The accuracy of machine vision systems is highly depending on the correct estimates of the camera intrinsic parameters. This accuracy is needed in numerous applications like telepresence and robot navigation. In this work, a novel technique is proposed based on the moving least-squares approach, to model the variation of the camera internal parameters as a function of focus and zoom. Compared to a previous technique using a global least-squares regression scheme with bi-variate polynomial functions, the new method results in a huge reduction of the mean estimation error. In addition, validation tests show that the estimated values of the interpolated data are enhanced substantially even with a small number of measured focus and zoom settings. Consequently, fewer measurement points are needed to obtain an accurate model of the internal parameters of a zoom camera system.
Robotic Binaural and Monaural Information Fusion Using Bayesian Networks

Auditory signal processing already starts outside the head. The external sound field has to couple into the ear canals. The relative positions of the two ear canals and the sound source lead to a coupling that is strongly depend on frequency. In this context not only the two pinnae but also the whole head have an important functional role, which is best described as a spatial filtering process. This linear filtering is usually quantified in terms of so-called head- related transfer functions, HRTFs, which can also be interpreted as the directivity characteristics of the two ears. In this work, the HRTF constitutes the cornerstone of a sound localization algorithm, which uses Bayesian information fusion to increase the localization resolution in a three-dimensional reverberant environment. The algorithm was tested in simulations as well as in a household environment. Compared to existing techniques, the method is able to localize, with higher accuracy, 3D sound sources under high reverberation conditions.
Embedding of noncontractive systems in lossless realizations

The lossless embedding problem for the class of real and non-contractive scattering matrices, and hence for nonpassive systems is addressed. The approach is based on a modified lossless inverse scattering (LIS) model. A recursive Schur-type algorithm is devised, which enables the computation of the lossless embedding for arbitrary scattering matrices including the boundary case of scattering matrices with singular values equal to one.<<ETX>>
No-reference video quality evaluation for high-definition video

A no-reference video quality metric for High-Definition video is introduced. This metric evaluates a set of simple features such as blocking or blurring, and combines those features into one parameter representing visual quality. While only comparably few base feature measurements are used, additional parameters are gained by evaluating changes for these measurements over time and using additional temporal pooling methods. To take into account the different characteristics of different video sequences, the gained quality value is corrected using a low quality version of the received video. The metric is verified using data from accurate subjective tests, and special care was taken to separate data used for calibration and verification. The proposed no-reference quality metric delivers a prediction accuracy of 0.86 when compared to subjective tests, and significantly outperforms PSNR as a quality predictor.
High performance 3D sound localization for surveillance applications

One of the key features of the human auditory system, is its nearly constant omni-directional sensitivity, e.g., the system reacts to alerting signals coming from a direction away from the sight of focused visual attention. In many surveillance situations where visual attention completely fails since the robot cameras have no direct line of sight with the sound sources, the ability to estimate the direction of the sources of danger relying on sound becomes extremely important. We present in this paper a novel method for sound localization in azimuth and elevation based on a humanoid head. The method was tested in simulations as well as in a real reverberant environment. Compared to state-of-the-art localization techniques the method is able to localize with high accuracy 3D sound sources even in the presence of reflections and high distortion.
Real time humanoid sound source localization and tracking in a highly reverberant environment

An algorithm for real time humanoid sound localization and tracking using only two microphones in a highly reverberant environment is proposed. Several recently developed 3D humanoid sound localization algorithms require the environment to be anechoic. Also, the resolution of front-back ambiguity problem during sound localization requires the knowledge about the reference signals. Using HRTF (head related transfer functions) based sound localization together with extended Kalman filtering, we are able to accurately track moving sound sources in real time in a highly reverberant environment. This algorithm uses only two microphones and requires no prior knowledge of the reference signals.
Camera-Pose Estimation via Projective Newton Optimization on the Manifold

Determining the pose of a moving camera is an important task in computer vision. In this paper, we derive a projective Newton algorithm on the manifold to refine the pose estimate of a camera. The main idea is to benefit from the fact that the 3-D rigid motion is described by the special Euclidean group, which is a Riemannian manifold. The latter is equipped with a tangent space defined by the corresponding Lie algebra. This enables us to compute the optimization direction, i.e., the gradient and the Hessian, at each iteration of the projective Newton scheme on the tangent space of the manifold. Then, the motion is updated by projecting back the variables on the manifold itself. We also derive another version of the algorithm that employs homeomorphic parameterization to the special Euclidean group. We test the algorithm on several simulated and real image data sets. Compared with the standard Newton minimization scheme, we are now able to obtain the full numerical formula of the Hessian with a 60% decrease in computational complexity. Compared with Levenberg-Marquardt, the results obtained are more accurate while having a rather similar complexity.
Development of a Web-Based Application to Evaluate Target Finding Algorithms

Hyperspectral imagery leverages the use of spectral measurements to detect objects of interest in a scene that may not be discernable from their spatial patterns alone. This paper describes a project to make available to the community a set of hyperspectral airborne images on which anyone can run a target detection algorithm to find selected objects of interest in an image. The image and target spectral signatures are publicly available but the pixel location of targets within the image is withheld to allow for independent algorithm evaluation and scoring. To distribute these data, a website has been created which allows users to download the hyperspectral data and upload their target detection results which are then automatically scored. The Target Detection Blind Test website is located at http://dirs.cis.rit.edu/blindtest/.
A Method for Assessing Spectral Image Utility

The utility of an image is an attribute that describes the ability of that image to satisfy performance requirements for a particular application. This paper establishes the context for spectral image utility by first reviewing traditional approaches to assessing panchromatic image utility and then discussing differences for spectral imagery. We define spectral image utility for the subpixel target detection application as the area under the receiver operating curve summarized across a range of target detection scenario parameters. We propose a new approach to assessing the utility of any spectral image for any target type and size and detection algorithm. Using six airborne hyperspectral images, we demonstrate the sensitivity of the assessed image utility to various target detection scenario parameters and show the flexibility of this approach as a tool to answer specific user information requirements. The results of this investigation lead to a better understanding of spectral image information vis-a-vis target detection performance and provide a step toward quantifying the ability of a spectral image to satisfy information exploitation requirements.
Analytical modeling of optical polarimetric imaging systems

Polarimetric imaging systems have shown promising applications in passive remote sensing. To better understand the effects of various system attributes and help optimize the design and use of polarimetric imaging systems, an analytical model is developed to predict the system performance. The model consists of scene, sensor, and processing system components. Some pre-processing procedures such as estimation of the Stokes vector and degree of linear polarization (DoLP) are currently considered in the processing model. Validation with data collected from a division of time polarimeter shows good agreement between model predictions and measurements. It has been shown that the analytical model is able to predict the general polarization behavior and data trends with different scene geometries.
First-Principle Simulation of Spaceborne Micropulse Photon-Counting Lidar Performance on Complex Surfaces

To advance the science of lidar sensing of complex surfaces as well as in support of the upcoming Ice, Cloud, and land Elevation Satellite-2 (ICESat-2) mission, this paper establishes a framework that simulates the performance of a spaceborne micropulse photon-counting detector system on a complex surface. A first-principle 3-D Monte Carlo approach is used to investigate returning photon distributions. The photomultiplier tube (PMT) detector simulation takes into account detector dead time and multiple pixels based on the latest ICESat-2 design, as well as photon detection efficiency for probabilistic modeling. To explore system behavior, Fourier synthesis is introduced to create a synthetic surface based on parameters derived from a real data set. A radiometric model using bidirectional reflection distribution functions is also applied in the synthetic scene. Such an approach allows the study of surface elevation retrieval accuracy for landscapes which have different shapes as well as reflectivities. As a case study, returning photon detection on an example snow surface is explored. Based on the simulation results for lidar sensing on synthetic complex surfaces with an elevation range of 10 m across the scene, the spaceborne photon-counting lidar system considered here is seen to have a derived elevation bias of up to 2 cm and a error standard deviation of 10 cm. Further study on multiple-pixel PMT performance for complex surfaces demonstrates that a less rough surface will result in higher accuracy and a surface with a smaller diffuse albedo will result in smaller bias.
Hyperspectral imaging phenomenology of genetically engineered plant sentinels

The phenomenology of genetically engineered plant sentinels as measured by spectral imaging remote sensors is investigated. Plant sentinels have been developed to cease chlorophyll production and rapidly turn white in the presence of a chemical inducer such as hazardous chemicals or environmental pollutants. This work investigates the use of spectral imaging technology to detect the de-greening phenomena remotely. Results demonstrate successful detection of the de-greening phenomena even in the presence of benignly stressed plants.
Spectral band and detector optimization for atmospheric sounding interferometers

In the specification and selection of spectral bands and detectors for optical remote sensing instruments, important consideration must be given to the primary scientific applications of the data. In this study, the line-by-line atmospheric code FASCODE and a Lincoln-developed detector model are used to identify the spectral band extent and peak location for Hg/sub 1-x/Cd/sub x/Te detector responsivity that result in the highest sounding sensitivity to atmospheric temperature and water vapor. The goal of this work was to minimize the background photon level that contributes to noise by reducing the spectral band extent and to relocate the peak detector response to the spectral region of greatest interest. An optimization score for each wavelength was computed as the product of terms accounting for sensitivity to the parameter of interest (temperature or water vapor) and insensitivity to the other, normalized by the product of the detector noise and the vertical height of the atmospheric contribution function. By examining the optimization score as a function of wavelength, reduced spectral coverage was seen to be possible without sacrificing sounding sensitivity. These reductions, along with the movement of the detector peak to above the spectral band low wavenumber edge, allowed an improvement of 15 to 33% in noise level and 10 to 15% in sounding performance as compared to the original specification.<<ETX>>
Foreword to the Special Issue on Modeling and Simulation of Remote Sensing Data

The papers in this special issue focus on the deployment modeling and simulation in remote sensing applications which play important roles in the development and application of remote sensing technology. While modeling occurs in many aspects of remote sensing including in the representation of natural phenomena and in the development and use of data analysis algorithms, a particular role of models is in the creation of the signatures and signals that lead to the remotely sensed measurements from which we can extract information about the scene.
Guest Feature: Remote Sensing Research and Education at Rochester Institute of Technology [Education]

Remote sensing research and education at the Rochester Institute of Technology in Rochester, New York, USA, traces its roots from an academic degree program in Photographic Sciences established in the 1960?s. Many students were educated (and continue to be) in the science behind photography, later going on to careers working for NASA, the US defense and intelligence community, and other research organizations. In particular, the Digital Imaging and Remote Sensing (DIRS) Laboratory was formed in the early 1980?s by Prof. John Schott while part of the Photo Science program. Prof. Schott and the lab transitioned to the Chester F. Carlson Center for Imaging Science (CIS) when it was established in the mid 1980?s and remote sensing activity has grown from one professor and a few students to now encompassing ten professors, dozens of students, and over 40 research projects ongoing at any one time.
Measurement of Soil Moisture with Synthetic Aperture Radar

An experiment was performed in Northern Maine recently to evaluate the use of Synthetic Aperture Radar (SAR) data in the measurement of soil moisture content in small area plots with varying surface cover and at various subsurface depths. Ground preparation efforts involved defining six contiguous 30 m x 30 m plots with three surface cover conditions: bare tilled soil, short grass and forest. For each surface cover we irrigated one of the two plots to obtain areas of contrasting soil moisture content. The NASNJPL DC-8 and its three frequency (P, L, and C) SAR made passes over the area on two days and at three different depression angles collecting a multifrequency and multiangle data set. Near the time of the SAR overflights, soil samples were taken at 2, 5, 10, and 25 cm depths and gravimetric moisture content (mg) determined. While significant spatial variability was observed in the moisture samples, and variations in the SAR backscatter data induced by speckle due to surface roughness caused problems, a proportional relationship was observed between the backscatter coefficient and moisture content for the tilled and grass areas, especially near the surface. L band data seemed to be the most sensitive to mg in the open areas, while P band data showed sensitivity to the surface moisture in the forest area. Inconclusive results were obtained in correlating the SAR data with moisture measurements at the 10 and 25 cm depths. Also, the variation in backscatter due to speckle seemed to be reduced through a Polarimetric Whitening Filter (PWF) transformation.
Modeling the Spectral Effects of Water and Soil as Surface Contaminants in a High Resolution Optical Image Simulation

The primary interest of this research is to introduce selected environmental effects into RIT's Digital Imaging and Remote Sensing Image Generation (DIRSIG) Model. DIRSIG is capable of producing high resolution images (meter scale) using Computer Aided Design models (CAD) of buildings, vehicles, trees, etc. across the full optical spectrum (0.35-25µm). Currently, these objects are modeled in a pristine manner and there is no option to simulate them after exposure to environmental effects. Ideally, we would like to subject a given material to these environmental effects and then accurately model the modified reflected or emitted spectrum. As a first step, we have chosen to model moisture and dust on surfaces by implementing a model of the effects of a thin layer of water and soil coverage, respectively, on the spectral reflectance and emittance of different materials. Using new techniques for field instruments in a laboratory setting, we have established the relationship between the surface contaminant and its effect on the target in question. These results will be incorporated into the DIRSIG modeling tool for wider use.
Overview of Passive Optical Multispectral and Hyperspectral Image Simulation Techniques

The simulation of optical images can play key roles in the development of new instruments, the quantitative evaluation of algorithms and in the training of both image analysis software and human analysts. Methods for image simulation include surrogate data collections, operations on empirical imagery, statistical generation techniques, and full physical modeling approaches. Each method offers advantages or disadvantages in terms of time, cost, and realism. Current state of the art suggests three-dimensional radiative transfer models capture most of the significant characteristics of real imagery and find valuable use in system development and evaluation programs. Emerging computational power available from multithreading, graphical processing units, and techniques from deep learning will continue to enable even more realistic simulations in the near future.
Unsupervised urban land-cover classification using WorldView-2 data and self-organizing maps

Fully automated land-cover classification from commercial remote sensing satellite imagery has had limited success in part due to their limited spectral bands. New promise of unsupervised analysis has emerged with the recent launch of the eight-band high resolution satellite WorldView-2. In this paper, a fully unsupervised classification algorithm is proposed based on self-organizing maps and watershed segmentation. The results demonstrate that the proposed algorithm performs better in classifying homogeneous regions while achieving better accuracy than k-means.