nils-herrmann/clean_scirepeval huggingface dataset:
Colour by Correlation in a Three-Dimensional Colour Space

We improve the promising Colour by Correlation method for computational colour constancy by modifying it to work in a three dimensional colour space. The previous version of the algorithm uses only the chromaticity of the input, and thus cannot make use of the information inherent in the pixel brightness which previous work suggests is useful. We develop the algorithm for the Mondrian world (matte surfaces), the Mondrian world with fluorescent surfaces, and the Mondrian world with specularities. We test the new algorithm on synthetic data, and on a data set of 321 carefully calibrated images. We find that on the synthetic data, the new algorithm significantly out-performs all other colour constancy algorithms. In the case of image data, the results are also promising. The new algorithm does significantly better than its chromaticity counter-part, and its performance approaches that of the best algorithms. Since the research into the method is still young, we are hopeful that the performance gap between the real and synthetic case can be narrowed.
Neural Network Colour Constancy and Specularly Reflecting Surfaces

We previously developed a neural network which estimates the chromaticity of the illumination under which an given image was taken[1]. This provides colour constancy since, given the chromaticity estimate, the image pixel chromaticities can be converted via a diagonal transformation to what they would be under a canonical illuminant. In tests on synthetically generated and real scene images, the accuracy of the illumination-chromaticity estimate generally surpassed that of most existing color constancy algorithms; however, the errors obtained with real images were significantly larger than those for the synthetic ones. After experiments with adding noise to the synthetic data, we concluded that there was a more fundamental problem than simply the influence of noise which remained to be explained. We hypothesized that specular reflection was causing the problem, so we modeled the specular reflection in the training set. The errors dropped by more than 20 percent.
Camera calibration for color research

In this paper we introduce a new method for determining the relationship between signal spectra and camera RGB which is required for many applications in color. We work with the standard camera model, which assumes that the response is linear. We also provide an example of how the fitting procedure can be augmented to include fitting for a previously estimated non-linearity. The basic idea of our method is to minimize squared error subject to linear constraints, which enforce positivity and range of the result. It is also possible to constrain the smoothness, but we have found that it is better to add a regularization expression to the objective function to promote smoothness. With this method, smoothness and error can be traded against each other without being restricted by arbitrary bounds. The method is easily implemented as it is an example of a quadratic programming problem, for which there are many software solutions available. In this paper we provide the results using this method and others to calibrate a Sony DXC-930 CCD color video camera. We find that the method gives low error, while delivering sensors which are smooth and physically realizable. Thus we find the method superior to methods which ignore any of these considerations.
Practical colour constancy

The ability of a vision system to diminish, or in the ideal case, remove, the effect of the illumination, and therefore “see” the physical scene more precisely, is called colour constancy. Interest in human vision, as well as robotics and image reproduction applications, has led to much research into computational methods to achieve colour constancy. In order to serve the needs of the proposed applications, it is necessary to develop and test computational colour constancy algorithms for real image data. This practical development of computational colour constancy is the focus of this work. 
In order to study and to use computational models for real image data, it is necessary to develop a model of the physical characteristics of the vision system of interest. 
The next part of this study is the comprehensive testing of current colour constancy algorithms. The results from this study, as well as the testing paradigm that was developed, provide a foundation for the rest of the work, which specifically sets out to improve computational colour constancy on image data. 
The first area studied to improve computational colour constancy is the sensor sharpening method of Finlayson et al. In this work, I set out to test the degree to which sensor sharpening can help current colour constancy algorithms. I find that the current sharpening methods do not address the needs of this domain, and thus as part of this work, I propose a pragmatic new sharpening method. 
I then propose several improvements to variants of Forsyth's CRULE algorithm. The first is a new method of choosing a solution from the feasible set. The second reduces the reliance of the method on the diagonal model. This method enables CRULE to be extended to work with fluorescent surfaces, and is the first algorithm to deal with such input. 
A third modification to gamut mapping algorithms proposed in this work allows them to use specularities to an advantage. Specularities have long been put to use by colour constancy algorithms, but existing algorithms that use specular information are limited in that they require such information to be present. The method presented here combines the use of specular and non-specular information. 
Another algorithm chosen for close study is Finlayson et al.'s chromaticity based Colour by Correlation method. This method is attractive because, unlike the CRULE derivatives, it can take advantage of statistical information about the world. However, the comparison work done as part of this thesis indicates that the pixel brightness is also a very important source of information.
White Point Estimation for Uncalibrated Images

Color images often must be color balanced to remove unwanted color casts. We extend previous work on using a neural network for illumination, or white-point, estimation from the case of calibrated images to that of uncalibrated images of unknown origin. The results show that the chromaticity of the ambient illumination can be estimated with an average CIE Lab error of 5∆E. Comparisons are made to the grayworld and white patch methods.
Color Constancy with Fluorescent Surfaces

Fluorescent surfaces are common in the modern world, but they present problems for machine color constancy because fluorescent reflection typically violates the assumptions needed by most algorithms. The complexity of fluorescent reflection is likely one of the reasons why fluorescent surfaces have escaped the attention of computational color constancy researchers. In this paper we take some initial steps to rectify this omission. We begin by introducing a simple method for characterizing fluorescent surfaces. It is based on direct measurements, and thus has low error and avoids the need to develop a comprehensive and accurate physical model. We then modify and extend several modern color constancy algorithms to address fluorescence. The algorithms considered are CRULE and derivatives [1-4], Color by Correlation , and neural net methods [6-8]. Adding fluorescence to Color by Correlation and neural net methods is relatively straight forward, but CRULE requires modification so that its complete reliance on diagonal models can be relaxed. We present results for both synthetic and real image data for fluorescent capable versions of CRULE and Color by Correlation, and we compare the results with the standard versions of these and other algorithms.
Color Constancy with Specular and Non-Specular Surfaces

There is a growing trend in machine color constancy research to use only image chromaticity information, ignoring the magnitude of the image pixels. This is natural because the main purpose is often to estimate only the chromaticity of the illuminant. However, the magnitudes of the image pixels also carry information about the chromaticity of the illuminant. One such source of information is through image specularities. As is well known in the computational color constancy field, specularities from inhomogeneous materials (such as plastics and painted surfaces) can be used for color constancy. This assumes that the image contains specularities, that they can be identified, and that they do not saturate the camera sensors. These provisos make it important that color constancy algorithms which make use of specularities also perform well when the they are absent. A further problem with using specularities is that the key assumption, namely that the specular component is the color of the illuminant, does not hold in the case of colored metals. In this paper we investigate a number of color constancy algorithms in the context of specular and nonspecular reflection. We then propose extensions to several variants of ForsythOs CRULE algorithm [1-4] which make use of specularities if they exist, but do not rely on their presence. In addition, our approach is easily extended to include colored metals, and is the first color constancy algorithm to deal with such surfaces. Finally, our method provides an estimate of the overall brightness, which chromaticity-based methods cannot do, and other RGB based algorithms do poorly when specularities are present.
Adaptive Illuminant Estimation Using Neural Networks

In this article we present results that show that a multi-layer neural network can be trained to estimate the chromaticity of the scene illumination. The network is trained with a set of artificially generated scenes and the corresponding illuminants that were used for generating the scenes. In the test phase, the network estimates the illuminant under which a given scene was taken. Tests with artificially generated as well as natural scenes show that the neural network outperforms many current color-constancy algorithms and that it is more stable and reliable for scenes that contain a small number of colors.
Is Machine Colour Constancy Good Enough?

This paper presents a negative result: current machine colour constancy algorithms are not good enough for colour-based object recognition. This result has surprised us since we have previously used the better of these algorithms successfully to correct the colour balance of images for display. Colour balancing has been the typical application of colour constancy, rarely has it been actually put to use in a computer vision system, so our goal was to show how well the various methods would do on an obvious machine colour vision task, namely, object recognition. Although all the colour constancy methods we tested proved insufficient for the task, we consider this an important finding in itself. In addition we present results showing the correlation between colour constancy performance and object recognition performance, and as one might expect, the better the colour constancy the better the recognition rate.
Experiments in Sensor Sharpening for Color Constancy

Sensor sharpening has been proposed as a method for improving color constancy algorithms but it has not been tested in the context of real color constancy algorithms. In this paper we test sensor sharpening as a method for improving color constancy algorithms in the case of three different cameras, the human cone sensitivity estimates, and the XYZ response curves. We find that when the sensors are already relatively sharp, sensor sharpening does not offer much improvement and can have a detrimental effect. However, when the sensors are less sharp, sharpening can have a substantive positive effect. The degree of improvement is heavily dependent on the particular color constancy algorithm. Thus we conclude that using sensor sharpening for improving color constancy can offer a significant benefit, but its use needs to be evaluated with respect to both the sensors and the algorithm.
Investigations into Multi-Scale Retinex

The main thrust of this paper is to investigate the multi-scale retinex (MSR) approach to image enhancement to explain the effect of the processing from a theoretical standpoint. This leads to a new algorithm with fewer arbitrary parameters that is more flexible, maintains colour fidelity, and still preserves the contrast-enhancement benefits of the original MSR method. To accomplish this we identify the explicit and implicit processing goals of MSR. By decoupling the MSR operations from one another, we build an algorithm composed of independent steps that separates out the issues of gamma adjustment, colour balance, dynamic range compression, and colour enhancement, which are all jumbled together in the original MSR method. We then extend MSR with colour constancy and chromaticity-preserving contrast enhancement.
Luminance-Based Multi-Scale Retinex

Multi-scale retinex (MSR) processing has been shown to be an effective way to enhance image contrast, but it often has an undesirable desaturating effect on the image colours. A colourrestoration method [2,3]can help mitigate this effect, but our experience is that it simply leads to other problems. In this paper we modify MSR so that it preserves colour fidelity while still enhancing contrast. We then add neural-net based colour constancy processing[7] to this modified version of MSR. The result is a principled approach that provides the contrastenhancement benefits of MSR and improved colour fidelity.
Color Constancy for Scenes with Varying Illumination

We present an algorithm which uses information from both surface reflectance and illumination variation to solve for color constancy. Most color constancy algorithms assume that the illumination across a scene is constant, but this is very often not valid for real images. The method presented in this work identifies and removes the illumination variation, and in addition uses the variation to constrain the solution. The constraint is applied conjunctively to constraints found from surface reflectances. Thus the algorithm can provide good color constancy when there is sufficient variation in surface reflectances, or sufficient illumination variation, or a combination of both. We present the results of running the algorithm on several real scenes, and the results are very encouraging.
Color Constancy for Scenes with Varying Illumination (Color Constancy with Varying Illumination)

We present an algorithm which uses information from both surface reflectance and illumination variation to solve for color constancy. Most color constancy algorithms assume that the illumination across a scene is constant, but this is very often not valid for real images. The method presented in this work identifies and removes the illumination variation, and in addition uses the variation to constrain the solution. The constraint is applied conjunctively to constraints found from surface reflectances. Thus the algorithm can provide good color constancy when there is sufficient variation in surface reflectances, or sufficient illumination variation, or a combination of both. We present the results of running the algorithm on several real scenes, and the results are very encouraging.
Analysis and Improvement of Multi-Scale Retinex

The main thrust of this paper is to modify the multi-scale retinex (MSR) approach to image enhancement so that the processing is more justified from a theoretical standpoint. This leads to a new algorithm with fewer arbitrary parameters that is more flexible, maintains color fidelity, and still preserves the contrast-enhancement benefits of the original MSR method. To accomplish this we identify the explicit and implicit processing goals of MSR. By decoupling the MSR operations from one another, we build an algorithm composed of independent steps that separates out the issues of gamma adjustment, color balance, dynamic range compression, and color enhancement, which are all jumbled together in the original MSR method. We then extend MSR with color constancy and chromaticity-preserving contrast enhancement.
Learning Color Constancy

We decided to test a surprisingly simple hypothesis; namely, that the relationship between an image of a scene and the chromaticity of scene illumination could be learned by a neural network. The thought was that if this relationship could be extracted by a neural network, then the trained network would be able to determine a scene's Illuminant from its image, which would then allow correction of the image colors to those relative to a standard illuminance, thereby providing color constancy. Using a database of surface reflectances and illuminants, along with the spectral sensitivity functions of our camera, we generated thousands of images of randomly selected illuminants lighting ‘scenes’ of 1 to 60 randomly selected reflectances. During the learning phase the network is provided the image data along with the chromaticity of its illuminant. After training, the network outputs (very quickly) the chro-maticity of the illumination given only the image data. We obtained surprisingly good estimates of the ambient illumination lighting from the network even when applied to scenes in our lab that were completely unrelated to the training data.
Colour Constancy for Scenes with Varying Illumination

We present an algorithm which uses information from both surface reflectance and illumination variation to solve for colour constancy. Most colour constancy algorithms assume that the illumination across a scene is constant, but this is very often not valid for real images. The method presented in this work identifies and removes the illumination variation, and in addition uses the variation to constrain the solution. The constraint is applied conjunctively to constraints found from surface reflectances. Thus the algorithm can provide good colour constancy when there is sufficient variation in surface reflectances, or sufficient illumination variation, or a combination of both. We present the results of running the algorithm on several real scenes, and the results are very encouraging.
Color constancy under varying illumination

Illumination is rarely constant in intensity or color throughout a scene. Multiple light sources with different spectra-sun and sky, direct and interreflected light-are the norm. Nonetheless, almost all color constancy algorithms assume that the spectrum of the incident illumination remains constant across the scene. We assume the converse, that illumination does vary, in developing a new algorithm for color constancy. Rather than creating difficulties, varying illumination is in fact a very powerful constraint. Indeed tests of our algorithm using real images of an office scene show excellent results.<<ETX>>
Slow feature analysis yields a rich repertoire of complex cell properties.

In this study we investigate temporal slowness as a learning principle for receptive fields using slow feature analysis, a new algorithm to determine functions that extract slowly varying signals from the input data. We find a good qualitative and quantitative match between the set of learned functions trained on image sequences and the population of complex cells in the primary visual cortex (V1). The functions show many properties found also experimentally in complex cells, such as direction selectivity, non-orthogonal inhibition, end-inhibition, and side-inhibition. Our results demonstrate that a single unsupervised learning principle can account for such a rich repertoire of receptive field properties.
Temporal slowness as an unsupervised learning principle: self-organization of complex-cell receptive fields and application to pattern recognition

II Acknowledgments I am aware that this work would not have been possible without the help of many people, to which I am greatly indebted. My first thought goes to my advisor, Laurenz Wiskott, whose mathematical intuition never ceased to amaze me. In these years he taught me too many things to mention, mostly by his example, and I am extremely grateful for this. I hope I will be able to work with you again, sometime in the future. Tobias Blaschke was already working in Laurenz' group when I first arrived. He introduced me both to the secrets of the ITB and to Berlin. Thank you for all the technical discussions we had and even more for a lot of insider tips about Berlin nightlife. Special thanks to Tiziano Zito, colleague and friend, with whom I wrote a nice piece of software (MDP) and heard the guitar of Max Cavalera, live at the Columbiahalle. I shared the office also with Irina Erchova, Mathias Franzius, and Henning Sprekeler, whom I thank for all the espresso we brew with our Krups espresso-machine and for discussions and nonsenses. I would like to thank everybody else at the ITB for the nice atmosphere they created, and especially Thomas Voegtlin. Many thanks also to Prof. Andreas Herz for having provided such a nice research environment , and to Andreas Hantschmann and Christian Waltermann for the administration of the computer network, a vital part of our job. I thank Tim Gollish and Martin Stemmler for useful comments on my first manuscript, and Thomas Neukircher at the mathematical department for helping me with some insidious details of the mathematics of manifolds. This work has been supported by a grant from the Volkswagen Foundation. Thank you for that. A kiss to Eva, with whom I divided everything during our stay in Berlin. This thesis is dedicated to you. I am grateful to my mother, Daniela, and to my father, Hans, for their constant emotional and financial support during my studies and my PhD.
What is the Functional Role of New Neurons in the Adult Dentate Gyrus

The exponential growth in research results in the area of regulation of adult hippocampal neurogenesis, the life-long addition of new neurons to the hippocampal dentate gyrus, is paralleled by an increasing puzzlement about the potential function of these new cells. To determine the functional relevance of these new neurons, several fundamental problems have to be overcome. Two of them are discussed here. First, it will remain impossible to define the functional contribution the new neurons in the dentate gyrus make to hippocampal function as long as we do not know how the dentate gyrus itself contributes to hippocampal function. Our hypothesis is that adult hippocampal neurogenesis serves to avoid a stability-plasticity dilemma between learning new information and preserving old information, by allowing the dentate gyrus to adapt to new input pattern statistics while preserving the ability to process old patterns appropriately. Second, we still do not know whether, in adult neurogenesis, the structural alteration follows a specific functional stimulus and serves to consolidate a functional change triggered by that stimulus, or if less specific stimuli of novelty or complexity induce more general structural changes that prophylactically prepare the ground to better process information in similar novel or more complex situations in the future. Herein, our experimental findings and theoretical considerations argue for the latter possibility.
How Does Our Visual System Achieve Shift and Size Invariance

The question of shift and size invariance in the primate visual system is discussed. After a short review of the relevant neurobiology and psychophysics, a more detailed analysis of computational models is given. The two main types of networks considered are the dynamic routing circuit model and invariant feature networks, such as the neocognitron. Some specific open questions in context of these models are raised and possible solutions discussed.