Active Sampling for Computer Vision
Creators: name: Duman, Yusuf
Publisher: name: University of Surrey
Year: 2023

description: Mammalian vision systems do not view an entire scene in one go. Instead, rapid eye movements
description: known as saccades point the high density areas of photoreceptors in the retina toward areas of
description: detail. Consequently, a detailed view of the scene can be built by the brain using a relatively
description: small amount of information. By integrating the imaging in this manner the quality of the visual
description: processing found deeper within the brain is improved as it only has to process the salient details.
description: A scanning pixel camera presents a way of realising this in hardware. A low cost, low power
description: sensor system that builds up an image of a scene by rapidly sampling a sensor that sits behind a
description: moveable set of optics. Advances in micro-actuation allows the low-cost optics to be scanned
description: across the scene in a programmable manner. This can lead to the lens-less zooming effects by
description: simply varying the scan speed or the sample rate. Furthermore, the amount of information that
description: this type of sensor provides can be varied by simply changing the scan pattern.<br>
description: <br>
description: However, a major drawback of this type of sensor system is that it takes a long time to image a
description: full scene when compared to a traditional CCD camera. This motivates the work of this thesis
description: to find a scan pattern that allows the best use of the saccade-like behaviour of a scanning pixel
description: camera. By focusing on scene details relevant to a predefined computer vision task, this thesis
description: demonstrates that it is possible to produce a scan pattern that allows us to overcome this major
description: issue. In this thesis we provide methods of generating useful sample maps that enhance the
description: abilities of a scanning pixel camera and make it an efficient part of a computer vision pipeline.
description: By actively providing sample patterns to the scanning pixel camera, the sensor becomes an
description: active part of the computer vision system, rather than simply a source of data. This is similar to
description: the purpose of saccades in a mammalian vision system. In doing this we create another challenge
description: that is addressed in this thesis. Namely, the downstream computer vision task has only a partial
description: view of the scene, that may be affected by different types of artefacting found in scanning pixel
description: cameras. Therefore, how do these tasks need to be adapted to deal with data in this form, both
description: during training and inference.<br>
description: <br>
description: This thesis approaches this problem by first making several assumptions about a scanning pixel
description: camera to adapt existing computer vision techniques to find useful sample patterns. These
description: initial assumptions include that scene is static and is imaged with full knowledge of its contents.
description: These are then used to create simple model of an scanning pixel camera to establish the best
description: possible way of generating sampling positions for a downstream task. These assumptions are
description: then progressively removed in order to finally reach a method that can be deployed on a real
description: system. The end result is a technique that requires no prior knowledge of the scene to begin with,
description: forcing the scanning pixel camera to explore the scene before it knows what it is looking at.
description: The sample maps generated are designed to generate images to be used by a downstream
description: computer vision, rather than viewed by a human. To evaluate this we apply this technique to
description: a variety of computer vision tasks and demonstrate that such a piece of hardware can form a
description: useful part of a computer vision system. These tasks include object classification, tracking and
description: instance segmentation.<br>