The HDR Photographic Survey

High-dynamic-range (HDR) imaging is one of the remaining frontiers for significant advancement in consumer and professional color imaging systems. Systems for the capture, processing, and display of HDR images are gradually moving from the research labs and specialized applications to more mainstream usage and impact on consumer photography. The HDR Photographic Survey is a research project and public-domain database of images and scene data aimed at improving these systems by providing images and data that can be freely used by researchers around the world. This allows for more efficient testing and improvement of HDR algorithms and displays through enhanced inter-comparisons of results from various researchers and the availability of images with colorimetric and color appearance reference data from the original scenes. This paper describes the process of collecting the images and scene data, the system characterization, and the creation and use of the database. Introduction In the 1860’s and 1870’s outstanding and dedicated photographers such as Timothy O’Sullivan, William Henry Jackson, John K. Hillers, and others took part in expeditions to the American west carried out by the U.S. Geological Survey to document the natural wonders that were being discovered. [1] These photographic surveys of the American west, as well as similar surveys around the world, produced immensely popular and politically important images. For example, the works of William Henry Jackson have been credited with being largely responsible for the creation of the US National Park Service and the establishment Yellowstone National Park. As described by Keiley,[2] “W.H. Jackson, who continues to this day to serve as a collaborator on national park studies, was a member of the Hayden party. He obtained a remarkably fine series of Yellowstone photographs, samples of which Dr. Hayden placed on the desks of all Senators and Congressmen . influencing the passage of the National Park Act. ” The profound influence of imaging on political matters had begun. The magnitude of these accomplishments is only amplified when one contemplates the imaging technology of the time. O’Sullivan, Jackson, and their contemporaries used large-format (up to 20x24 inches) cameras, glass plates, and the wet collodion process that required the plates to be coated with a mixture of collodion and potassium iodide, immersed in a sensitizing solution of silver nitrate, then exposed, processed and fixed before the plates dried; all in the field. These pioneers, of course, were followed by the likes of Ansel Adams, whose images furthered the cause of preservation and documented the US National Parks in fine detail. [3] Ansel Adams can also be considered a technical innovator in imaging science and an early master of HDR imaging. He was also well-known for teaching and sharing his knowledge. [4] His techniques, described as the Zone System, involved pre-visualization of the desired print while at the original scene, careful exposure of the photographic negative as a relatively-HDR record of the scene, and then careful printing with dodging, burning, and retouching to create lowdynamic-range prints that matched the actual or imagined appearance of the original scene. So it could be said that he recorded the appearance of the scene with local adaptation and then rendered that appearance in the darkroom. This is exactly what current researchers try to automate in some way with visual models of spatial adaptation aimed at developing HDR rendering algorithms. [5] In fact, Adams’ techniques have rather directly inspired one very successful HDR rendering algorithm. [6] Figure 1. Eighteen individual exposures, each separated by one stop, used to construct the Luxo Double Checker HDR characterization image. These photographic surveys serve as important documentation of our natural world as the global climate changes and have inspired more recent photographers to capture images of many of the same scenes over a century later. These efforts included the Rephotographic Survey Project in the 1970’s and the followup Third View at the turn of the this century. [7] The changes illustrated over the first 100 years, and then the more recent 30, are remarkable. These works point out the value of carefully recording the time, place, and imaging variables for the recorded scenes and provided inspiration for much of the current project, including the name. While photographic negatives have long been capable of capturing a wide dynamic range of scene information, and HDR-capable film, called XR for extended range, was created with multiple fast and slow emulsions to photograph atomic and nuclear bomb explosions,[8] the field has become of more widespread technical and scientific interest with the development of digital techniques of image rendering and multiple-exposure image capture. The recent, and comprehensive, book on HDR Imaging[9] illustrates the arrival of such techniques into the mainstream of imaging. Of particular note was the detailed exposition of techniques for creating HDR images from multiple photographic exposures by Debevec and Malik in the late 1990’s [10] which is the basis of the methods used to create the images described herein. Careful measurement of scene data for use in tone reproduction research also has a long history that is highlighted by the classic work of Jones and Condit [11] which also describes earlier work and highlights the importance of flare in both the imaging process and photometric measurements of scene elements. 15th Color Imaging Conference Final Program and Proceedings 233 Research in HDR imaging has been very active in the development and psychophysical testing of computational algorithms to render HDR image data for display on low-dynamic-range displays or printers. Akyüz and Reinhard [5] and Kuang et al.[12] provide reviews of recent work. The development of HDR-capable displays has also grown recently with at least one such display [13] commercially available and methods being proposed for their use to enhance color imaging capabilities. [14] They are also being used in psychophysical experiments aimed at testing HDR imaging algorithms. Standard image data sets, such as the SCID images for example,[15] have been useful in imaging research to allow direct comparison of algorithm output, systems performance, and psychophysical results across research from various scientists and engineers worldwide. One aim of the HDR Photographic Survey is to provide such images in the public domain to researchers working on HDR systems and perception with a key feature being the inclusion of camera characterization data to allow conversion to accurate device-independent image data, colorimetric measurements of original scene elements, color appearance scaling of scene elements, and other scene data allowing increased utility of the images. Such a database of common images should allow improved evaluation of algorithms and displays along with a further understanding of the perception and visual modeling of HDR scenes and reproductions (e.g., improvements of models such as iCAM[16]). All of these works have inspired the current HDR Photographic Survey described in this paper with the hope that it will both document fully a variety of scenes in the early 21st century and provide scientific images and data to improve capture, processing, and display technologies for future generations of imaging systems and their users. Imaging Procedures Gone are the days where large-format, wet-collodion plates are required to make high-quality renditions of scenes. The HDR Survey images are based on the combination of multiple exposures made with a modern digital SLR. A Nikon D2x was used with a selection of lenses. Most of the images were obtained with a Nikon 17-55mm f/2.8 ED-IF AF-S DX Zoom-Nikkor lens. This is a very high-quality lens with minimal flare and an equivalent focal-length range of 25.5mm-85.5mm for 35mm format. In conditions of extreme potential flare, the brightest scene element was placed in the center of the frame to assure that most of the flare would also fall in the bright scene areas. The D2x is a professional digital SLR with a 12.4 megapixel CMOS sensor. Three reasons it was selected for this project were the availability of lenses, the high-speed performance (5 frames/sec.) and its auto-bracketing function that allows for nine exposures to be made with one stop increments in exposure time at a fixed aperture with one depression of the shutter release. This, combined with the speed, allows nine-exposure HDR sequences covering a nine-stop exposure range to be made in less than two seconds with sufficient light, a feature that is helpful for subjects that might tend to move. Figure 2. Linear (left) and local-adaptation (right) renderings of the HDR Luxo Double Checker image. Pertinent imaging information such as focal length, exposure times, number of exposures, lens used, etc. were recorded both manually and in the image meta-data for each scene. Typically, nine exposures in one-stop increments were captured for each scene. However some were captured with two sequences (18-stop range) or with three filter conditions to allow spectral (9-channel analysis). All images were saved in Nikon’s raw electronic format (NEF). They were converted to 16-bit TIFF files using the absolute minimal processing possible in Adobe Photoshop CS2 raw conversion utility. The TIFF files were then combined into a single HDR image using the Adobe Photoshop CS2 “merge to HDR” process. Resulting images were cropped for aesthetic purposes, retained in the native maximum resolution, and saved as OpenEXR files. The OpenEXR format retains the image data as 32-bit floating-point values. [17] These minimally-processed, and photometrically linear, OpenEXR files are what is available in the database. The process of white-balance is the main variable step in this procedure. Since may users would not