PURSUING AUTOMATED CLASSIFICATION OF HISTORIC PHOTOGRAPHIC PAPERS FROM RAKING LIGHT PHOTOMICROGRAPHS

Surface texture is a critical feature in the manufacture, marketing and use of photographic paper. Raking light reveals texture through a stark rendering of highlights and shadows. Though raking light photomicrographs effectively document surface features of photographic paper, the sheer number and diversity of textures used for historic papers prohibits efficient visual classification. This work provides evidence that automatic, computer-based classification of texture documented with raking light photomicrographs is feasible by demonstrating an encouraging degree of success sorting a set of 120 photomicrographs made from samples of historic silver gelatin paper. Using this dataset, four university teams applied different image processing strategies for automatic feature extraction and degree of similarity quantification. All four approaches were successful in detecting strong affinities among similarity groupings built into the dataset as well as identifying outliers. The creation and deployment of the algorithms was carried out by the teams without prior knowledge of the distributions of similarities and outliers. These results indicate that automatic classification of historic photographic paper based on texture photomicrographs is feasible and should be aggressively pursued. To encourage the development of other classification schemes, the 120 sample “training” dataset used in this work is available to other academic researchers at www.PaperTextureID. org. 1 TEXTURE IN PHOTOGRAPHIC PAPER Texture is a defining attribute of photographic paper. Starting in the early 20th century, manufacturers manipulated texture to differentiate their products and to satisfy the aesthetic and functional requirements of photographers. Especially prior to WWII, when black and white silver gelatin paper was the dominant photographic medium (Messier 2008), dozens of manufacturers worldwide produced a wide array of surfaces. From this period a book of specimen prints by the Belgian company Gevaert lists twenty five different surfaces made up of combinations of texture, reflectance, color and paper thickness (Gevaert Company of America c.1935). Around the same time, a sample book from the Defender Company of Rochester New York lists twenty seven surfaces (Defender Photo Supply Company c.1935), Mimosa twenty six (Mimosa AG c.1935) and DR AF T Johnson, Messier et al. :: Pursuing Automated Classification of Historic Photographic Papers:: 2013 3 Kodak twenty two (Eastman Kodak c.1935). Each listed surface was proprietary to the different manufacturers and each was used across their multiple brands of paper with changes, additions, and deletions occurring over a span of many years. A vital factor in the evaluation of paper surface, texture impacts the visibility of fine detail and thus provides insight into the intent of the photographer and the envisioned purpose of a particular print. For example, prints made for reproduction or documentary functions tend to be better suited to smooth-surface papers that render details with sharpness and clarity whereas more impressionistic or expressive subjects, especially those depicting large unmodulated masses of shadows or highlights, are best suited for papers with rough, broadly open textures (Eastman Kodak Company c.1935). A result of a careful and deliberate manufacturing process, texture applied to silver gelatin paper is designed to be distinct and distinguishable through processing and post-processing procedures. Likewise manufacturer-applied texture endures despite localized defects such as abrasions and deterioration caused by poor handling, storage environment and enclosures. Given these attributes, an encyclopedic collection of surface textures could reveal vital clues about a photographic print of unknown origin. Likewise a method for classifying textures could provide a means to link prints to specific photographers or to other prints of known provenance. Previous work (Parker and Messier 2009; Messier, Messier and Parker 2010) not only established the practicality of the image data collection procedure described in the next section, but also suggested that more sophisticated approaches to automated classification would yield sufficient levels of success such that the ultimate aim of developing a highly reliable texture matching methodology might be achieved. This paper heads in that direction. 2 TEXTURE IMAGE PREPARATION Sample papers dating from 1908 to 1977 were selected from a large reference collection of photographic paper. Each sample was identified by manufacturer, brand, date, and manufacturer-assigned surface designation. The reference collection and the methods used to identify the samples have been described elsewhere (Messier et al. 2004; Connors Rowe et al. 2007). The texture images were acquired with a microscope system assembled DR AF T Johnson, Messier et al. :: Pursuing Automated Classification of Historic Photographic Papers:: 2013 4 using an Infinity 2-3 imager manufactured by the Lumenera Corporation fitted with an Edmund Optics VZM 200i lens, as shown in Figure 1. The imager incorporates an Interline Sony ICX262 3.3 megapixel color progressive scan CCD sensor producing images that incorporate 1536 x 2080, 3. 45μm, square pixels. The imaged area on each sample measured 1.00 x 1.35 cm. Raking light photomicrographs were made using a fixed point illumination source using a 3 inch LED line light manufactured by Advanced Illumination placed at a 25◦ raking angle to the surface of the photographic paper. Each raking light photomicrograph generated a 16-bit TIFF. Typical samples are shown in Figure 2. The image capture technique is non-contact / non-destructive and therefore easily adapted for use on photographic prints of high intrinsic value. 3 COLLABORATIVE COMPETITION As part of a materials-based characterization project of modernist photographs at The Museum of Modern Art (MoMA), raking light photomicrographs were made from each print from the Thomas Walther Collection to document surface texture. This work stimulated interest in developing an automated scheme to cluster like prints based on surface texture. An appeal was made to university teams with signal processing experience to initiate a collaborative competition to develop methods for sorting texture images. Four university teams joined this project: • University of Wisconsin: Sethares • Worcester Polytechnic Institute: Klein, Brown, Do, and Klausmeyer • Ecole Normale Superieure de Lyon: Abry, Jaffard, Wendt, Roux, and Pustelnik • Tilburg University: van Noord, van der Maaten, and Postma Each team adopted a different approach to the development of the two standard parts of an automatic classifier: (1) feature vector extraction and (2) degree of similarity quantification. These strategies stem from a broad variety of basic approaches to texture image classification (Haralick 1979) and are described in the following section. DR AF T Johnson, Messier et al. :: Pursuing Automated Classification of Historic Photographic Papers:: 2013 5 Prototype algorithms were constructed by the four teams using a training set of 50 samples with some known texture matches. This preliminary work established that the orientation of the primary paper fiber direction relative to the raking light had no significant impact on results (this finding does not exclude a priori that photo paper textures may possess other forms of anisotropy). This work also resulted in the design of a 120 sample dataset of raking light photomicrographs of photographic papers with known metadata including manufacturer, brand, date, gloss, and texture classification, and offering varying degrees of self-similarity (the Appendix lists all samples used in this study). The dataset delivered to the teams for testing was largely composed of nine groups of ten paper samples each. Within these groups, there were three similarity subsets: (1) images made from the same sheet of paper, (2) images made from sheets taken from the same manufacturer package of paper and, (3) images from papers made to the same manufacturer specifications over a period of time. The remaining thirty samples were picked without concern for texture similarity but instead were selected to span the range of textures associated with historic silver gelatin paper. Conventional wisdom suggests that any raking light photomicrograph taken from different spots on a single sheet of paper would appear nearly identical. Likewise, texture images from different sheets of paper taken from the same manufacturer package also should show strong similarity. Furthermore, shots from papers manufactured to the same specifications but made at different times should show strong similarity, but to a somewhat lesser degree. For the thirty remaining samples, selected to demonstrate diversity, some would appear similar to the group of ninety textures and some would appear to be unique. The challenge posed to the teams was to discover these similarity groupings and isolate unique textures by producing a system of texture affinities that described the entire set. 4 TECHNICAL APPROACHES The approaches taken by the four teams can be divided into two categories (Haralick 1979; Gonzalez and Woods 2008) based on the approach to feature definition: (1) non-semantic / Wisconsin and Tilburg and (2) multiscale / Lyon and WPI. The fundamental difference is that non-semantic features are derived directly from the image data where multiscale features are based on a structural model presupposed as relevant to the encountered data. DR AF T Johnson, Messier et al. :: Pursuing Automated Classification of Historic Photographic Papers:: 2013 6 4.1 EIGENTEXTURES (WISCONSIN) In the eigentexture approach, a collection of small patches are chosen from each photographic image. These patches are gathered into a large matrix and then simplified to retain only the most relevant eigendirections using a singular value decomposition (SVD) (Moon a