Design Principles of Convolutional Neural Networks for Multimedia Forensics

Convolutional neural networks (CNNs) have received significant attention due to their ability to adaptively learn classification features directly from data. While CNNs have helped cause dramatic advances in fields such as object and speech recognition, multimedia forensics is fundamentally different problem compared to other deep learning applications. Little work exists to guide the design of CNN architectures for forensic tasks. Furthermore, it is still unclear which forensic tasks can be performed using CNNs. In this work, we investigate the design of CNNs for multiple multimedia forensic applications. We show that CNNs are capable of performing image manipulation detection as well as camera model identification. Through a series of experiments, we systematically examine the influence of several important CNN design choices for forensic applications, such as the use of a constrained convolutional layer or fixed high-pass filter at the beginning of the CNN, the use of nonlinearity after the first layer, the choice of activation and pooling functions, etc. We show that different CNN design choices should be made for different forensic applications and identify design choices to maximize the performance of CNNs for manipulation detection and camera model identification. Introduction Multimedia information, such as digital images, is frequently used in numerous important settings, such as evidence in legal proceedings, criminal investigations, and military and defense scenarios. Since this information can easily be edited or falsified, information forensics researchers have developed a wide variety of methods to forensically determine the authenticity and source of multimedia data [27]. Many early forensic approaches were developed by theoretically or heuristically identifying a set of traces left in an image by a particular processing operation or source device. For example, techniques have been developed to detect specific traces left by resampling and resizing [22, 14], contrast enhancement [26], median filtering [15, 13], sharpening [2], and many other operations. Similarly, forensic algorithms have been developed to identify the model of an image’s source camera using specific traces left by different elements of the camera’s internal processing pipeline [28, 3, 8]. More recent data-driven forensic approaches have leveraged techniques from steganalysis research that capture local pixel dependencies using high dimensional feature sets [9, 20]. These approaches have been used to detect image editing [23] and perform camera model identification [4, 19]. While these techniques have shown significant improvements in manipulation detection or camera model identificaiton accuracy, researchers are still left with questions such as: Are these the best set of classification features for forensic tasks? Can forensic traces and classification features be learned directly from data? Recently, researchers have shown significant interest in using techniques from deep learning to address problems in multimedia security such as image manipulation detection and steganalysis [1, 21]. Tools such as convolutional neural networks (CNNs) [17] show particular promise due to their ability to adaptively learn decision features from large sets of data. While CNNs have been successfully applied to computer vision problems such as object recognition [16, 29], lessons learned from this field do not necessarily translate to multimedia forensics. Because multimedia forensics is a fundamentally different problem, certain design principals that are successful when building CNNs to perform object recognition are suboptimal for multimedia forensic tasks. In their standard form, CNNs tend to learn features related to the image’s content, whereas, in image forensics tasks we need to suppress an image’s content and capture pixel value dependencies finduced by an editing operation (e.g., image tampering detection task [23]) or the camera’s image processing pipeline (e.g., camera model identification task using models of camera’s demosaicing algorithm [28]). If CNNs are used in their existing form, this will lead to a classifier that identifies the objects and scenes associated with the camera as opposed to learn image forensics classification features. In response to this problem, two methods have emerged namely the predetermined high-pass filter based deep learning approach used in steganography [21] and the ‘constrained convolutional’ layer adaptive approach used to perform image manipulation detection [1]. Since the use of deep learning approaches for multimedia security applications is still in its infancy, little work exists to guide the design of CNN architectures for forensic tasks. In addition, no work has explicitly examined and compared the performance of different proposed network topologies. As a result, several open questions currently exist with regard to the design and training of CNNs for multimedia forensics. For example: Which problems in multimedia forensics can CNNs be successfully applied to? Multiple approaches have recently been proposed for the design of the initial CNN layer (i.e. high-pass filter or constrained convolutional layer). Which of these yields the best performance? Do certain other design parameters such as the pooling technique or choice of activation function have significant effects on the CNN’s accuracy? Do these design choices vary depending on the forensic task being considered? What effect do different training techniques such as batch normalization and local contrast normalization have on the CNN’s classification accuracy? In order to guide future research into the application of deep learning techniques for multimedia security, it is important to address these questions. In this paper, we systematically investigate several CNN design choices, then use the results of our investigation to present IS&T International Symposium on Electronic Imaging 2017 Media Watermarking, Security, and Forensics 2017 77 https://doi.org/10.2352/ISSN.2470-1173.2017. 7.MWSF-328 © 2017, Society for Imaging Science and Technology