A Preliminary Study on Convolutional Neural Networks for Camera Model Identification

Camera model identification is paramount to verify image origin and authenticity in a blind fashion. State-of-the-art techniques leverage the analysis of features describing characteristic footprints left on images by different camera models from the image acquisition pipeline (e.g., traces left by proprietary demosaicing strategies, etc.). Motivated by the very accurate performance achieved by feature-based methods, as well as by the progress brought by deep architectures in machine learning, we explore in this paper the possibility of taking advantage of convolutional neural networks (CNNs) for camera model identification. More specifically, we investigate: (i) the capability of different network architectures to learn discriminant features directly from the observed images; (ii) the dependency between the amount of training data and the achieved accuracy; (iii) the importance of selecting a correct protocol for training, validation and testing. This study shows that promising results can be obtained on small image patches training a CNN with an affordable setup (i.e., a personal computer with one dedicated GPU) in a reasonable amount of time (i.e., approximately one hour), given that a sufficient amount of training images is available. Introduction The rapid proliferation of inexpensive image capturing devices has driven the widespread diffusion of digital pictures on the web. As sharing any type of images through websites and social media is everywhere. Verifying the veracity and authenticity of these widely distributed (and re-distributed) images is far from being an easy task [1, 2]. For this reason, the multimedia forensic community has investigated methodologies to assess the trustworthiness of digital images in the last few years [3, 4]. Among the many investigated forensic issues, great attention has been devoted to camera model identification [5, 6, 7]. Indeed, detecting the camera model used to take a picture can be crucial for criminal investigations and trials. This information can be exploited for solving copyright infringement cases, as well as indicating the authors of illicit material (e.g., child pornography). Even when deeper source identification granularity is needed (i.e., detecting the unique camera instance rather than just the make and model), camera model identification can be considered an important preliminary step to reduce the set of camera instances [7]. Moreover, being able to detect the camera model by analyzing small image patches is a possible way to expose splicing operations [8]. The rationale behind blind state-of-the-art camera model identification detectors is that each camera model performs peculiar operations on each image at acquisition time (e.g., different JPEG compression schemes, proprietary algorithms for CFA demosaicing, etc.). These operations leave on each picture characteristic “footprints” that can be exploited as an asset to reverseengineer the camera model identity. Following this idea, some methods focus on capturing characteristic footprints left during one specific step of the image acquisition pipeline. As an example, in [6] noise traces left by sensors on acquired images are exploited. Conversely, in [9] an algorithm tailored to detect lens distortion is proposed. In [10, 11, 12], traces relative to the used demosaicing strategy are investigated. Alternatively, the effect of dust traces on image sensors is studied in [13]. Given the difficulty of properly modeling complex chains of operations typical of the image acquisition pipeline, other camera model identification methods exploit features mainly capturing statistical image properties paired with machine learning classifiers, rather than focusing on specific processing operations. As an example, an technique based on local binary patterns is proposed in [14]. More recently, the authors of [15, 16, 17] exploit pixel co-occurrence statistics computed in different domains fed to a variety of supervised classification techniques. These methods guarantee very accurate results, especially on full resolution images that provide sufficient pixel statistics. A common aspect of all the aforementioned algorithms is that they rely on manually defined procedures to expose traces characterizing different camera models. This means that they rely on some model assumptions a priori made. However, recent advancements established by deep learning techniques in computer vision [18] have shown that it is possible to improve the accuracy in detection and classification tasks by taking advantage of great amount of data in order to learn characteristic features directly from the data itself. These methods are known as data-driven, as they learn directly from data rather than following an analytic model. Considering that the feature learning paradigm has recently proved fruitful for forensics applications [19, 20, 21, 22], in this paper we investigate the use of feature learning techniques in the camera model identification context, further investigating our first exploratory solutions [23, 24]. Our objective is to show that it is possible to use convolutional neural networks (CNNs) to learn discriminant features directly from the observed known images, Figure 1. Simple CNN architecture consisting of commonly used layers. A small color image patch is processed through convolutional, max pooling, inner product and ReLU layers. Finally, SoftMax is used to obtain a probability vector. rather than relying on hand-crafted descriptors. In principle, this enables to possibly capture also characteristic traces left by nonlinear and difficult to model operations during the acquisition pipeline. To conduct our study, we investigate the behavior of different CNN architectures to select a proper network for discriminant feature learning on 64×64×3 (i.e., height × width × colors) image patches, while keeping computational complexity at bay. In particular, we compare a series of CNN architectures differing in the number of convolutional, pooling, inner product and rectified linear unit (ReLU) layers. For each type of architecture, several hyper-parameters choices (e.g., kernel size, stride, number of feature maps) are examined. We focus on the importance of a proper training protocol, which is essential to make sure that the CNN learns important characteristics (e.g., properties discriminating camera models) rather than biased information (e.g., the semantic of the captured scenes). To this purpose, strongly inspired by [7], we consider different amounts of training images, depicting either the same or different scenes, and show how different training choices affect classification results. The rest of the paper is structured as follows. We first report some background on CNNs. Then, we show the algorithmic pipeline devised to perform camera model identification using CNNs. Afterwards, we report all the details about the experimental setup, from the tested CNN architectures to the used dataset splits. Then, we report the numeric results achieved through our study in order to evaluate the different tested setups. Finally, we wrap up our final considerations and conclude the paper. Background on CNNs In this section, we provide a brief overview of convolutional neural networks (CNNs) sufficient to understand the rest of the paper. For a more in depth description, please refer to one of the many available tutorial in the literature [18, 25]. Deep learning and in particular CNNs have shown very good performance in several computer vision applications such as image classification, face recognition, pedestrian detection and handwriting recognition [25]. A CNN is a complex computational model partially inspired by the human neural system that consists of a very high number of interconnected nodes, or neurons. Connections between nodes have a numeric weight parameter that can be tuned based on experience, so that the model is able to learn complex functions. The nodes of the network are organized in multiple stacked layers, each performing a simple operation on the input. The set of operations in a CNN typically comprises convolution, intensity normalization, non-linear activation and thresholding and local pooling. By minimizing a cost function at the output of the last layer, the weights of the network (e.g., the values of the filters in the convolutional layers) are tuned so that they are able to capture patterns in the input data and automatically extract distinctive features. This is different than traditional use of “handcrafted” features, in which the features used are driven by human intuition. In a CNN the features used are driven by data. Such complex models are trained using backpropagation coupled with an optimization method such as gradient descent and the use of large annotated training datasets. The first layers of the networks usually learn low-level visual concepts such as edges, simple shapes and color contrast, whereas deeper layers combine such simple information to identify complex visual patterns. Finally, the last layer consists of a set of data that are combined using a given cost function that needs to be minimized. For example, in the context of image classification, the last layer is composed of N nodes, where N is the number of classes, that define a probability distribution over the N visual category. That is, the value of a given node pi, i = 1, . ,N belonging to the last layer represents the probability of the input image to belong to the visual class ci. Depending on user choices, it is possible to select the class maximizing pi as classification result, or to use all pi values as feature vector to train an external classification tool (e.g., a support vector machine). To train a CNN model for a specific image classification task we need: 1. to define the metaparameters of the CNN, i.e., the sequence of operations to be performed, the number of layers, the number and shape of the filters in convolutional lay