Fingerprints of Super Resolution Networks
Authors: Jeremy Vonderfecht
Feng Liu
Venue: Accepted by TMLR ()

Abstract: Several recent studies have demonstrated that deep-learning based image generation models, such as GANs, can be uniquely identified, and possibly even reverse-engineered, by the fingerprints they leave on their output images. We extend this research to single image super-resolution (SISR) networks. Compared to previously studied models, SISR networks are a uniquely challenging class of image generation model from which to extract and analyze fingerprints, as they can often generate images that closely match the corresponding ground truth and thus likely leave little flexibility to embed signatures. We take SISR models as examples to investigate if the findings from the previous work on fingerprints of GAN-based networks are valid for general image generation models. We show that SISR networks with a high upscaling factor or trained using adversarial loss leave highly distinctive fingerprints, and that under certain conditions, some SISR network hyperparameters can be reverse-engineered from these fingerprints.

Published in Transactions on Machine Learning Research (10/2022)
Fingerprints of Super Resolution Networks
Jeremy Vonderfecht
wonder edu
Department of Computer Science Portland State University
Feng Liu
flu@@pla com
Department Computer Science Portland State University
Reviewed on OpenReview: https: // openreview. net/ mett forum? id- Jj0gSb tudb
Abstract
Several recent studies have demonstrated that deep-learning based image generation models, such as GANs, can be uniquely identified, and possibly even eeverse-nngineered, by the fingerprints they leave on their output images. We extend this research to single image super-resolution (SISR) networks. Compared to previously studied models, SISR networks are a uniquely challenging class of image generation model from which to extract and analyze fingerprints, as they can often generate images that closely match the corresponding ground truth and thus likely leave little flcxibility to embed signatures. We take SISR models as examples to investigate if the findings from the previous work on fingerprints of GAN-based networks are valid for general image generation models. We show that SISR networks with a high upscaling factor or trained using adversarial loss leave highly distinctive fingerprints, and that under certain conditions, some SISR. network hyperparameters can be reverse- engineered from these fingerprints.
1 Introduction
Recent progress in deep-learning based image synthesis has dramatically reduced the effort needed to produce realistic but fake images (Tolosana et al., 2020). But just as a criminal may leave fingerprints at a crime scene, image synthesis networks leave telltale "fingerprints" on the images they generate (Marra et al. 2019
Researchers have sought to extract increasingly detailed information about image synthesis ner from these fingerprints. A popular form of this problem deepfake detection (Dolhansky et al. (019)), which seeks to extract a single bit of information: is particular image real fake? Going further, model attribution seeks to identify the particular image generation model that produced image (Yu et al., 2019). Model parsing goes even further, seeking to infer design details of the image generation model (Asnani et al. 2021
Model fingerprints have been studied primarily to identify and track down sources of misinformation. There- fore, the types models used for deepfakes, such as generative adversarial networks (GANs), have received the most attention. The tasks performed by such models are open-ended: they permit many possible valid outputs. For example, unconditional GANs may generate any image in the training domain. We hypothesize that this open-endednesss is key requirement for distinctive model fingerprints. To test this hypothesis, we are interested in studying model fingerprinting problems, such attribution and parsing, for a less open- ended image synthesis task. Single-image super-resolution (SISR) is good choice of task for two reasons. First, SISR is a highly active area of research, with many stale-offthe-arl, models available online for free. Second, a SISR network's 's output is highly constrained by its input, and often hews closely to some optimal solution. For example, different L, L.ooptimized SISR models are known to converge on super-resolved outputs which are visually very similar (Sajjadi al.) 2017]. our hypothesis about fingerprints correct, such SISR models will be much harder to differentiate from each other.

---

Published in Transactions on Machine Learning Research (10/2022)
To understand the fingerprints of SISR networks, collect photographs from Flickr and super-resolve each of them with 205 different STSR models. These 205 models consist of 25 pretrained models published online by other researchers, and 180 models which we have trained ourselves by systematically varying four experimental hyperparameters: architecture, super-resolution scale, training datasct, and loss function. We then train an extensive collection of image classifiers to perform model altribution, and to predict the values of our experimental hyperparameters. By systematically reserving different subsets of the SISR models for testing, we investigate how our model attribution and parsing classifiers generalize. Our contributions are as follows:
We develop novel dataset of 205,000 super-resolfe resolvee images generated by 205 different SISR models, all of which will be made publicly available.
We analyze the factors that contribute to the distinctiveness of an SISR model fingerprint. We show that the choice of scaling factor and loss function significantly impacts distinctiveness, corroborating our hypothesis that more open-ended training objectives lead to more distinctive fingerprints.
As Yu et al. 2019 showed for GANs, we show that the fingerprints of SISR models trained with adversarial loss are highly sensitive to small changes in hyperparameters, such as random seed.
We study the generalization of our SISR model attribution classifier to models outside the training set. We show that our attribution classifier generalizes well from our contrived training set to real-world models, with architectures and loss functions not seen during training.
We train a set of model parsing classifiers to predict the hyperparameters of the SISR models. We show that under certain conditions, it is possible to reverse-engincer some of model's hyperparam- eters from its output images.
2
Related Work
Single image super-resolution: Recent years have seen rapid progress in deep-learning based SISR models. These days, there are profusion of such methods available. We choose SISR models as our subject study for this reason. A diverse set of of-te-of-thheart SISR models form the foundation of our experiments, We select collection of SISR models presented in recent papers based on their reproducibility and their
high-quality results (Chen et al.
2021
Dai et al.] 2019 Guo et al. 2020 Kim & Son 2021 Li et al. 2019
Liang et al.) 2021 Lim et al. 2017
Ma et al. 2020 Mei et al. 2021 (Sajadii et al.) 2017 Wang et al.
2021 2018b Zhang et al.
80.8a.
Our 180 custom- SISR models were trained using the BasicSR
framework [Wang ct al. 2018a).
Model attribution: To identify the source of synthetic images, model attribution methods can look either for watermarks (signatures deliberately encoded into each output image by the network author) (Adi et al. 2018 Hayes al. 2020: Skripniuk al. 2020: Yu et al. 2021 Zhang et al.) 2020 for unintentional statistical anomalies ies in the generated images which are unique to particular model, which we call "finger- prints". We focus on detecting these fingerprints, which can appear without deliberate intervention by the network author.
There no consensus on a precise definition of the term "model fingerprint.. Other works have defined fingerprints in terms of some specific image feature extraction technique. Marra al. (2019) formulate model fingerprints
the average noise residual across many of the model's output images. In Yu et al. (2019) and (Asnani et al.) 2021), fingerprints are feature vectors encoded by deep image classifiers trained to do model attribution /ptp//aa.......... For our purposes, we use the term "fingerprint" more generally, to refer to all statistical irregularities in an image which reveal information about the generating model.
Generative adversarial networks have been shown to possess uniquely identifying fingerprints (Marra et al.
2019
Yu et al. 2019). These fingerprints have been extracted with convolutional networks (Xuan et al.
2019
Yu al.) 2019 Jain al. 2021),, and with hand-crafted features (Goebel et al., 2020 Guarnera al.
2020
Marra et al. 2019h. Other works have shown that GAN inversion can be useful way to attribute
2

---

Published in Transactions on Machine Learning Research (10/2022)
LR Images (1,000)
SISR Models (205)
SR Images (205,000)
Classifiers
Features/Class Predictions
"Model: EDSR-L1"
EDSR-L1
Model
"Model: RCAN-L1"
"Loss: VGG+GAN"
RCAN-L1
Loss
"Loss: L1"
"Arch: EDSR"
RDN- VGG+GAN
Architecture
"Arch: RCAN"
Figure 1: An overview of our experimental setup. 1,000 low-resolution images are fed through each of 205 SISR models to produce a dataset of 205,000 super-resolved images. We then train our model parsing and attribution classifiers on this dataset of super-resolved images.
images to GANs (Albright McCloskey 2019 Zhang et al. 2021p. All of these methods focus on finding the fingerprints of GANs or Variational Auto Encoders (VAEs). However, the relevance of these results to image enhancement models, such as SISR networks, is priori unclear. We present the first study of fingerprints for STSR models, and show that STSR models also leave unique fingerprints which are identifiable by CNNs.
Reverse-engineering/nnee model parsing: We are not the first to attempt to reverse-engineer the hyperpa- rameters of black-box neural networks. (Oh et al 2019) reverse-engined many fine-grained architectural choices of small image classification networks However, their approach requires that the reverse-engineer can feed their own specially designed inputs into the black-box network and observe the resulting output. By contrast, we attempt to infer the model hyperparameters from an arbitrary output image.
In a concurrent work, Asnani et al. 2021 train convolutional network to extract fingerprint from generated image, and to predict the hyperparameters of various image generation models from this finger- print. Their method, like ours, can be used for attribution and model parsing. Their study covers diverse domain of 116 image generation models, including GANs, variational autoencoders, and two SISR networks: ESRGAN (Wang et al. 2018b) and SRFLOW (Lugmayr et al. 2020). Our work, by contrast, is focusod specifically on SISR models, which generate images very close to a ground truth and likely leave less flex- ibility to embed fingerprints. We are the first to study this hyperparameter reverse-enogincering problem specifically for SISR models.
3
Study Setup
We investigate what choices of SISR model hyperparameters lead distinctive model fingerprints, and which model hyperparameters can be reverse-engineered using model parsing. To explore these questions carefully, we need a dataset of SISR models that varies the hyperparameters holding all other hyperparameters constant. We choose 5 SISR model hyperparameters to analyze: model architecture, super-resolution scale factor, loss function, training dataset, and random seed. We train 180 SISR models with various combinations of these experimental hyperparameters, and add in 25 more SISR models from previous works for additional diversity. We construct our main dataset by super-resolving 1,000 images by each of our 205 SISR models. We use this dataset train several image classifiers for model attribution and parsing tasks. Finally, we analyze the performance of these classifiers. Figure shows overview of our process. Further details about about our setup are available in the supplementary material.
wish to experiment on, while
We want our collection of SISR models to meet the following criteria:
1. Realistic: Models representative of those in recent literature; they are not contrived toy models.
3

---

Published in Transactions on Machine Learning Research (10/2022)
2. Diverse: Our models should span many architectures, loss functions, and training sets.
3. Large: We want large number of SISR models, so that the model classifiers can begin to generalize across the SISR model space.
Uniform: The hyperparameters of our SISR models should be uniformly distributed and indepen- dent from each other prevent spurious correlations that could confound our analysis.
3.1 Single Image Super-Resolution Models
Table 1: Pretrained SISR models we use.
To make our model collection realistic and diverse, we include 25 real-world pretrained super-resolution models, published ween 2017 and 2021 (See Table 1. Unfortunately, there not enough pretrained SISR models available online to make our dataset very large. But even more importantly, distribution of hyperparameter values among these 25 SISR models is neither uniform nor inde- pendent. For example, seven out of the eight 2X scale pretrained models are Ln-oplimized. Such correlations are confounding fac- in our analysis, and we would prefer to avoid them.
Name
Loss
Scale(s)
EDSR
L1
2X,
4X
EnhanceNet RDN
Adv.
4X
L
2X, 4X
ProSR
L1
4X
ProGanSR
Adv.
4X
RCAN
L
2X, 4X
ESRGAN
Adv.
4X
SAN
L1
4X
In the most common setup, SISR models are determined by four key factors: network architecture, training dataset, super- resolution scale, and loss function. All hyperparameters of SISR model training and inference are dotermined by these factors, with the exception optimizer-based hyperparamelers such as learning rate, batch size, etc. We were interested studying the impact of each of these key factors on model fingerprints. So we built our collection of custom-trained SISR models from different choices of these four key factors, which we will refer to as experimental hyper- parameters. We add fifth experimental hyperparancter, random training seed, as kind of control group, under the assumption that changing any meaningful parameter the SISR model will have an impact least as signíficant as changing the random soed. their values as follows:
SREBN
L1
2X, 4X
SPSR
Adv.
4X
DRN
L1
4X
NCSR
Adv.
4X
SwinIR
L1, Adv.
2X, 4X
LIIF
L1
2X, 4X
Real-ESRGAN
Adv.
2X, 4X
NLSN
L1
2X, 4X
There 3 pretrained SwinIR models:
(L1, 2X), (L1, 4X), and (Adv, 4X)
Our experimental hyperparameters and
Architecture:
The choices are EDSR Lim et al. 2017). RDN (Zhang et al. 2018b RCAN
[Zhang et al. 2018a). NLSN (Mei et al. 2021 and SwinIR (Liang et al. 2021
Dataset: The super-resolution datasct used for training the model. The choices are DIV2K (Agustsson & Timofte 2017) or Flickr2K, originally collected by Lim et al. (2017). To see if using a smaller training dataset might lead to more distinctive model fingerprint, we also trained SISR models with just one quarter of the total training data available from these two datasets, effectively creating two more dataset choices, DIV2K and Flickr2K
3. Scale:
Scaling factor by which to upsample the low-resolution input image; either 2X or 4X. To be clear, this is the scaling factor for the linear dimension of the image, not the total number of pixels; a 2X-upsampled image has four times as many pixels.
Loss: Loss function optimize during training. Choices are the L, norm (which standard the super-resolution literature), VGG+adv. loss, or ResNet+ady. loss. +GG+adv. loss is the same linear combination of VGG-based perceptual loss and adversarial loss that used in SRGAN Waan et al. 2018b):
LyGG+adv perceepp + 10
lpercep/ = (SF(6(TTH)) (((S)) ((S
Lado log D(IRR)
4

---

Published in Transactions on Machine Learning Research (10/2022)
Where 1 is the high-resolution ground truth image, and I is the SISR, model's output image, & is the perceptual feature extractor, and D is the discriminator network. The adversarial loss term lodo is standard for GANs: the negative log likelihood of the discriminator D's probability that s is a natural image. The perceptual loss ppeccep/ measures the mean squared error between two feature embeddings of 1SR and THR, from a pre-trained classification network Φ. For VGG-au -adv. loss, & outputs internal activations from a pre-trained VGG net (Simonyan Zisserman 2014). ResNet-tadv. loss is the same, except uses a pretrained ResNet instead (He al. 20166)
4. Seed: Yu et al. 2019 found that changing the random seed used for network initialization is sufficient to produce a GAN with a distinct fingerprint. To determine the effect of random seeds in our setting, we train copies of our models with three different seeds: 1, 2, or 3.
In total, there are 360 possible SISR models that could be trained from different combinations of these only train the subset of these models whose random
hyperparameters. To save time and computation, seed is 1 or whose training dataset is DIV2K. (We chose to limit these two parameters because we found that they had the smallest influence on the learned model.) This leaves us with 180 custom-tras -rained SISR models.
3.2 Image Datasets
As discussed in Section 3.1 We employ two existing super-resolution image datasels, DIV2K and Flickr2K, to train our SISR models. We also create our own dataset of super-resolved images which we use to train the model attribution and parsing classifiers.
Our new image dataset consists of 1,000 photographs from Flickr. We query Flickr for 200 images from each of the following 5 image tags: food, architecture, people, animals, and landscapes. We select only images with at least two megapixel resolution. At full resolution, many images contain visible JPEG artifacts, we downsample (Gaussian blur followed by bicubic downsampling) them to 960 pixels in their largest dimension, at which point any jpeg artifacts are imperceptible to us. We refer to this collection of images as the "FlickriK dataset". Our final dataset comprises images from over 500 Flickr users and over 300 types of cameras, as measured by the metadata provided by the Flickr API.
To generate the final super-resolution dataset upon which we train our model attribution and parsing clas- sifiers, we super-resolve each image in our Flickr1K dataset by each of our 205 SISR models. For each SISR model, for each Flickr image, we first. downsample the image by the model's scaling factor using bicubic interpolation, and then super-resolve the downsampled image using the model. This gives a dataset of 205.000 super-resolved images. Figure 2 shows several super-resolution examples.
3.3
Classification Networks
We train an extensive collection of model classification networks to perform model attribution and parsing. First, needed to select an image classification network architecture. Wang et al. (2020) found that ResNet500-based classifier (He et al. 2016) was capable of distinguishing CNN-generated images from natural images with surprisingly good generalization. Rössler et al. (2019) used XceptionNet (Chollet 2017) to effectively detect deepfakes in the FaceForensics dataset. We also tried two additional stato-of-the-art image classification networks: EfficientNNtt B2 (Tan & Le 2019) and Conv (Liu al. (022)). We chose the network with the high- est accuracy the 205-way model attribution problem, i.e., "which of these 205 SISR models generated this image?" As shown in Table 2 we found that ConvNext. achieved the highest accuracy.
Table 2: Accuracy (%) on 205-class model attribution.
Architecture Acc.
ConvNext
94.3
XceptionNet
93.2
Efficientnet
92.9
ResNet50
81.8
To adapt a ConvNext network pretrained on ImageNett to our classification problems, we replace the final fully-connected layer of the network with randomly initialized one of the appropriate shape output the right number of classes. The network is trained with cross-entropy loss. As in Rössler et al. (2019) we train just the final layer for three epochs, then we unfreeze all network weights and fine-tune the entire network
5

---

Published in Transactions on Machine Learning Research (10/2022)
Low- Row-Resolu
Custom-Trrined Models (4X)
2X
4X
EDSR
RDN
RCAN
SwinIR NLSN
4
ResNet VGG
Ground Truth
Pretrained
Models
EDSR
LUF
RCAN
RDN
REGAN
Sw-L,
SREBN NLSN
Modoooo
DRN
EDSR
LIFF
ProSR
RCAN
RDN
SAN
SRFBN NLSN
0ooooo
4X
Sw-Li
E.Net
G.GAN
NCSR
proGAN REGAN
SPSR
Sw-Adv.
4X
O
Figure 2: A small image patch super-resolved by sample of the SISR models in our dataset. Abbreviations: REGAN: Real-FSRGAN, Sw-I,: L.1-optimized SwinIR, Sw-Adv: adversarially-optiizzed SwinIR, E.Net: EnhanceNet, E.GAN: ESRGAN. Best viewed zoomed in.
for 15 epochs. We use the Adamax optimizer with a learning rate of 0.0005, Batch size of 16. Our classifiers are trained with our super-resolved image dataset on just 800 images from each SISR model. We reserve an additional 100 images for validation, and 100 for testing. Images are cropped down to the network's input size. In the training set, images are randomly cropped. In validation and testing, they are center-cropped. computed from this test set of 100 images per SISR model (205,00
All analyses presented in Section images in total).
To account for random variation in our results, we train each classifier three times, starting from three different random seeds. Throughout Section 4 we report the mean and standard deviation of our accuracy scores across these three random seeds.
4 Experiments
Our experiments are organized around the analysis of two different problems: model attribution (Section 4.1 and model parsing (Section 4.2 We formulate both as classification problems, and train ConvNext models to solve them as described in Section 3.3
Excopt where otherwise stated, all accuracy scores are reported as a fmean) + (standard deviation| across three different classifiers, initialized with different random seeds but otherwise trained identically. T-SNE- based feature visualizations (Figures 3 and 5] generated from just of these three classifiers.

---

Published in Transactions on Machine Learning Research (10/2022)
Table 3: Accuracy (%) of our custom model attribution classifier grouped by different hyperparameters. We report the average and std. of classification accuracies across three versions of each classifier (trained with three different random seeds but otherwise trained identically). For example, the average accuracy of our classifier SISR models whose scale is 2X is 95.4%.
scale
accuracy
loss
accuracy
architecture
accuracy
dataset
accuracy
2X
95.4+0.7
L1
89.2+1..5
DIV2K
96.1#1.0
EDSR
94.9=11.3
4X
96.6:10.4
VGG+advv Resnet+adv.
99.2++0.2
Flickr2K
96.6-t011
RDN
96.3++0.5
99.6:t0.1
1 DIV2K
94.9-10.5
RCAN
94.7#111
7 Flickr2K
96.3+10.4
SwinlR
98.0#0.4
NLSN
96.1
4.1 Model Attribution
How reliably can an SISR model be uniquely identified by its output images? What combinations of hyper- parameters lead to distinctive fingerprints? To answer these questions, we train and analyze two attribution classifiers: the custom model attribution classifier, which is trained to distinguish between the 180 custom- trained SISR models, and the pretrained model aftribution classifier, which is trained to distinguish between the 25 pretrained models. We discuss the comparative benefits and drawbacks of these two subsets of models in Section 3.1 Essentially, the custom models are a larger and more controlled sample, while the pretrained models are more realistic and diverse. (For the rest of the paper, we use "model" as shorthand for "SISR model".)
4.1.1 When are SISR Model Fingerprints Distinctive?
L1
Adv.
Do certain hyperparameter choices make SISR model finger- prints more or less distinctive? Super-resolution seeks to invert an image downscaling function whose input domain (high res- olution images) is larger than its output range (low-resolution images). This means that each super-resolution input corre- sponds to many possible valid outputs. If the number of pos- sible valid outputs is small, it is more likely that two different SISR networks will converge upon the same output. Therefore, we hypothesize that the distinctiveness of super-resolution method increases with the size of this space of possible out- puts.
88.6%
98.7%
From this hypothesis, it easily follows that 4X upscaling should produce more distinctive fingerprints than 2X. As discussed in Sajjadi et al. (017), SISR models trained with an adversar- ial loss are incentivized to sample from this space of possible output images, while Lg-optimized networks are incentivized to aggregate over this space with a pixel-wise mean, effectively re- ducing the number of valid outputs. Sajjadi et alobserve that different Lz-optimized SISR models tend to converge upon the same unnaturally smooth super-resolved images, while SR im- ages from adversarially-traan trained models are more diverse. We can expect L loss to behave similarly, except it will aggregate over the space of possible images with a pixel-wise median in- stead of a mean. Therefore, we hypothesize that 4X SISR models leave more distinctive fingerprints than 2x, and adversarially-traiee models leave more distinctive fingerprints than L1 models.
90.2%
99.8% 99.8%
Figure 3: T-SNE visualizations of super- resolved image feature embeddings, grouped by scale and loss. Each point represents an image from the test set, colored according to the SISR model that generated it. Accuracies for each group (for this classifier) are in the lower left.
To test this hypothesis, we examine the accuracy of our 180-class custom model attribution classifier grouped by each hyperparameter, shown in Table Classification accuracy varies significantly by scale and loss
7

---

Published in Transactions on Machine Learning Research (10/2022)
"Seed triplet" (3 models)
SR Images (300)
loss x scale
Ours
PRNU
LR test images (100)
I1 2X
95.6+1.1
82.8
RDN-VGG+ady--sectf
Which model from triplef?
L1 4X
96.810.4
81.3
VGG 2X
99.7#0.1
70.5
P(model 1)
RDN-VGG+adv.seed2
Model classifier
P(model
VGG 4X
99.9+0.0
46.2
P(model 3)
RosNet 2X
99.8+0.11
45.4
RDN-VGG+adv.seed3
ResNet 4X
100.0:10.0
46.8
Total
98.6#0.2
62.2
Figure & Table 4: Distinguishing between models which differ only by seed. The table shows the accuracy (%) our eustom model attribution classifier vs the PRNU-based classifier from Marra (2017). One of the 30 seed triplets, the (4X, Lı, NLSN) triplet, had relatively low distinction accuracy of 87.3%. So the table is computed with NLSN models omitted.
function: average classification accuracy for 4X SISR models 1.2% higher than for 2X models, and average classification accuracy for adversarially-trainee models is 10.2% higher than for L1 models.
Figure 3 shows T-SNE embedding of the super-resolved image features disaggregated by scale and loss. We define image features the 1024-dimensional vector of activations from the last layer of an attribution classifier. Class separation is better for 4X SISR models than 2X, and better for adversarial than L1 models. This data supports our hypothesis that adversarial loss functions and higher super-resolution scales lead to more distinctive model fingerprints.
Distinguishing Among Random Seeds:
Yu et al. (2019) show that small variations in GAN's hyperparameters lead to highly distinctive GAN fingerprints. To test if this finding holds for SISR models, evaluate the accuracy of our custom model classifier at distinguishing between groups of models which differ only by their random seed (as shown in Figure 4 Our mustom-trained SISR model dataset contains 30 "seed triplets"; sets of three SISR models which are identical except for their seed. Our custom-trained model classifier can distinguish between models in each triplet with an average of 18.6% accuracy. We interpret this as a confirmation that Yu al's finding extends to this new domain.
For baseline comparison, we evaluate the PRNU-based model attribution scheme from Marra et al. (0019) this seed distinction problem. Marra et al.develop simple scheme for image attribution based on handcrafted features. They denoise each image using BM3D (Dabov et al. 2007). and subtract that denoised image from the original image to obtain a "noise residual". The noise residuals all training images from the same source are averaged together into a "fingerprint" " for that image source. To perform attribution, they find the noise residual of the image question, and attribute to the source with the closest fingerprint, by cuclidean distance. We find that our method achieves significantly better accuracy on this problem, which is significantly harder for the PRNU-bbsed attribution scheme than the GAN attribution problems from (Marra et al. )019). In an interesting reversal of trend for our ConvNext classifier, Li-optimized SISR models appear to leave more distinctive PRNU fingerprints than adversarially-optimizz models.
4.1.2 Pretrained Model Attribution
Our 25 pretrained models have a greater diversity of architectures, loss functions, and datasets, and may be more representative of the kinds of SISR models encountered in real, practical use. So do our attribution results still hold for them? To test this, we train an attribution classifier to predict which of the 25 pretrained models produced & given super-resolved image. We find that the performance of this classifier improves significantly if we initialize it with weights from the custom model altribution classifier, instead of starting from the ConvNext model used for ImageNet classification.
Overall test-set attribution accuracy for these 25 pretrained models 87.5+0.6%. Our hypothesis about scale and loss function still holds: accuracy among the (2X, L1) models is 77.442.2% while accuracy among
8

---

Published in Transactions on Machine Learning Research (10/2022)
Pretrained model classifier
Custom model classifier
87.1%
EDSR-24 2x
SRFBN-2x
LIIF-4x
SRFBN-4X
Real ESRGAN-4x
Li-2X
SwinIR
NLSN-4x
SwinIR-44 4x
SwinIR-GAN-45
Adv-2X
NLSN-2X 2x
Real ESRGAN-2X
RCAN-4x
proSR-4x
NCSR-4x
L:-4X
RCAN-2x
DRN-4x
RDN-4x
ESRGAN-4x
proSR-4x
Adv-4X
RDN-2x
EDSR-4x
EnhanceNet-4x
SPSR-4X
Figure 5: T-SNE Feature embeddings for images generated by our pretrained SISR models. Left: embeddings as encoded by one of the pretrained model attribution classifiers (attribution accuracy lower-left). Right: embeddings by one of the custom model altribution classifiers, which was not trained on these models.
the (4X, L1) is 87.9110%% The average classification accuracy for the (4X, adv.) group is 98. 7...4%. Figure shows a T-SNE embedding of the features of the test images classified by the pretrained model attribution classifier. The figure depicts similar trend to Figure adversarially--rained models (the square markers) are highly separable, (4X, L1) models (triangles) less so, and (2X, L1) (circles) least separable of all.
4.1.3 Fingerprinting Unseen SISR Models
So far, we have assumed that the full set set of SISR models our attribution classifiers will ever encounter is known and available during training. In real-world applications, such as scraping the web for super-resolved images which make illicit use of proprietary model, this condition is unlikely to obtain. More likely, attribution classifiers will be met with images from numerous unknown sources, and will need to handle them gracefully. So does our attribution classifier still detect meaningful fingerprints for these unseen models?
To answer this question, we ran the super-resolved images from our pretrained models through the custom model attribution classifier, which has only seen our 180 eustom-trained models during training. A T-SNE embedding of the resulting image features is displayed on the right in Figure 5 Notice that these features, taken from the custom attribution classifier, follow very similar trend to those from the pretrained attribution classifier on the left, which has seen these particular SISR, models. Class separation is not good, but the (4X, adv.) models are still modestly separable.
To make this comparison more quantitative, we introduce a metric for cluster similarity: intra- to inter-class distance ratio. We define intra-class distance as the expected distance between two random points in the same class. Inter-class distance is the expected distance from a random point class A a random point in class B. Formally, the intra- to inter-class distance ratio R(A, B) is defined
9

---

Published in Transactions on Machine Learning Research (10/2022)
Pretrained Model Classifier
Custom Model Classifier
EDSA 2x
1.6
NLSN 2x
RON 2x
SwiniB-22
0.8
ESRGAN-25
Dmn-4x
EDSI-4x
LuF-4x
0.6
RCAN-4x
SAN-4x
0.5
SwiniR-4x
E5RGAN-42
0.3
Real ESRGAN 4x
NESR 4x
SPSR 4x
EDS
LIF 2x-
NLSW
RCAN
ROW
Swinn SHFBB
Roal ESHGAN-c
DRN
RCANS4
SHPBBI-K
prodR dx
Enhan cowee 4x ESRGAN--4
ESRGAN-4e
SWIAR GAN
erooR dx
5P58
LiFT 2X-2
NLSW-2x
RCAA--X
RDW-2x
Sar miteo
Swinlf22
Rewl ESRCAN--X
DRie-xx
UIF F---d--
RCAN--a de
SAN-4
SA RIFDN
Swini
pross
ESRGIN de
Enhancohees
neal ESRGAN- Swinis GAN
arosR
Figure 6: Intra- to inter-class distance ratios for each pair pretrained STSR models, based on the features
encoded by the pretrained vs. custom model classifiers.
E 01,02EA
||a
02ll2)
+
E lbb
b2ll2
biibEB
R(A, B)
|a
6|12
aEEA
If A and B the same, R(A,B)=1. As the clusters become highly separated, R(A,B) approaches zero.
Figure f shows this distance ratio for each pair of pretrained models, using the same feature embeddings visualized in Figure 5) This figure shows that class separation is worse across the board when the models were not seen during training. The (4X, adv.) models in the lower-right corner (the last rows of the similarity matrices) are the best-separated from the other models (with the exception of Real-ESRGAN-2x and ProSR-4x). Among these models, average intra- inter- class distance ratio is around 0.5 on average. In other words, the average distance between two samples in the same class is about half the average distance between two samples from different classes.
4.2 Model Parsing
We have demonstrated that small variations in SISR model hyperparameters can lead to distinctive finger- prints. Are these variations model fingerprints random, or do they contain information about the un- derlying model hyperparametors? If they contain such information, this could be leveraged towards model parsing. To lest. this, we train an extensive collection of classifiers, referred to here as parsers, predict each of the experimental hyperparameters of our custom-traine models.
In real-world applications, such parsers would be used to reverso-cngineer the hyperparameters of unknown SISR models, which may use architectures, loss functions, elc. which are not in the parser's training data. To simulate this scenario, we pick some hyperparameter value to exclude from each parser's training data. We then evaluate the parser on models with that excluded value. We call this excluded value the test hyperparameter value. For example, if parser's test hyperparameter value RCAN, this means the parser trained the EDSR, RDN, SwinIR, and NL.NN models, and tested the RCAN ones.
10

---

Published in Transactions on Machine Learning Research (10/2022)
Table 5: Test accuracy (%) of 18 model parsers. The "chance baseline" column shows the percent chance of predicting the parameter correctly by random guess. Dashes (*-") indicate parser experiments that would not make sense; e.g. withholding all L1 models during training, and then testing the parser's ability predict the loss function of those L1 models. Rows and columns are ordered that the parsing problems become more difficult one moves down or to the right across the table.
Predicted hyperparam.
Chance baseline
Test hyperparameter value
Seed=3
Flickr2K
SwinIR
RCAN
VGG+adv.
L
scale
50.0
100.0=0.0
100.0=0.0
99.620.3
99.4
98.9
51.5.10.5
loss
33.3
98.6:10.2
98. 98.4=0.2
94.3+0.2
95.5
arch. 20.0
74.2#11.3
74.4=2.2
44.3
20.8
dataset 1
25.0
33.0+11.4
36.2+1.0
31.9+0..3
26.1
1 All models with seeds and were trained with the DIV2K dataset. Therefore, to keep the class frequencies balanced, models with seeds 2 and were excluded during both training and testing of the dataset parsers.
Table | shows the test accuracy of 18 model parsers trained and tested on the 180 custom SISR models. The results in this table can be explained as follows: some hyperparameters change the model considcrably (scale, choice of L1 or adversarial loss), and some hyperparameters change the model only subtly (dataset and random seed). Parsers perform best when the models in the training and testing sets are quite similar (e.g. they differ only by random seed), and the classes the parser must predict are quite different (e.g. the parser predicting if the model performs 2X or 4X-scale super resolution).
Performance is worst when the models in the training and testing sets are very different. Figure 2 shows the significant difference in character between images from models trained with Li loss and those trained adversarially. If we train parser the adversarially-trainee models and evaluate it on the Ln models, there is a big distributional shift between the training and testing data. This is reflected in the accuracy scores in the L1 column, which hover around the chance bascline. But if we choose to withhold the VGG+advv. models for our lest data, our parsers achieve much higher accuracy. They have been trained on models optimized with RasNet+adv. loss, a loss function very similar to the one in the test set. So the test distribution is much closer to the training distribution, and generalization is accordingly better.
These experiments demonstrate the viability of parsing SISR models, but in limited way. Only some hyper- parameters can be reliably parsed, and only when the models are not too far from the training distribution.
4.2.1 Pretrained Model Parsing
To study model parsing for pretrained models, we train parsers on all 180 custom- testom-trained SISR models to prodict the models' scale, loss, and architecture. (We omit the datasct parser because DIV2K is the only dataset shared by both the custom and pretrained models) We then apply these parsers to images from the 25 pretrained models.
Figure 7 presents a full table of the model parser predictions for each pretrained SISR model. As with the custom model parsers, we find it easy to parse the scale of models with loss functions in the training set. The scale parser we used here was trained on Ia, VGG+adv. and RedNet losses. This parser can predict the scale of our Li-optimized pretrained models with 100% accuracy. EnhanceNet (E.Net) and ProSR are also easy to parse, and are trained with losses very similar VGG+adv. loss: both leverage a pretrained VGG network for perceptual loss, in combination with adversarial loss. But shown in Table 5 with test hyperparameter value L,, prediction accuracy for unscen loss functions can be much worse. NCSR and SPSR use very different approaches lo super-resolution than the models in the parser's training set, and generalization to these models is accordingly worse. However, ESRGAN also uses a form of VGG loss plus adversarial loss, but our scale parser still performs poorly on it. We consider the exact reason why to be an open question.
11

---

Published in Transactions on Machine Learning Research (10/2022)
Predicted hyperparameter value
Our loss classifier easily dis- tinguish between L1-optimized and adversarially--rainee models: all
VGG+AR.Net-A
RDN
SwintR
EDSR-2X
LIF-2x
100
100
0
D
Ln-optimized pretrained models are identified as such with 100% accu-
RCAN-24
100
100
2=2
63m4
3024
2=1
1=1
RDN-2x
100
100
4=3
54:1
38#3
2=1
racy. The true loss functions of the adversarial SISR models (ES- RGAN, EnhanceNet, ProSRGAN, NCSR, SPSR, SwinlR, and Real ESRGAN) are not in the loss classifier's training set. Yet the loss classifier always predicts that
R ESRGAN-2X
97
2
0
81+10
18m:10
SRFBN-2x
100
SwiniR 2x
100
100
0
2=2
68
21
6*2
2
NLSN-2x
100
100
1=1
68
2421
3=1
3=1
EDSR-Ix
100
100
0
0
11:3
69:11
4=2
1427
100
100
Actual model
proSR-4x
100
100
these mothods were produced with
RCAN 4x
100
100
75
23
1
RDN-4x
100
36:10
an adversarial loss function (either RadNet+adv. VGG+adv.).
SAN 4x-
99
100
SRFBN-4x
100
100
Architecture prediction is unsuc- cessful. Our set of pretrained mod- els contains eleven models whose ar- chitecture was in the training dis- tribution. Among these models, the architecture classifier can pre- dict the architecture correctly just 22.8% of the time, similar to the random chance accuracy of 20.0%. We Wespeculate that this poor general- ization is due to differences train- ing, such as the learning rale and number of epochs, between custom and pretrained models.
SwiniR
NLSN-Ix
100
100
0
68:4
20
0
10
ESRGAN
49s7
11:4
E.Net-Ix
0
99
98
NCSR-4x
81:9
18:9
71
28:29
pro5R-4x
R ESRGAN 4x
91:3
9:3
49
5026
SP5R-4x
7?=7
22=7
2=2
58=8
3926
SwiniR adv
7828
21:8
55=7
42s11 Architecture
Scale
Loss
Figure 7: Parser predictions for the pretrained models. For example, out of the 100 test images for ESRGAN, 5117 were predicted to come from a model with 2X scale, 49=7 from 4X (values are rounded to integers). Blue boxes inscribe correct predictions. Rows with no blue box have actual values outside the training set. Abbreviations: R. ESRGAN: Real ESRGAN, E.Net: EnhanceNet.
4.3 Key Findings
As Yu et al. (2019) showed for GANs, any small change to an SISR model's hyperparameters is sufficient to detect a unique fingerprint: see Section [.1.I Our model attribution experiments from Section 4.1 consistently show that model fingerprints are much more distinctive for models trained with adversarial loss than L1 loss, and are slighty more distinctive for 4X models than 2X ones. We posit that this due the "open-endedness" of the image synthesis task: the larger the space of valid network outputs, the more opportunity there is for the network to leave distinctive fingerprints. In Section 4.2 our model parsing experiments met with mixed results: our parsers can be either very accurate or very inaccurate, depending on the particular parsing task.
5 Conclusion
We have presented the first exploration of model fingerprinting and image attribution specifically focused single image super-resolution networks. We create datasct of 205 super-resolution methods. We show that networks trained on more open-ended image synthesis lasks leave more distinctive fingerprints on the images they generate. Our attribution classifiers can learn to detect distinctive fingerprints even for SISR models outside the training set. Our model parsing experiments show that model parsing possible, but only when the predicted hyperparameter has large impact on the resulting images, and the model being parsed is similar to those in the training set.
12

---

Published in Transactions on Machine Learning Research (10/2022)
References
Yossi Adi, Carsten Baum, Moustapha Cisse, Benny Pinkas, and Joseph Keshet. Turning your weakness into a strength: Watermarking deep neural networks by backdooring, In 27th USENIX Security Symposium (USENIX Securily 18), 2018.
Eirikur Agustsson and Radu Timofte. Ntire 2017 challenge on single image super-resolution: Dataset and study. In TEFE Conference Computer Vision and Paltern Recognition. Workshops, July 2017.
Michael Albright and S. McCloskey. Source gemerator attribution via inversion. IEEE Conference on Com- puter Vision and Pattern Recognition Workshops, 2019.
Vishal Asnani, Xi Yin, Tal Hassner, and Xiaoming Liu. Reverse engincering of generative models: Inferring model hyperparameters from generated images. urXiv/2106.0788 07873, 2021.
Yinbo Chen, Sifei Liu, and Xiaolong Wang. Learning continuous image representation with local implicit image function. In IEEE conference on computer vision and pottern recognition, pp. 8628 8628-8638, 2021.
François Chollet. Xception: Deep learning with depthwise separable convolutions. In IEEE conference computer vision and pattern recogmition, pp. 1251-1258, 2017.
Kostadin Dabov, Alessandro Foi, Vladimir Katkovník, and Karen Egiazarian. Image denoising by sparse 3-d transform-domain collaborative filtering. IEFE Transactions on Image Processing, 16(8):2080-2)) 2095, 2007. doi:
Tao Dai, Jianrui Cai, Yongbing Zhang, Shu-Tao Xia, and Lei Zhang. Second-order attention network for single image super-resolution. In CVF Conference on Computer Vision and Pallern Recogratton, June 2019.
Brian Dolhansky, Russ Howes, Ben Pflaum, Nicole Baram, and Cristian Canton Ferrer. The deepfake detection challenge (dfdc) preview dataset. arXiv/1910. 08.554 2019.
Michael Goebel, L. Nataraj, Tejaswi Nanjundaswamy, T. Mohammed, S. Chandrasekaran, and B. S. Man- junath. Detection, attribution and localization of gan generated images. arXit/2007.10466, 2020.
Luca Guarnera, Oliver Giudice, and Sebastiano Battiato. Fighting deepfake by exposing the convolutional traces on images. IEEE Access, 2020.
Yong Guo, Jian Chen, Jingdong Wang, Qi Chen, Jiczhang Cao, Zeshuai Deng, Yanwu Xu, and Mingkui Tan. Closed-loop matters: Dual regression networks for single image super-resolution. In TEFE Conference Computer Vision and Pattern Recognation, 2020.
J. Hayes, Krishnamurthy Dvijotham, Yutian Chen, S. Dieleman, P. Kohli, and Norman Casagrande. Towards transfornatioon-rssllient provenance detection of digital media. arXit/2011.07355 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE conference on computer vision and paltern recognition, pp. 770 778, 2016.
Anubhav Jain, Pavel Korshunov, and Sébastien Marcel. Improving generalization of deepfake detection by training for attribution. In International Workshop on Multimedia Signal Processing (MMSP), 2021.
Younggeun Kim and Donghee Son. Noise conditional flow model for learning the super-resolution space. In IEEE Conference on Computer Vision and Pattern Recognétion, 2021.
Zhen Li, Jinglei Yang, Zheng Liu, Xiaomin Yang, Gwanggil Jeon, and Wei Wu. Feedback network for image super-resolution. In IEEE Conference on Computer Vision and Pattern Recognition, 2019.
Jingyun Liang, Jiezhang Cao, Guolei Sun, Kai Zhang, Luc Van Gool, and Radu Timofte. Swinir: Image restoration using swin transformer. In IEEE International Conference on Computer Vision Workshops, 2021.
13

---

Published in Transactions on Machine Learning Research (10/2022)
Bee Lim, Sanghyun Son, Heewon Kim, Seungjun Nah, and Kyoung Mu Lee. Enhanced deep residual networks for single image super-resoution. In TEEE Conference on Compuler Vision and Pallern Recognation Workshops, July 2017.
Zhuang Liu, Hanzi Mao, Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. JEEE/CVE Conference on Computer Vision and Pattern Recognition, 2022.
Andreas Lugmayr, Martin Danelljan, Lue Van Gool, and Radu Timofte. Srflow: resolution space with normalizing flow. In European Conference on Computer Vision, 2020.
Learning the super-
Cheng Ma, Yongming Rao, Yean Cheng, Ce Chen, Jiwen Lu, and Jie Zhou. Structure-ppe preetervr-rreeriigg super resolution with gradient guidance. In IEEE Conference on Computer Vision and Pattern Recognition, June 2020.
Francesco Marra. Source identification in image forensics. PhD thesis, DIETI, University Federico II of Naples, 01 2017.
Francesco Marra, Diego Gragnaniello, Luisa Verdoliva, and Giovanni Poggi. Do gans leave artificial finger- prints? In IEEE Conference on Multimedia Information Processing and Retrieval, 2019.
Yiqun Mei, Yuchen Fan, and Yuqian Zhou. Image super-resolution with non-local sparse attention. In IEEE conference computer vision and pattern recognition, pp. 3517-3526, June 2021.
Seong Joon Oh, Bernt Schiele, and Mario Fritz. Towards reverse-engineering black-bor neural networks. Erplainable AI: Interpreting, Explaining and Visualizing Deep Learning. Springer, 2019.
Andreas Rössler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nießner. FaceForensicc Learning to detect manipulated facial images. In IEEF International Conference on Computer Vision, 2019.
Mchdi S. M. Sajjadi, Bernhard Schölkopf, and Michael Hirsch. Enhancenet: Single image super-resolution through automated texture synthesis. In IEEE International Conference on Computer Vision, 2017.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint orXii:1409. 1556, 2014.
Vladislav Skripniuk, Ning Yu, Sahar Abdelnabi, and Mario Fritz. Black-bor watermarking for generative adversarial networks. SrXin/2007.08.33 2020.
Mingxing Tan and Quoe Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning, pp. 6105-6114. PMLR, 2019.
Ruben Tolosana, Ruben Vera-Rodriguez, Julian Ficrrez, Aythami Morales, and Javier Ortega-Garcia. Deep- lakes and beyond: A survey of face manipulation and fake detection. Information Fusion, 2020.
Sheng- Yu Wang, Oliver Wang, Richard Zhang, Andrew Owens, and Alexei A Efros. Cnn Enn-generated images are surprisingly easy to spot...for now. In IEEE conference computer vision and pottern recognition, 2020.
Xintao Wang, Ke Yu, Kelvin C.K. Chan, Chao Dong, and Chen Change Loy. BasicSR: Open source image and video restoration toolbox. https://github. com/ wwwixxinnnttnaaaaa BBasicSR Ccinntao/BasicSR 2018a.
Xintao Wang, Ke Yu, Shixiang Wu, Jinjin Gu, Yihao Liu, Chao Dong, Yu Qiao, and Chen Change Loy. Esrgan: Enhanced super-resolution generative adversarial networks. Computer Vision Workshops, September 2018b.
Tn The European Conference.
Xintao Wang, Liangbin Xic, Chao Dong, and Ying Shan. Real-osrgan: Training real-world blind super- resolution with pure synthetic data. In International Conference on Computer Vision Workshops (IC- CVW), 2021.
14

---

Published in Transactions on Machine Learning Research (10/2022)
Yifan Wang, Federico Perazzi, Brian McWilliams, Alexander Darkine-Hormungg O. Borkine-Hormung and Christopher Schroers. A fully progressive approach to single-image super-resolution. IEEE Conference Computer Vision and Pattern Recognition Workshops, 2018c
Xinsheng Xuan, B. Peng, Wei Wang, and Jing Dong- Scalable fine-grained generated image classification based on deep metric learning. ArXio/1912.11006 2019.
Ning Yu, Larry S Davis, and Mario Fritz. Attributing fake images to gans: Learning and analyzing gan fingerprints. In JEEE International Conference on Compuler Vision, 2019.
Ning Yu, Vladislav Skripniuk, Sahar Abdelnabi, and Mario Fritz. Artificial fingerprinting for generative models: Rooting deepfake attribution training data. In Proceedings of the IEEE/CVP International Conference on Computer Vision, pp. 14448 14457, 2021.
Baiwu Zhang, Jin Peng Zhou, Ilia Shumailov, and Nicolas Papernot. On attribution of deepfakes. Ar WrXin/2007.84230 2021.
J. Zhang, Dongdong Chen, Jing Liao, Han Fang, Weiming Zhang, Wenbo Zhou, Hao Cui, and Nenghai Yu. Model watermarking for image processing networks. In AAAI Conference on Artificial Intelligence, 2020.
Yulun Zhang, Kumpeng Li, Kai Li, Lichen Wang, Bineng Zhong, and Yun Fu, Image super-resolution using very deep residual channel attention networks. In European Conference on Computer Vision, 2018a.
Yulun Zhang, Yapeng Tian, Yu Kong, Bineng Zhong, and Yun Fu. Residual dense network for image super-resolution. In IEEE conference on computer vision and pottern recognition, 2018b.
A
Appendix
A.1 Flickr1K Dataset
Table 6: Main characteristics of the three SR datsets used in this project: Flickr2K (Lm et al. 2017) Div2K (Agustsson & Timofte 2017), and our 'FlickriK' dataset. Table shows the mumber of images, pixels per image (ppi), bits per pixel using PNG compression (bpp PNG), and shannon-entropy of the images' greyscale histograms (entropy). For bpp and entropy, we report average (1standard deviation).
Dataset
Images
Ppi
Bpp PNG
Entropy
Flickr2K train
800
2793045
12.71(+2.53)
7.34
Flickr2K test
100
2794881
12.52(=2.45)
7.38 (#0.53)
Flickr2K validation
100
2749737
12.96(+2.67)
7.52(++0.29)
DIV2K train
800
2779971
12.68(+2.79)
7.48(=+0.3))
DIV2K validation
100
2733370
13.24
(+2287)
7.51
Flickr 1K train
100
649958
12.93
(+2.75)
7.30(+0.45)
Flickr1K test
798
646188
12.89)
7.32(+0.52)
Flickr validation
100
633369
12.39
(+2.39)
7.24 (+24(+)))4)
A.2 SISR Networks Dataset
Our 180 customm-rrained models were trained using the BasicSR Loolkit (Wang et al. 2018a). Besides the experimental hyperparameters, all training hyperparameters were held constant across the models. For example, all models were trained for 50,000 iterations. This Icads to substandard performance of the SwinIR models, which would need to be trained for times longer to match the quality of the published models. This poor quality is reflected qualitatively in Figure 2 and quantitatively in Table
15

---

Published in Transactions on Machine Learning Research (10/2022)
A.2.1 ResNet Loss
One third of our custom-trained super-resolution networks are trained with ResNet + adversarial loss, a combination of discriminalor-basedd adversarial loss, and perceptual loss based on a ResNet18 classifier trained on ImageNet.
Our PeaNet-based perceptual loss is computed by passing both the super-resolved image and the ground- truth high-resolution image through the ResNet. We extract the activations output by layer 3 of the ResNet. and take the L2 distance between HR and SR feature vectors as our ResNet loss term. We use this loss term as drop-in replacement for the VGG-based perceptual loss term used in SRGAN Wang et al. (0018)) and implemented in BasicSR Wang et. al. (2018a).
Table 7: Papers which provide the 25 pretrained super-resolution models use in our dataset (some рapers provide multiple models).
Name
Loss
Scale(s)
EDSR Lim et al. (2017)
L
DIV2K
2x, 4x
Enhance Vel. Sajjadi el al.
(2017)
Adv.
MSCOCO
4x
RDN Zhang
2018b
Li
DIV2K
2x, 4x
ProSR Wang al.
2018c
L1
DIV2K
4x
ProGanSR Wang al.
2018c
Adv.
DIV2K
4x
RCAN Zhang al. 200
8a
T
DIV2K
2x, 4x
ESRGAN Wang et a..
(2018b)
Adv.
DIV2K, Flickr2K, OST
4x
SAN Dai et al. (2019)
L1
DIV2K
4x
SRFBN T.i et al. (2019
I
DIV2K
2x, 4x
SPSR Ma et al. 2020
Adv.
DIV2K
4x
DRN Guo al. 2020
L1
DIV2K
4x
NCSR Kim & Son (2021
Adv.
DIV2K
4x
SwinlR Liang et. a.
[2021
L1, Adv.
DIV2K
2x, 4x
LUIF
Chen et al.
2021
L1
DIV2K
2x,
4x
Real ESRGAN
Vang et. al.
2021
Adv.
DIV2K
2x, 4x
NLSN
Mei et a..
2021
L1
DIV2K
2x, 4x
Table 8: Full list of all 205 super-resolttion models in dataset, along with their average PSNR and LPIPS scores when compared to the ground truth HR images. Model names are of the form (architecture) (traiuurreet(ttinnnggggee (training
dataset>- (sscale--(ooss (-loss---ossssss (-seed)
SISR model
PSNR (+ std)
LPIPS (= std)
DRN-dtr22oss-JL--AApetrri
28.31(#6.26)
0.284(+0.151)
EDSRRchi22e-22111111PPmrrrrrni
32.56(+7.03)
0.134(+0.115)
EDSR-div24=x2-LI-sl
32.27(16.90)
0.132(10.115)
EDST-diiv26x22111-222
32.54(+7.06)
0.133
EDSR-div24=22-1--33
32.49(+7.03)
0.134(10.115)
EDSR-dir22::22R<eeetttttt
31.33(+6.60)
0.074(+0.102)
EDST:-li22--2--eeeee-------
32.25(+7.00)
0.086(+00.03)
31.90(+6.90)
0.078(+0.102)
EDSR=-div22x22VVCCC-Ak---1
31.91(+6.78)
0.080(+00103)
EDSR:dinvle:22VVGGG-lt....
31.91(#6.79)
0.080(+0.102)
ADST-div22.22-VVCG--ll--..
32.01(+6.91)
0.082(+0.102)
28.08(+6.26)
0.293(10..122
EDSRRdiiv22+x44-11111
27.95(+6.222)
0.296(
EDSRRdiiv2<x44L11122
27.95(+6.22)
0.299
EDSRRdiiv2Kxx4111-33
(7.92(+6.22)
(+0.151))
EDSRR-livok-x-leeaeett----bbb
26.96(15.58) (5.58)
26.96(
0.188(10.112)
EDSR-dli22exx-ReeNNtt--A
(6.51((5..2)
0.192
16

---

Published in Transactions on Machine Learning Research (10/2022)
EDSI-dii22exA-ReeVottt-dh..
25.08(#5.49)
0.205(+0.115)
FDSttvli22exx-VCC--AAvvv..
26.57
0.188(+0.109)
26.66(+5.43)
0.185(10.107)
26.54(#5.69)
0.182(+0.105)
EDSTe-lickr22-:2--111111
32.46(+7.04)
0.133(+0.115)
31.98(+6.86)
0.086(+0.103)
EDSRRdlieer2--22-VCG-AArraa
31.38(+6.60)
0.083(10.103)
EDSRRffleckr22:x44-1L11
27.90(+6:21)
0.307(#0.152)
26.51
0.195
(5..6)))
0.207(110.14)
32.46(++6.99
0.126(+10.13)
32.00(=6..5)
0.075(+0.102)
EDSR-quartor -
32.22(+6.99)
EDSR-quarter div2k-x4-L1-s]
27.94
0.301
EDSR-quarter dive::xd-RResNet--Adv...
26.39(#5.83
0.185(#0.106)
(6.45
0.184(10.107)
EDSR-quarter flick22kx22-L-ss1
32.45
0.135
EDSR-quarter Nicerz)l-x22ReeNett+
31.20(+6.31)
0.093(++.104
EDSR-quarter ficckkk-x2-VGCC-Add....
31.86(+6.79)
0.089(+0.102)
EDSR-quarter -lickr2k-44-L-ssl
27.89(+6.18)
0.305(10.152)
EDSR-quarter
26.82(#6.00)
0.187(#0.105)
27.11
0.199(+0.112)
27.48(+6..9)
0.161
26.48 (55.83)
0.206(#0.113)
LIl--div2k--22---NAAppetrrenddd
32.35(+6.77)
0.130(10.115)
28.34(+6.34)
0.283(+0.151)
NCSSR-liv2&+x<-NCCI
GAN-NA-ppretaiinee
27.76(+6.24)
0.178(+0.112)
34.21(+5.20)
0.103(10.073)
NISSN-div22::22-111111
32.59(+7.06)
0.133(+0.115)
NLSN-diiv24:<2-11-22
32.22(+6.77)
0.134(#0.115)
NTSSN-divv24--211111-33
32.54(+7.06)
0.130(+0.115)
32.08(+6.83)
0.086(10.103)
31.64(#6.59)
0.085(+0.102)
31.88(+6.70)
0.090(+0.103)
N.SI-vli22--22VCGG+4hv....
31.95(+6.74)
0.091 (+0000++)
N.SV--liz2=-22VGG--Ahv...
32.00(+6.82)
0.091(10.103)
N.SV--hh222-22-VG64444d..tt.
31.57(+6.54)
0.095(+0.103)
29.51
0.259
NLSNNditv26xx4II1-ssl
27.91(+6.03)
0.289(+0.150)
NLSNN-div2X-xx4111122
28.06(+6.18)
(.204(+0.110)
NLSN-div226x4411-33
28.15(#6.25
(1201(11
26.49 (+5.54)
0.170(+0.105)
N.SX-din22-x<-Reeee444k----
27.17
0.193(+0.111)
26.53(+5..7
0.164(+0.104)
N.S1--dhv22-x44VV00444kk-1
26.05
0.180(10.106)
26.24
0.179(+0.108)
26.56(+5..2 (-5.22)
0.196(+0.113)
NLSN-HHick:22-22-L11al
(2.29(+6.90)
0.136(+0.116)
24.44(+3.75)
0.117(10.106)
31.93 31.93(+6.78) (6.78
0.095(10.104)
N.SSN-Hhick22b<<44111111
27.89(+6.05)
0.291
26.72(+5.61)
0.169
25.97
0.185(+0.110)
(6695)
0.132
17

---

Published in Transactions on Machine Learning Research (10/2022)
NLSN-quarter div2k-x2- Adv
31.62(=6.62)
0.073(#0.102)
NI.SN-quarter div2l=x2-VGGG
31.57
0.090(+0.104)
NLSN-quarter div2k-x4-L1-s1
28.10(#6.25)
0.294(+0.151)
NLSN-qqarterr Adrt
25.91(#5.27)
0.175(+0.106)
NI.NN-quarter Viv2l=x4-VGG+++
26.16(+5..0)
0.186(+0.106)
NLSN--qaarte flickr2k-x2-111-s]
32.46(+7..1)
0.136(+0.116
NLSN-qquarter
31.30(+6.76)
0.089(+0.104)
NLSN-quarter
31.32(=6.44)
0.091(#0.103)
NLSSN-quarter flckkr2k-<4-1111111
27.87
0.299(+0.151)
NLSN-quarter Nickkz)-x<h-eeeNeettAAAk
26.81(+5.62)
0.172(10.108)
NLSN-quaarter hickh
36.66(++..2)
0.189(+0.110)
32.79(=7.06)
0.126(+0.114)
RCAN-div22-x2--11-sl
(2.00
0.135
RCAN-divv2l-x2-I1-22
32.46(+7.03)
0.139
CCAN-div2l=x2-LI1-33
32.67
0.132(#0.116)
CCAN--iiz2kxx2-Reeetttt-at
31.88
0.079(10.102)
ACAN-diiv22::22Reseettt-AAAi
(2.14
0.080(#0.103)
32.07(++6.00)
(..10(+0.102)
RCANNdin2hex22VGGG+khh...l
31.5(
0.091(+0.102)
RCAN-Ltv22-:2-VGG-4440....
31.99
0.083(10.102)
RCAN--dh222:222VGGG444----
32.06(+6.88)
0.090(+0.103)
RCANddli2eex4IIMMMMprrhdah
28.32
0.282(+0.152)
RCAN-div2kex44111-al
28.17(+1.225)
0.290(10.151)
RCAN-div22:x44111s5
28.00
0.302(#0.151)
RCAN-div24-x4-11-s)
(66.26))
0.299(10.152)
26..4)+
0.174
27.41
0.199(+0.117)
27.19
0.187(10.099)
2+.33(+).90)
0.176(+0.107)
RCAV--dv222xx44VV66--4----
27.26(+5.85)
0.176(#0.110)
27.06(+5.88)
0.176(+0.106)
RCAN--Hicdr22:22-11111
32.64(+7.09)
0.132(100116)
RCANifiidzzkk222RRsetttttttt
31.87
0.092(+0.104)
RCAN-llik22k-2--VGG--li
32.13(+6.91)
0.083(+0.102)
RCAN--fickk22e:44411111
28.06(+6.25)
0.300(+0.151)
RCAN-liir22--x44deeeee44-------d
26.21
0.168(10.105)
RCAN-
27.55(+6.08)
0.167(+0.107)
32.55(+6.96)
0.128(+0.116)
div2k
32.21
0.083(+0.103)
32.08 ++-681))))))))))))))))))))))))))
0.093(+0.104)
RCAN-quarter div2k-44-LI-s1
08.15(#6.25)
0.289(+0.151)
RCAN-quarter div22xx--RsNot-Adv...
27.04
0.173(+0.101)
RCAN-quarter div2k=x4VVGG+Adlv..
27.38(+5.99)
0.170(10.112)
RCAN-quarter flcck22--x2-111s11
32.55(+7.08)
0.135(+0.117)
RCAN-quarter
32.02(+6.89)
0.090(+0.103)
RCAN-quarter flcca22hxx2-VCCG--AAA
31.90(#6.79)
0.091
RCAN-quarter flickr2k-x4-111-s1
28.03(++..5))
0.302(+0.151)
RCAN-quarter ficcee---s---eeeeeeetttttttt
27.24(#6.09)
0.180(+0.105)
MCAN
27.34
0.197(10.118)
RDN-dir2hee22--LXXXAeerttteeee
32.34(+6.77)
0.131
RDN--div2l<x2-11lsssl
32.47(++6.96
0.128(+0.115)
RDN-div24=x2-L1-22
31.75(+6.49)
0.133(10.116)
RDN-divvlex2-L1-s8
32.51(+7.06)
0.136(#0.117
RDN-din22ex2RReeeett-AAAet
30.76(+5.94)
0.088(10.103)
18

---

Published in Transactions on Machine Learning Research (10/2022)
RDN-div2e=x2-ReNNer---Wdy
32.19(#6.97)
0.088(H0.103)
RDN-iiv2k==22RRsseettt------
32.04
0.081 0.0.0(+0.10))
RUNNddiz2kee2VVGGG--Atvvvaa
31..7
0.079(10.102)
RDD-cdi/21xx2VVGGG-Adv....
(1.58
0.089
31.88(+6.70)
0.087(+0.102)
RDNVaivzlexx4IA-AA4ettraann.
28.25(+6.35)
0.290(+0.152)
RD.N-div2l:x4-L11al
28.05(+6.22)
0.292(10.150)
RDN-div2l:x44L11s5
28.11(#6.27)
0.287
RDN-divv2e:44111133
27.90(+6.12)
0.295 (+.15(++
24..2)+4.22)
0.211(+0.109)
25.81 25.81 (+-5.52)
0.184(+00106)
RDD-adis2-oddRReNeee---d----
26.12 (5..99
0.187(=0.102)
RDNVdiv2lixx4VVGG--dly...ll
26.96(+5.75)
0.179(#0.109)
RDN-ddi22:xx4VVGG--Wll-->>>
27.21(45.99)
0.189(+0.108)
RDN-div2kex2&VGGG--Att..
26.92 #5..11)
0.184(+0.109)
RDN-flickr22=x2211-1]
32.47
0.136(10.116)
HDN-fficcr2ks222ReeeettttAAA
29.80(#5.63)
0.092(#0.103)
HDDNIfickr222222VVG--Ak
31.82(+6.67)
0.090(+0.103)
LINN-ficcr22kx44-L1-11
28.09
0.300(+0.151)
26.81
0.187(+0.104)
RDNe
27.11
0.186(#00106)
RDN-quarter 1iv11-22--11111
32.59
0.136(+0.117)
31.07(+1::22)
0.094(10.104)
RDN-quarter divv
32.05
0.086(+0.103)
RDN-quarter -i--s
28.11(+6.30)
0.288(100149)
RDN-quarter All
26.52(+5.78)
0.174(+0.104)
RDN-quarter div2k=x4-VG+++Advv....
26.95
0.175(+.0108)
RDN-quarter fickr2l-x2--L1ss])
32.57
0.135(H0.117)
RDN-quarter flick22:x2-RReeNet-AAdi
31.34(+6.59)
0.084(+0.102)
flckr2h-x22VVGG-+AAtV
32.18(+6.94)
0.09
RDN-quarter fliokr2h-x4-111-s1
27.99(+6.19)
0.299(+0.152)
RDN-quarter hiwtee/eeeeeeeeeeeeeeee--------e
26.30(+5.81)
0.208(10.112)
RDN-quarter Sicgaaatccc.iihcaaa2a2voVV----a
27.16(#5.78)
0.184(+0.110)
Real BSInGAAVacii2-2222AAAAAAAApeeee
27.79(+6.10)
0.129(+0.107
Real DSScCANNaiyyax<eAAAAAVAApaaaaaa
24.84(+5.53)
0.206(+00109)
28.31(+6.27)
0.284(10.152)
SPSR-div22ex44-5PSRR
27.44(+6..0)
0.166(+00111)
STFRNNMM--21111M--Perrriii
32.69(+7.00)
0.129
SBPPNN.Msxx41.-VAppccrrn..
28.20(+6.25))
0.292
32.85
(.124(+0.114)
SwinllR-div2::2--L1-Lll
32.23 #6.72)
0.134(+0.115)
SwinIRR-i1v2::2--1111222
32.53(+7.07
0.130(+0.114)
Swin1RR-iiv2-=22-111-33
31.77(+6.43)
0.129
30.05(+5.86)
0.093(+0.102)
29.35
0.097(10.102)
30.89(+6.19)
0.083(+0.102)
31.60(+6.64)
0.086(+0.103)
SvinlR-div22-22VVG--A11---
31.38(+6.33)
0.095(+0.104)
30.94(16.08)
0.092(10.104)
24.74(#5.54)
0.200
28.39(+6.22
0.279(+0.152)
SwinIBR-div24xx4111111
27. 27561
0.295(10.149)
SwinIR-divv3<<44111929
27.98(+6.15) (5)
27.98
0.294(+0.149)
SwinIRRdiiv2<x44111s33
27. 27.97
0.295(+10.19)
19

---

Published in Transactions on Machine Learning Research (10/2022)
SvinlB-divvkex--RReeeeed-ddddddd
25.45(#5.15)
0.217(+0.107)
26.30(++5.34
0.194
24.20(+4.77)
0.240(10.111)
26.51
0.195(+0.106)
24.97(++..5)
0.216(+0.109)
26.39
0.194(+0.112)
Swiinll-fficc22:::2--111111
32.01(+6.71)
0.132(H0.115)
(1.03
0.094(+0.104)
31.14(+6.28))
0.090(+0.104)
Swinnlt-fiick22-><4-111-1111
28.13
0.286(10.149)
25.88(+53) ++5.15)
0.228(+00103))
24.99 (4.99( #4.28)
0.215(+0.109)
32.26(+6.77
0.133(+0.115)
31.43(+66.3)
0.087(+0.102)
SwinlR--quarter -iv
#1.52(46.50)
0.088(+0.103)
SwinlR-quarter div-sx---lL1
(6.99
0.285(+0.148)
SwinIR-quarter -it
25.67)+
0.217(+0.107)
SwinIR-quarterr -ib
26.10(+-..1)
0.201
SwinlR-quarter flickr2kxx22LL-ss1
(7.00)+
0.132(+0.115)
SwinlR-quarter
28.88(+5.04)
0.081(+0.101)
SwinlR-quarter flcck22h-xx--VGGG++AA
31.56(+6.52)
0.096(+0.105)
SwinlR-quarter fickkl2-kkcit2---411-1-1lll
27.97
0.296(+0.1500
SwinlR-quarter flickrbexxd-ReeNNeettAk....
25.38(+4.89)
0.219(10.108)
26.06(
0.222(#+011)))
pooRRdivzkox--LLAAppeeaadddd
28.16(
0.292(+1..51)
27.58(+6.17)
0.167(+0.109)
20


Reviews:
[{"review_id": "OSIuvywHi8", "reviewer": "Reviewer_oBKa", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength\n* Interesting observation on the fingerprint of SISR\n* Construction of dataset of SISR networks\n* Extensive experimental fingerprint results on diverse hyper-parameter of SISR\n\nWeakness\n* I think it is difficult to find the strong contribution of this paper. The existence of fingerprints is already known and shown in Yu et al (2019). It is difficult to say that it is a large contribution to experimentally confirming the fingerprint characteristics in the SISR model rather than the GAN model. \n* Moreover, it is difficult to know what is a big difference in fingerprints between SISR and GAN.\n* Presentation is poor in Figure 5. It is difficult to distinguish the dot. And is there any meaning of shape? \n* Statical results should be included. I hope the author conducts at least 3 random seeds on the classifier.\n\nQuestion\n* I do not quite understand why \"-\" indicates no sense experiments in table 5.\n* I might miss, how did you split the train set and test set for the attribution classifier?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper explores the fingerprints of single-image super-resolution networks. The author generates super-resolved images by 205 different models with diverse hyper-parameter (e.g. scale, loss, etc.). With these datasets, the author demonstrates attribution classifier is able to discriminate the attribute from the image itself. Moreover, it is possible to reverse-engineer some of the model's hyperparameters from the images.", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "Please address the questions listed in weaknesses.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "4AX7T4RT7V", "reviewer": "Reviewer_Kicq", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The proposed dataset of generated images from a large number of SISR models can benefit future research.\n- The experiments are extensive and the analysis of empirical results provides insights for two primary tasks in the paper, model attribution and model parsing.\n\nWeaknesses:\n- The paper did not include comparison with baseline methods and it was not clear whether the proposed classifier model can be compared with methods in model attribution and model parsing respectively.\n- I am curious whether the analysis of fingerprints in model attribution and model parsing can be further extended to improving performance of the original SISR models and that would be more exciting.\n- The generalization of model attribution classifiers was only concluded from the embedding visualization in Figure 5. A more rigorous study is necessary to come to the conclusion.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper studied the problem of fingerprints of super resolution networks. Specifically, it crafted a benchmark dataset from 205 SISR models and conducted extensive experiments on those models to evaluate fingerprints for model attribution and model parsing. Results have shown some interesting observations such as the generalization of model attribution classifier, and the potential reverse-engineering of a black box model.", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "- A comparison of baseline methods is necessary and can make the paper more convincing (if applicable).\n- I hope that authors can elaborate more on the potential practical use of the conclusions in the paper to improve performance of the SISR models. \n- Experiments can be further improved, such as including some statistical results of the generalization performance of model attribution classifiers rather than visualization.", "broader_impact_concerns": "N/A.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "PoXUB2JLFo", "reviewer": "Reviewer_KmiB", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n+ Model parsing/reverse engineering of model fingerprints is an exciting and novel research problem.\n\n+ The collection of SISR models would benefit future studies on reverse engineering of model fingerprints. \n\n+ The empirical study is extensive. Fig. 6 is insightful for the model parser. \n\nWeaknesses:\n\n- It is not clear to me about the novelty of the classification network part. \n\n- I would like to see a more detailed analysis of the choice of model architecture, super-resolution scale factor, loss function, training dataset, and random seed. What is the rationale behind focusing on these hyperparameters and how they are set in experiments.  ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper studied the problem of model parsing/model fingerprint identification for single image super-resolution (SISR) networks. I find this problem formulation interesting and agree that this research area is under-explored. The collection of SISR models and the extensive study of attribution classification are plausible. I am generally optimistic about this submission. ", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "Please address the questions listed in weaknesses. ", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "4GIKbeozkC", "reviewer": "Reviewer_dd7R", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n* novelty of studying the fingerprint of SISR models exclusively.\n* very extensive study, using a variety of models and data.\n* provide a lot of data about the experiments\n\nWeaknesses:\n* The authors' hypothesis does not have strong statistical evidence besides scaling and loss (in most cases).\n you can dispute that comment, but you also ran each experiment once, as with different data/different seeds you might be able to support your claims better.\n* The current setup and dataset make it limited to add newer models in the future (this is semi-weakness since it is not the goal of the paper)\n\n*Post-rebuttal review:*\n\nI have carefully reviewed the revised version of the manuscript taking into account my and other reviewers' feedback and accordingly authors' responses. I believe that the revised version is substantially improved and the current version of the manuscript is in solid shape and will be beneficial to the community. Hence I vote for its acceptance.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper studies the problem of fingerprinting SISR models, making hypotesis about the fingerprints and test them with varios current SISR models with diffrent setups.\nUsing those models, the paper create a SR dataset.\nIt test diffrent attribution models and conclude the use of pre-trained model, as well as test over OOD models (i.e SISR models the classification model didn't train with their data).\n\n", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "You have twice in the paper two consecutive \"the\" (\"the the\"); please erase those.\n\nIn section 4.2: I suggest you don't explain table 5 in such detail (the paragraph about moving down and left to the easier task, that's also by convention usually easier to harder ordered by the left to right), it seems easier to just look at the table, but if you want to guild the reader more, I suggest it would be part of the table description above it.\n\nFigure 6: predicting the scale looks too easy if you have so many 100% acc. is the prediction between all SISR models? if so, I'm not sure I understand what sums to 100, but if you predict each parameter separately it makes more sense, but then it does not support the claim that the models with 4x have a larger fingerprint if you exclude predicting the scale and the loss. \n\nAppendix of Classifier Training Procedure: I think you supplemented all the data in the paper; besides maybe the optimizer, I don't think you need this appendix.\n\nLoss: I think you should explicitly write the perceptual loss (VGG loss etc..).\n\nYou state that you made a dataset from querying flicker; there is room to elaborate if you used different artists, specifically different camera sets, to avoid data leakage.\n\nI don't feel like the conclusion about fingerprints is very decisive; I also don't see strong statistical evidence for the fingerprint; please add to the conclusion what you learned about your hypothesis that you mentioned in this paper.\n", "broader_impact_concerns": "I think that this paper asks very interesting questions about SISR models, and would like to see future research about it that might mitigate newer models with some general framework. also, the dataset should be published with the paper since it can be used for future research on the topic.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "99QID9D4aY", "reviewer": "Reviewer_JV1y", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n* The paper constructed a dataset for studying the fingerprinting problem on the single-image super-resolution task, which may help future research if the dataset is made public.\n\n* The paper compared the effect of different hyperparameters of the target models (single-image super-resolution models) on fingerprinting. \n\nWeaknesses:\n\n* [Major] Considered factors are hyperparameters, which are relatively minor factors. In addition to the hyperparameters of the generative models, the hyperpameters of the detection models also need to be considered, which may also affect results. \n\n* [Major] I think it is important to study the effect of methods that try to bypass detection in model training or inference (if applicable). For example, a generative network may try to perturb its output to increase the difficulty of detection. Such factors are directly related to fingerprinting.\n\n* [Minor] Only single-image super-resolution models are considered. It will be better to consider a few different generative tasks.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper studies the fingerprinting problem (including deepfake detection, model attribution, model parsing, etc.) The paper focused on the single-image super-resolution task and considered a wide choice of model architecture, dataset, scale, loss and seed. The paper studied the impact of different hyperparameters on fingerprinting.\n", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "*  Consider the effect of methods that try to bypass detection in model training or inference.\n* Consider the effect of hyperparameters of the detection models, in addition to the hyperparameters of the generative models. \n* Consider a few different generative tasks.\n", "broader_impact_concerns": "It may be better to discuss whether fingerprinting will pose security concerns to applications with generative models.\n", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper studies fingerprints of single-image super-resolution networks. The author investigates 205 different SISR models with a number of hyper-parameter settings. On a created dataset, the paper demonstrates that 1) the choice of scaling factor and loss function significantly impacts model distinctiveness; 2) the fingerprints of SISR models trained with an adversarial loss are highly sensitive to small changes in hyperparameters; 3) the SISR model attribution classifier generalizes to models outside the training set; 4) it is possible to reverse-engineer some of a model’s hyperparameters from the output images.

This is a purely empirical study paper, which tries to study the fingerprint property of SISR models. Due to multiple rounds of reviewers invitations, the reviewing time got a little delayed, with totally 5 reviewers involved. All reviewers agree the problem is interesting, and the created dataset could benefit the research community. Some major concerns from the reviewers are:

1. The contribution does not seem significant, given that fingerprints in GAN are also studied. I agree on this. As SISR can be in a smaller domain of GAN, the current studies and discoveries do not seem to be broadly useful. For improvement, the authors are encouraged to consider some potential applications of the findings, e.g., image forensics as mentioned by the authors.
2. The conclusion is not statistically verified. Since this is an empirical study, to claim some conclusions, it is necessary to do it statistically, e.g., by multiple runs.

Overall, the problem of fingerprints in SISR is interesting to investigate. Based on the two acceptance standards: 1) I believe some individuals in TMLR's audience will be interested in the findings of this paper; and 2) all the claims made in the submission have not been well supported by accurate, convincing and clear evidence. Some potential suggestions for improvement: conduct multiple runs of the models tp achieved statistical verification, as suggested by the reviewers; encouraged but not necessary: extend the paper with broader factors, for example, investigating if it is possible to break the fingerprints via some adversarial robustness methods such as adversarial training.

Decision: Accept (poster)