## 6 Results
### Evaluation of Privacy Protection  
In this section, we evaluate the privacy protection offered in our proposed framework by studying different hyperparameter configurations. Our goal is to find a configuration that offers the best performance while preventing the exposure of sensitive information to the cloudlet. In the following experiments, we evaluate our model based on how well the cloudlet can reconstruct the original video frames using the feature maps sent from the drone. We model the cloudlet's attempt to learn sensitive information as an _image reconstruction attack_, with the threat model illustrated in Figure 3. The attack is implemented using an Autoencoder architecture. The attack results are shown in Figure 4 and they are based on the methodology in Figure 5. The autoencoder model comprises an encoder that learns a latent representation of the input (the raw video frames/images) and a decoder that can reconstruct the original input. During training, we choose the drone's backbone feature extraction model as the encoder, which generates the feature maps sent by the drone as the latent representation. The decoder is an adversarial model that the cloudlet uses to learn sensitive information. Here, we freeze the encoder because the adversary (cloudlet) does not have access to the parameters of the onboard model. Figure 4: Comparison without our proposed scheme. The first column shows original images from the PennFudan dataset. The second column shows images reconstructed using unprotected feature maps (without our scheme), and the third column shows images reconstructed with an annulment factor of \(\lambda=0.3\), noise variance of \(\sigma^{2}=0.4\), and mean of \(\mu=0.1\). The annulment value used is based on a Gaussian distribution (i.e., \(k\sim\mathcal{N}(0,1)\)). Figure 5: Training methodology for the adversarial decoder. Figure 3: Simplified attack diagram exploiting unprotected features. In this setting, we assume that the cloudlet has access to image-feature map pairs based on the onboard drone model similar to a black-box scenario. For the CelebA dataset, we train the adversarial decoder from scratch with a learning rate of \(1\times 10^{-4}\) and a batch size of 64 with a weight decay of \(1\times 10^{-5}\) for 7 epochs. The results can be reproduced with sufficient accuracy even when using approximately 5000 training images (including augmentation). For the PennFudan dataset, we do not train the model from scratch since the number of samples is too small (\(\sim\)500). Instead, we fine-tune the model on the dataset using the ImageNet pre-trained weights for 20 epochs. We use Gaussian noise (\(\sigma^{2}=0.15\), \(\mu=0.05\)), horizontal flips, and vertical flips for data augmentation. We conduct both _quantitative_ and _qualitative_ evaluations of the effectiveness of the reconstruction attack. The quantitative measures are useful for observing trends for hyperparameter configurations, but we also present a qualitative analysis of our system because the existing metrics are not designed with privacy in mind. The quantitative results are presented for the CelebA dataset and the model configurations are evaluated based on two classic metrics that are known for measuring image similarity: the Structural Similarity Index (SSIM) [19] and Multiscale Structural Similarity Index (MS-SSIM) [20]. In Figures 6 and 7, the impacts of the annulment factor (\(\lambda\)) and noise variance (\(\sigma^{2}\)) on SSIM and MS-SSIM are shown. In Figure 6, we see that the reconstruction quality is inversely proportional to the annulment factor (\(\lambda\)). We choose \(\lambda=0.3\) as the annulment factor because it is the smallest value that offers sufficient protection against reconstruction from the cloudlet. Ideally, we want the SSIM and MS-SSIM scores to be as low as possible to preserve privacy. Figure 6: Reconstruction quality (SSIM, MS-SSIM) vs. annulment factor (\(\lambda\)) for a victim MobileNetV3 backbone model on the CelebA dataset.