# A Novel Generative Model for Face Privacy Protection in Video Surveillance with Utility Maintenance
## 4 Method
### Architecture and Working  
Our research aims to develop a multi-input generative model that uses several face images with privacy information removed to synthesize one image with quality maintenance. QM-VAE offers an inspiring framework for efficiently generating de-identified images with utility preservation through neural network training. QM-VAE is inspired by the concept of Vector Quantization (VQ), initially proposed for VQ-VAE, which integrates discrete latent variables into the structure of the model. Compared with common autoencoders, VQ-VAE creates a latent embedding space \(e\) between the model input \(x\) and the low-dimensional variables \(z\). The model's whole parameter set may be split into three sections: the encoder, the decoder, and the embedding space \(e\). The  
Figure 1: The workflow of the expression recognizer. Figure 2: The overall structure of the de-identification module and QM-VAE. model's prior distribution is used to learn to generate discrete variables. The output of the encoder will be mapped to the embedding space, and the mapped embedding vector will be fed to the decoder. The input \(x\) of the model is formed by splicing images processed by four privacy protection methods. The encoder is fed \(x\) and outputs \(ze(x)\). Different from the ordinary autoencoder model, \(ze(x)\) does not go straight to the decoder, but searches for the closest embedding in the embedding space \(e\). This embedding will replace \(ze(x)\) and enter the decoder as the latent variable \(z\). The probability of the posterior classification distribution \(q(z\mid x)\) is defined as Equation (5), which means that the index of the vector nearest \(ze(x)\) will be set to 1, and the rest will be set to 0. \[q(z=k\mid x)= \begin{cases}1,&\text{if }k=\arg\min_{j}\mid\mid z_{e}(x)-e_{j} \mid\mid_{2}\\ 0,&\text{other}\end{cases}  \tag{5}\]  
\(ze(x)\) finds the element closest to itself in the embedding space and transmits it to the decoder as its own mapping in the embedding space, thus realizing the process of discretization, as shown in Equation (6). \[z_{q}(x)=e_{k},\quad\text{where }k=\arg\min_{j}\mid\mid z_{e}(x)-e_{j}\mid\mid_{2} \tag{6}\]  
Our model extracts the latent feature space of image data. In the nonlinear process of mapping the latent vector to the embedding vector, the model simultaneously copies the gradient from the decoder's input \(zq(x)\) to the encoder's output \(ze(x)\). The gradient promotes the update of assignment in Equation (5) and, in turn, influences the result of the encoder's discretization. Since the encoder's output and decoder's input share the same \(D\)-dimensional space, the gradient will instruct the encoder to adjust its output to reduce the reconstruction loss of image fusion. Equation (7) presents the model's total loss function. The \(L_{e}\) in the first term measures the service quality loss. In this work, we took the scenario of facial expression recognition as an example, and the specific solution process is described in detail in Equation (8). The second term describes the image reconstruction loss of the decoder and encoder, that is the Mean-Squared Error (MSE) between the original images and the model outputs. The third term of the function is designed to optimize the vector-quantized embedding space and constantly update the dictionary in the process of model training. The last term is commitment loss, which constrains the encoder, preventing it from updating too quickly, so that its output deviates from the embedding vector. To sum up, the complete loss function for model training is as follows:  
\[L=\alpha L_{e}+(1-\alpha)\log p(x\mid z_{q}(x))+\mid\mid sg[z_{e}(x)]-e\mid \mid_{2}^{2}+\beta\mid\mid z_{e}(x)-sg[e]\mid\mid_{2}^{2}, \tag{7}\]  
where \(\alpha\) is used to assign the proportional weight of the loss of service and the loss of image fusion. The value of \(\alpha\) ranges from 0 to 1. In our experiment, we adjusted the value of \(\alpha\) and analyzed the corresponding quality of service. The specific results are presented in Section 4.3.