# A Survey of Deep Learning for Electronic Health Records
## 5 Deep Learning Applications in EHR
### Privacy Preservation
#### 5.4.2 Patient Representation Protection  
In most cases, trained patient representation is required before the following procedure (such as disease prediction). This representation contains a lot of information about patients, and attackers can use it to obtain confidential information. [PERSON] et al. [78], which is inspired by [PERSON]'s adversarial learning, adds a discriminator after representation learning, intending to obtain a patient representation that can predict tasks effectively but cannot detect private information. The final objective function is shown in Equation (8), where \(X\) is the cross-entropy function, \(x\) is the input, \(y\) is the prediction goal, and \(b_{i}\) is the prediction objective of the \(i_{th}\) privacy information. The joint learning is formed by the model parameter theta \(M\) and the identifier parameter \(\theta_{D}\).  
\[\hat{\theta}=\min_{\theta_{M}}\max_{\theta_{D}}=X(\vartheta(x;\theta_{M}),y)- \sum_{i=1}^{N}\lambda_{i}X\Big{(}\hat{b}(x;\theta_{D_{i}}),b_{i}\Big{)} \tag{6}\]  
The model parameter \(\theta_{M}\) and the discriminator parameter \(\theta_{D}\) are utilized to construct the joint learning utilizing the prediction target of privacy information. [PERSON] et al. [34] also exploited the adversarial concept, dividing the network into two sections for alternate training utilizing self-joining adversarial network simulation attacks. To begin, the de-recognition and representation models are pre-trained simultaneously to optimize the de-recognition loss value \(l_{dial}\). The second step is to freeze representation model training for confrontation model training and maximize the confrontation model's loss value \(l_{adv}\). The alternate training of the two continuously improves the overall system's ability to resist attacks, and the overall system's F1 scores are above 96 percent. [PERSON] et al. [81] studied two protective measures: the confrontational learning method and de-clustering, and verified the multi-corpus model. On any corpus, the model outperformed the baseline model by a wide margin. [PERSON] et al. [82] proposed PATE-GAN with differential privacy based on DP-GAN [105]. The model protects the information of the original data while ensuring the availability of the data. The model outperforms DP-GAN with the same privacy parameters by an average of 5%. PATE-GAN is also able to limit the impact of individual data on the model to ensure the performance of the model.  
In addition, the researchers consider the fact that in a real-world environment, many healthcare organizations intelligently train locally using a small amount of data, and data sharing is needed to have enough training data. Thus, a distributed training approach using patient representations extracted by neural networks is proposed. However, in this process, there is still a possibility that the representation information can be recovered to the original data [106]. [PERSON] et al. [80] combined the parallel training method proposed by [PERSON] et al. [107] with the de-identification network proposed by [PERSON] et al. [77] to design a collaborative privacy representation learning method to minimize the risk of PHI leakage.