# Privacy Auditing in Differential Private Machine Learning:
## 3 Privacy Attacks
### Type of Attacks
#### 3.3.3 Model Inversion Attack  
These attacks exploit the released model to predict sensitive attributes of individuals using available background information. Existing DP mechanisms struggle to prevent these attacks while preserving the utility of the model [135; 136; 108]. The model inversion attack is a technique in which the attacker attempts to recover the training dataset from learned parameters. For example, [PERSON] et al. [58] use model inversion attacks to reconstruct training images from a neural network-based image recognition model. In these attacks, the attackers use the released model to infer sensitive attributes of individuals in the training data or the outputs of DPML models. These attacks allow attackers to infer sensitive attributes of individuals by exploiting the outputs of the model [29; 135; 136]. The idea of _model inversion_[137] is to invert a given pre-trained model, \(f_{\theta}\), in order to recover a private dataset, \(D\), such as images, texts, and graphs. The attacker who attempts to use the _model inversion attack_[28; 138; 139; 140] queries the model with different inputs and observes the outputs. By comparing the outputs for different inputs, the attacker identifies and recognizes patterns. By testing each feature, the attacker can consequently infer the pattern in the original training data, resulting in a data leakage. A common approach for this type of attack is to reconstruct the input data from the confidence score vectors predicted by the target model [100]. The attacker trains a separate attack model on an auxiliary dataset that acts as the inverse of the target model [141]. The attack model takes the confidence score vectors of the target model as input and tries to output the original data of the target model [113]. Formally, let \(f_{\theta}\) be the target model and \(g_{\theta^{\prime}}\) be the attack model. Given a data record \((x,y)\), the attacker inputs \(x\) into \(f_{\theta}\) and receives \(f(x)\) and then feeds into \(g_{\theta^{\prime}}\) and receives \(g(f(x))\), which is expected to obtain \(g(f(x))\approx x\); that is, the outputs of the target models and the attacks model could be very similar. These attacks can be categorized as follows:  
* Learning-based methods. * White-box inversion attacks. * Black-box inversion attacks. * The gradient-based inversion attacks. **Learning-based methods:** These methods can reconstruct diverse data for different training samples within each class. Recent advances have improved their accuracy by regularizing the training process with semantic loss functions and introducing counterexamples to increase the diversity of class-related features [142]. **White-box inversion attacks:** In such attacks, the attackers have full access to the structure and parameters of the model. Auditors use the white-box inversion as a worst-case scenario or use the model's parameters of the model [143] to assess whether the DP mechanisms preserve privacy even with highly privileged access. **Black-box inversion attacks**: In such attacks, the attackers only have access to the output labels of the model [144] or obtain the confidence vectors. Black-box attacks simulate typical model usage scenarios, allowing auditors to assess how much information leakage occurs purely through interactions with the model's interface. **Gradient-based inversion attacks** (also known as input recovery from gradients): The attacker accesses gradients shared during the training rounds (especially in FL) and uses these to infer details about the training [145]. Auditors use these attacks by masking sensitive information, particularly in collaborative and decentralized working environments. Traditional DP mechanisms often fail to prevent model inversion attacks while maintain model utility. Model inversion attacks are a significant challenge for DP mechanisms, especially for regression models and graph neural networks (GNNs).