# Comparative Analysis of Local Differential Privacy Schemes in Healthcare Datasets
## 2 Preliminaries
### Bayesian Ridge Regression  
BRR is a linear regression technique that integrates Bayesian principles into the determination of model parameters. The method assumes that the coefficients of the regression model adhere to a Gaussian distribution with a zero mean and a precision parameter (inverse of the variance), which is estimated from the data. The incorporation of Bayesian principles into ridge regression is designed to mitigate overfitting by introducing a regularization term [24]. The model can be represented as follows [17]:  
\[t=\mathbf{\beta}\mathbf{w}+\mathbf{\phi} \tag{7}\]  
where \(t\) corresponds to the dependent variable, \(\mathbf{\beta}\) denotes the coefficient vector, \(\mathbf{w}\) represents the independent variable, and \(\mathbf{\vartheta}\) stands for the error term, which is assumed to follow a Gaussian distribution. The BRR model introduces a prior distribution over \(\mathbf{\beta}\) and estimates the posterior distribution using Bayesian inference. The regularization term in this model helps in controlling model complexity, making it less prone to overfitting. BRR offers several advantages over traditional linear regression and ridge regression:  
* Effective Handling of Multicollinearity: BRR incorporates a regularization term, which is essentially a penalty for large coefficients. This helps to shrink the coefficients of correlated features, making them more stable and less sensitive to small changes in the data [17].
* Prevention of Overfitting: BRR's shrinkage mechanism effectively reduces overfitting by preventing the model from overemphasizing specific features [25].
* Automatic Shrinkage: BRR automatically determines the optimal degree of shrinkage based on the data, reducing the need for manual tuning of the ridge parameter.  
[PERSON] et al. [26] discuss the applicability of LASSO regression for automatic feature selection, where certain features are deemed irrelevant by setting their coefficients to zero. This highlights a fundamental difference between LASSO and BRR when estimating joint probability distributions. While LASSO tends to eliminate certain coefficients, BRR assumes that all features contribute to some extent, resulting in coefficients rarely reaching zero.  
As a graphical example, we tackle the matter from a Bayesian perspective by employing a linear model with normal errors, integrating it with a particular prior distribution for \(\beta\). For ridge regression [27], the prior follows a Gaussian distribution with mean zero and a standard deviation determined by \(\lambda\). Conversely, for LASSO [23], the distribution is a Laplacian distribution with mean zero and a scale parameter determined by \(\lambda\). Figure 2 presents a comparison of the posterior mode of \(\beta\) under a Gaussian prior and a Laplacian prior. With LASSO, the prior distribution peaks at zero, indicating an expectation (a priori) that many coefficients \(\beta\) will be exactly equal to zero. In contrast, for ridge regression, the prior distribution is flatter at zero, implying an expectation of coefficients to be normally distributed around zero. For further information on BRR, readers can consult the works [17; 24], which offer comprehensive discussions of Bayesian Ridge Regression.