# A Federated Generalized Linear Model for Privacy-Preserving Analysis
## 1 Introduction
### Generalized Linear Models
#### 1.2.1 Estimation of a Centralized GLM  
In order to estimate a GLM, we need to calculate the maximum likelihood estimation (MLE) for \(\beta\). Using the canonical link \(\eta=\theta\), the log likelihood can be written as in Equation (3):  
\[l=\sum_{i=1}^{n}log\big{(}f(y_{i},\theta_{i})\big{)}=\sum_{i=1}^{n}\frac{y_{i }\theta_{i}-b(\theta_{i})}{a(\phi)}+c(y_{i},\phi) \tag{3}\]  
To find the MLE, we use [PERSON]'s scoring algorithm for which the generic \((t+1)\)-th step can be calculated using Equation (4):\[\beta^{(t+1)}=\beta^{(t)}+\Big{(}-\mathbb{E}\Big{[}l^{\prime\prime}\big{(}\delta^{ (t)}\big{)}\Big{]}\Big{)}^{-1}l^{\prime}(\beta^{(t)}) \tag{4}\]  
where \(l^{\prime}\) and \(l^{\prime\prime}\) are the first and second derivative of the log-likelihood, which are given by Equation (5) and Equation (6), respectively:  
\[l^{\prime}(\beta^{(t)})=\frac{\delta l}{\delta\beta_{j}}=\sum_{i=1}^{n}\frac{ \delta l_{i}}{\delta\theta_{i}}\frac{\delta\theta_{i}}{\delta\mu_{i}}\frac{ \delta\mu_{i}}{\delta\eta_{i}}\frac{\delta\eta_{i}}{\delta\beta_{j}}=\sum_{i=1 }^{n}\frac{y_{i}-\mu}{Var(y_{i})}\Bigg{(}\frac{\delta\mu_{i}}{\delta\eta_{i}} \Bigg{)}x_{ij} \tag{5}\]  
\[\mathbb{E}\Big{[}l^{\prime\prime}\big{(}\beta^{(t)}\big{)}\Big{]}=\mathbb{E} \Bigg{(}\frac{\delta^{2}l}{\delta\beta_{j}\delta\beta_{k}}\Bigg{)}=-\sum_{i=1 }^{n}\frac{1}{Var(y_{i})}\Bigg{(}\frac{\delta\mu_{i}}{\delta\eta_{i}}\Bigg{)} ^{2}x_{ij}x_{ik} \tag{6}\]  
where \(x_{ij}\) and \(x_{ik}\) are the \(j\)-th and \(k\)-th element of the covariate vector for the \(i\)-th observation. Using algebra and matrix notation, we can rewrite them as Equations (7) and (8):  
\[\frac{\delta l}{\delta\beta}=X^{T}A(y-\mu) \tag{7}\]  
\[-\mathbb{E}\Bigg{(}\frac{\delta^{2}l}{\delta\beta_{j}\delta\beta_{k}}\Bigg{)} =X^{T}WX \tag{8}\]  
where \(X=(x_{1},\ldots,x_{K})^{T}\), \(A=diag\Big{[}Var(y_{i})\Big{(}\frac{\delta\eta_{i}}{\delta\mu_{i}}\Big{)} \Big{]}^{-1}\) and \(W=diag\Big{[}Var(y_{i})\Big{(}\frac{\delta\eta_{i}}{\delta\mu_{i}}\Big{)}^{2 }\Big{]}^{-1}\).  
With this, the Fisher Scoring iteration of Equation (4) can be rewritten as Equation (9):  
\[\beta^{(t+1)}=\beta^{(t)}+\big{(}X^{T}WX\big{)}^{-1}X^{T}A(y-\mu) \tag{9}\]  
and considering that \(X\beta=\eta\) and \(A=W\frac{\delta\eta}{\delta\eta}\), we can rewrite it as Equation (10):  
\[\beta^{(t+1)}=\big{(}X^{T}WX\big{)}^{-1}X^{T}Wz \tag{10}\]  
where \(z=\eta+\frac{\delta\eta}{\delta\mu}(y-\mu)\). This way, Fisher Scoring can be regarded as Iteratively Reweighted Least Squares (IRWLS) carried out on a transformed version of the response variable. The IRWLS algorithm can be described as in Algorithm 1, where \(g(\cdot)\) is the link function, \(\Delta g^{\prime}=\frac{\delta\mu}{\delta\eta}\) is the derivative of the inverse-link function \(g^{\prime}(\cdot)\) with respect to the linear predictor, and \(w=w_{1},\ldots,w_{n}\) are arbitrary weights assigned to the units (which equal to 1 by default).  
In this paper, we present our federated implementation of a GLM for horizontally partitioned data. The manuscript is organized as follows. Section 2 describes the algorithm of the presented _federated_ GLM in detail, how it was implemented, and its validation process for both accuracy and execution time. These results are shown in Section 3, where they are also discussed in detail. We also show examples where our federated GLM is being used as well as discuss possible improvements for it. Section 4 closes the paper with our overall conclusions.  
```
1:procedure
2: initialize \(\beta^{(0)}\) \(\eta=X\beta^{(0)}\) \(dev^{(0)}\)
3:loop
4: compute \(\mu=g^{\prime}(\eta)\) \(z=\eta+\frac{y-\mu}{\Delta g^{\prime}}\) \(W=w\frac{\Delta g^{\prime 2}}{Var(\mu)}\)
5: update \(\beta^{(j)}=\left(X^{T}WX\right)^{-1}X^{T}Wz\) \(\eta=X\beta^{(j)}\)
6: compute \(dev^{(j)}\)
7:if\(|dev^{(j)}-dev^{(j-1)}|<\epsilon\)thenreturn\(\beta^{(j)}\)break loop
8:else \(j=j+1\)
9:endif
10:endloop
11:endprocedure
```  
**Algorithm 1** Fisher Scoring-based Centralized GLM