## 2 Materials and Methods
### Federated Learning and Averaging  
The purpose of federated learning is to enable the training of machine learning models in a decentralized manner while preserving data privacy. Federated learning aims to leverage the collective knowledge from multiple devices or clients without requiring them to share their raw data with a central fog or cloud server. In a typical federated learning setup, a large number of client devices, such as smartphones or IoT devices, participate in the training process [36]. Each client holds its local dataset, which may contain sensitive or private information. Instead of uploading their data to a central server, clients collaborate by sharing model updates. This approach helps to overcome data privacy concerns and reduces the need for a large-scale data transfer, as only model updates are communicated between clients and the central server.  
The popularity of this technique started after the introduction of the federated averaging (FedAvg) algorithm proposed by Google's researchers in 2016 [37]. If we consider that \(N\) clients are indexed by \(i\), the fraction of clients that perform each round is \(F\), the local minibatch size is \(B\), the number of local epochs is \(M\), and the learning rate is \(\eta\), the FedAvg algorithm could be defined using the following steps [37]:  
1. Initialization: a global model is initialized on a central server (initialize \(w_{0}\)).
2. Client selection: a subset \(S_{t}\) of max(\(F\times N\), 1) clients is randomly or strategically selected for participation in each round of training.
3. Model distribution: The current global model is sent to the selected clients in parallel. For each client \(i\in S_{t}\) in parallel : \[w_{t+1}^{i}\leftarrow\text{ ClientUpdate}(i,\ w_{t})\] \[w_{t+1}\leftarrow\sum_{i=1}^{N}\frac{n_{i}}{n}w_{t+1}^{i}\] (1)
4. Local training: Each client trains the model on its local dataset using the received model parameters. This training can involve multiple local iterations to improve accuracy. \[\text{ClientUpdate}(i,\ w)\text{:}\ /\ /\ \text{run on client i}\] \[B\leftarrow\text{ (split partition }P_{i}\text{ into batches of size }B)\] for each local epoch j from 1 to M do for batch \(b\ \in\ B\) do \[w\ \leftarrow\ w-\eta\
abla\ell(w,\ b)\] (2) In the previous expression, \(\ell\) represents the loss term of a chosen loss function for training a neural network model, which varies based on the task the model is set up for.
5. Model aggregation: After the local training, updated client models are sent back to the central server, which aggregates the models' parameters by computing their average; return \(w\) to server.
6. Global model update: The aggregated model becomes the updated global model for the next round of training.  
Iterative process: Steps 2-6 are repeated for multiple rounds until convergence is reached, or until a desired performance level is achieved.  
The loss function used in both audio and video modalities is the categorical cross-entropy loss function, commonly used in image classification tasks. Since the audio data were pre-processed to visual form (spectrograms), we were able to use the same loss function. Each of our three separate clients owns local weights (\(w\)), which are unique to the client. These weights represent all trainable model parameters (i.e., layer weights and biases) that local models use. Since the models are trained in a federated fashion, the weights of the local models are also affected by other clients' parameters (global model).