## 3 FedCKA
### Modifications to FedAvg
By regularizing naturally similar, but not naturally dissimilar layers, during local training, performance improves beyond previous studies. We also show that FedCKA is currently the one of the best regularization methods with adequate scalability when trained with a moderatly sized model. FedCKA shows that the proper regularization of important layers improves the performance of Federated Learning on heterogeneous data. However, standardizing the comparison of neural networks is an important step in a deeper understanding of neural networks. Moreover, there are questions as to the accuracy of CKA in measuring similarity in models such as Transformers or Graph Neural Networks. These are some topics we leave for future studies. Conceptualization, H.M.S. ; formal analysis, H.M.S., M.H.K. and T.-M.C. ; funding acquisition, M.H.K. and T.-M.C. ; Investigation, H.M.S. and M.H.K. ; methodology, H.M.S. ; project administration, M.H.K. and T.-M.C. ; software, H.M.S. ; supervision, T.-M.C. ; validation, H.M.S. and M.H.K. ; visualization, H.M.S. ; writing--original draft, H.M.S. ; writing--review and editing, H.M.S., M.H.K. and T.-M.C. All authors have read and agreed to the published version of the manuscript. This work was supported by the Institute of Information and communications Technology Planning and Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2020-0-00990, Platform Development and Proof of High Trust and Low Latency Processing for Heterogeneous-Astypical-Large Scaled Data in 5G-IoT Environment). The APC was funded by the Sungkyunkwan University and the BK21 FOUR (Graduate School Innovation) funded by the Ministry of Education (MOE, Korea) and National Research Foundation of Korea (NRF). Not applicable. The CIFAR-10/CIFAR-100 dataset is available at [[https://www.kaggle.com/c/tiny-imagenet](https://www.kaggle.com/c/tiny-imagenet)]([https://www.kaggle.com/c/tiny-imagenet](https://www.kaggle.com/c/tiny-imagenet)) and the Tiny ImageNet dataset is available at [[https://www.kaggle.com/c/tiny-imagenet](https://www.kaggle.com/c/tiny-imagenet)]([https://www.kaggle.com/c/tiny-imagenet](https://www.kaggle.com/c/tiny-imagenet)). The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results. The following abbreviations are used in this manuscript:

Federated Learning

Independent and Identically Distributed

Centered Kernel Alignment

## References

* [1] [PERSON]; [PERSON]; [PERSON] Deep Learning. _Nature_**2015**, _521_, 436-444. [CrossRef] [PubMed]
* [2] [PERSON] (Ed.) _The Deep Learning Revolution_; MIT Press: Cambridge, MA, USA, 2018. * [3] [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]. Communication-Efficient Learning of Deep Networks from Decentralized Data. In Proceedings of the 20 th International Conference on Artificial Intelligence and Statistics (AISTATS), Fort Lauderdale, FL, USA, 20-22 April 2017. * [4] [PERSON]; [PERSON]; [PERSON] [PERSON]; [PERSON] [PERSON]; [PERSON]; [PERSON] A Survey on Distributed Machine Learning. _arXiv_**2019**, arXiv:1912.09789. * [PERSON] et al. (2020) [PERSON]; [PERSON]; [PERSON]; [PERSON] Distributed Federated Learning for Ultra-Reliable Low-Latency Vehicular Communications. _IEEE Trans. Commun._**2020**, _68_, 1146-1159. [CrossRef]
* [PERSON] et al. (2020) [PERSON]; [PERSON]; [PERSON]; [PERSON] Federated Learning via Over-the-Air Computation. _IEEE Trans. Wirel. Commun._**2020**, _19_, 2022-2035. [CrossRef]
* [PERSON] et al. (2018) [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON] Federated learning of predictive models from federated Electronic Health Records. _Int. J. Med. Inform._**2018**, _112_, 59-67. [CrossRef] [PubMed]
* [PERSON] et al. (2021) [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; et al. Advances and Open Problems in Federated Learning. _arXiv_**2021**, arXiv:1912.04977. * [PERSON] et al.