###### Abstract  
TinyML/DL is a new subfield of ML that allows for the deployment of ML algorithms on low-power devices to process their own data. The lack of resources restricts the aforementioned devices to running only inference tasks (static TinyML), while training is handled by a more computationally efficient system, such as the cloud. In recent literature, the focus has been on conducting real-time on-device training tasks (Reformable TinyML) while being wirelessly connected. With data processing being shift to edge devices, the development of decentralized federated learning (DFL) schemes becomes justified. Within these setups, nodes work together to train a neural network model, eliminating the necessity of a central coordinator. Ensuring secure communication among nodes is of utmost importance for protecting data privacy during edge device training. Swarm Learning (SL) emerges as a DFL paradigm that promotes collaborative learning through peer-to-peer interaction, utilizing edge computing and blockchain technology. While SL provides a robust defense against adversarial attacks, it comes at a high computational expense. In this survey, we emphasize the current literature regarding both DFL and TinyML/DL fields. We explore the obstacles encountered by resource-starved devices in this collaboration and provide a brief overview of the potential of transitioning to Swarm Learning. TinyML; decentralized federated learning; swarm learning +
Footnote †: journal: Computer Engineering  
in these devices. The training phase of a model that is going to be deployed on an edge device takes place on a server, where both the required energy and a plethora of necessary data are abundant. Then, one uses suitable tools (e.g., TensorFlow lite) so that the model's size is then increased to some KB of RAM. The converted compact model can be applied to resource-limited target devices, and this is what _TinyML_ is called [1; 4]. There are a lot of works (see the section on pruning) in which a DL model undergoes processing with the aim of minimizing its size, like pruning, quantization, or both of them, in order to meet the memory capacity requirements of edge devices. Adversely, the inference phase is a static procedure since its main purpose is to make predictions, based on the learned task, on new unseen data that the model is exposed to. The neural network model remains unchanged at its core during the inference phase, which limits its ability to learn new patterns and improve generalizations, or else it lacks _interpretability_. In order to alleviate the problem of model adaptability to new tasks, _reformable_[5] and _on-device_ TinyML were developed. Of course, reformable or otherwise _dynamic_ TinyML has to be conducted under conditions that will not make the model \"forget\" about its previous acquired tasks; otherwise, it will cause the \"catastrophic forgetting\" effect [6] while trying to expand its possibilities. Although it has been proven that there is a trade-off between the precision of saved information and non-harmful forgetting, known as the _stability/plasticity dilemma_[7], it is of great importance to minimize this phenomenon as much as possible. A lot of nascent techniques have been introduced, such as Continual Learning (CL) (see Section 3.2.2); however, the real question is how could we enable real-time energy-and-storage-efficient training (without inference) on resource-starving devices? This is how _federated learning_ (FL) [8] came to light. In FL, many node devices cooperatively train a model either by sharing their parameters (not data themselves) with a server--centralized federated learning (CFL)--which makes the global aggregations and forwards the new updated weights back to the nodes in order to continue training, or by exchanging their parameters with one another--decentralized [9] or distributed federated learning (DFL)--in order to perform training without a central coordinator. There is also a hybrid form of both of them (see the section on Hybrid FL).