## 3 TinyML Implemenatation
### Implementation Methodology  
The implementation methodology consists of the following steps:  
1. TinyML algorithm selection on a computer. 2. Training the selected model on a computer. 3. Testing and compressing the model on a computer. 4. Programming, deploying, and testing the trained model on the embedded system. We use Python version 3.9.9 for programming, and _Tensorflow_ and _Keras_ are the libraries that support the computer's implementation of the training algorithm. The _Tensorflow_ _lite_ library in Arduino IDE supports deploying the DNN model on the selected embedded systems. The following paragraphs explain each step in detail. **Remark 4**. : _The computer used to train the DNN and program the embedded systems has the following hardware requirements: CPU Core i5 2.4 GHz, 12 GB RAM, 1 TB solid state hard disk, no graphic board._  
**TinyML algorithm selection on a computer**. First, the dataset is selected. We utilize the Iris dataset [48]. This dataset comprises 150 samples; each is a set of entry data integrated by four measurements describing the flower's dimensions. Figure 6 represents the information in the Iris dataset. Each entry dataset corresponds to a set of four values that indicate the length and width of the sepal and the length and width of the petal in centimeters. Thus, the sepal and petal measurements determine whether the flower is setosa, virginica, or versicolor; there are three classes. Based on the results obtained in [88], where the authors used four hidden layers with the following distribution: 300, 200, 100, and 16 neurons, respectively, we decided to carry out a similar implementation and use the DNN architecture illustrated in Figure 7.  
Other works that support this implementation can be seen in [89; 90]. The Python code, which illustrates the statements that contain the libraries and plugins required for loading and data preparation as well as DNN selection and configuration, appears in Figure 8. The Iris dataset is loaded through a CSV file. Figure 6: Content of the Iris dataset. The selected DNN architecture is the following:  
1. Number of inputs: 4. 2. Total number of Layers: 4. 3. Number of Hidden Layers: 2. 4. Number of Neurons for Layer (input, output, and hidden): 16. 5. Activation function: \"Relu\" (first three layers), \"Sigmoid\" (out layer). We modified the configuration implemented in [88], and our architecture of the DNN consists of one input layer, two hidden layers, and one output layer. The first layer is configured with sixteen neurons, the two hidden layers consist of sixteen neurons, and the output layer consists of three neurons (see Figure 7). The statements \(tf\_keras.Sequential\) and \(model\_add\) allow the DNN model configuration; see code in Figure 8.  
The basis of neural networks theory establishes many other considerations about DNN architectures, such as the correct activation functions and their dependence on the task performed (classification or regression), the number of hidden layers, and the number of neurons in each layer. The criteria and considerations for selecting the DNN architecture are based on the system type, application, and the desired degree of accuracy in predictions. This implementation does not address a detailed and specific analysis of the parameters and hyperparameters of the selected DNN and the performance of hardware-constrained embedded systems under the selected architecture. However, the discussion on choosing an optimal DNN architecture plays a crucial role, and this analysis opens up exciting possibilities for future works that address optimization issues of TinyML algorithms. Figure 8: Code snippet in Python loading the Iris dataset and DNN architecture configuration. Figure 7: DNN architecture tested in Pi Pico, ESP32 and Arduino Nano 33. We selected two activation functions: \"Relu\" for the input and two hidden layers and \"sigmoid\" for the output layer. **Training the selected model on a computer**. Using DL techniques, a training algorithm is programmed for a DNN.