## Appendix D
### Phase 2: Automation of ML-Based Model Management and Deployment with Rancher Pipelines  
Rancher pipelines [18] enable CI/CD of code into production. Rancher pipelines was integrated with the SBN-implementation [23] for enabling automatic creation of Docker images. The demonstration was executed as follows (Figure 10):  
1. A K8S cluster was created to a Virtual Machine (VM) (VM1) in the private cloud. The cluster was registered into Rancher (VM2).
2. Docker registry [56] was initialised to another VM (VM3) in the private cloud for storing of Docker images.
3. Code of the existing SBN-implementation was published in GitLab (VM4). A Docker-file was added to the code base for building a Docker image based on the code.
4. An application was added to GitLab for integration with Rancher.
5. The GitLab-application was authenticated with Rancher (Tools/Pipeline).
6. A pipeline was added into the K8S-cluster with Rancher. Location of the Dockerfile and Docker registry were provided.  
* Building of the Docker image was triggered from Rancher UI, or based on a Git commit to GitLab. The SBN-implementation was downloaded from GitLab, and placed into the K8S-cluster for building.  
Feasibility evaluation:  
* Building of the Docker image based on the SBN-implementation (at GitLab) was triggered successfully based on the Git commit. However, the Docker image could not be built due to an error encountered in building. Particularly, the latest version of pip could not be installed, and Docker image creation stopped.
* It was concluded that Rancher pipelines may be utilised for CI/CD of code, but the feature was not applicable for our purpose. Thus, we decided to proceed without Rancher pipelines for further development.  
Phase 3: Continuous Training and Switching/Lplating (CI/CD) of ML-Based Model(s)/Uninterrupted Operation of Prediction Services: Multi-Thread Predictors in a K8S Pod  
An architecture was designed for a system, in which multiple (2) predictors (running in separate threads) were placed within a K8S Pod (Figure A3). With this approach, a predictor was loaded/updated based on a weight file, which had been trained earlier. Also, the old predictor (created based on an old model weight file) was deactivated. The following experiments were executed:  
1. Two K8S clusters were created and registered to Rancher (VM1). One cluster was running in the private cloud (VM2). The other k3s cluster was executed in the edge node (RPi 4).
2. The architecture consisting of multi-threaded predictors, a weight receiver, and a prediction service was placed to the K8S cluster of the private cloud (VM2).
3. The predictor was queried for energy consumption prediction from the predictor service, which forwarded queries to the active predictor.  
* Model switching was experimented by transferring new weight files to a weight receiver, which managed switching of the predictors. Each time a new weight file was received, the inactive predictor was loaded with a new weight file, and the old predictor was deactivated.
* Location of the active predictor was updated to the prediction service.  
Feasibility evaluation:  
* First, a 64-bit OS (Ubuntu 20.04; aarch64) had to be installed to RPi 4 in order to be compatible with k3s (32-bit Raspbian installation wasn't compatible with k3s; See the issue in k3s GitHub [57]). After the installation of Ubuntu 20.04 in RPi 4, the k3s cluster was registered successfully to Rancher.
* It was discovered that a Keras/Tensorflow-session of the old predictor had to be closed, before the new predictor could be initialised successfully. Thus, predictions couldn't be provided to end users from the old predictor during the switching operation. This caused an additional delay to the provisioning of predictions. Based on the discovery, it was decided to place predictors to separate K8S Pods in further design of the architecture.