## 4 Design Principles and Novelties of the Introduced MAME-ZsL Algorithm  
As it has already been mentioned, the proposed MAME-ZsL algorithm employs MAML++ for the development of a robust _Hyperspectral Image Analysis_ and _Classification_ (HIAC) model, based on ZsL. The basic novelty introduced by the improved MAME-ZsL model, is related to a neural network with Convolutional (CON) filters, comprising very small receptive fields of size \(3\times 3\). The Convolutional stride and the spatial padding were set to 1 pixel. Max-pooling was performed over \(3\times 3\) pixels windows with stride equal to three. All of the CON layers were developed using the Rectified Linear Unit (ReLU) nonlinear Activation Function (ACF), except for the last layer where the Softmax ACF [3] was applied, as it performs better on multi-classification problems like the one under consideration (18). \[\sigma_{j}(z)=\frac{e^{z_{j}}}{\sum_{k=1}^{K}e^{z_{k}}},\ j=1,\ldots,K \tag{18}\]  
The Sigmoid approach offers better results in binary classification tasks. It is a fact that in the Softmax, the sum of probabilities is equal to 1, which is not the case for the Sigmoid. Moreover, in Softmax the highest value has a higher probability than the others, while in the Sigmoid the highest value is expected to have a high probability but not the highest one. The fully Convolutional Neural Network (CNN) was trained based on the novel _AdaBound_ algorithm [17] which employs dynamic bounds on the Learning Rate and it achieves a smooth transition to stochastic gradient descent. Algorithm 3 makes a detailed presentation of the AdaBound [17]:  
```
0:\(x_{1}\in F\), initial step size \(\alpha\), \(\left\{\beta_{1t}\right\}_{t=1}^{T}\), \(\beta_{2}\) lower bound function \(\eta_{l}\), upper bound function \(\eta_{u}\)
1: Set \(m_{0}=0\), \(u_{0}=0\)
2:for\(t=1\)to T do
3:\(g_{t}=\
abla f_{t}(x_{t})\)
4:\(m_{t}=\beta_{1t}m_{t-1}+(1-\beta_{1t})g_{t}\)
5:\(u_{t}=\beta_{2}u_{t-1}+(1-\beta_{2})g_{t}^{2}\) and \(V_{t}=diag(u_{t})\)
6:\(\eta_{l}=Clip\Big{(}\frac{\pi}{\sqrt{V_{t}}},\eta_{l}(t),\eta_{u}(t)\Big{)}\) and \(\eta_{l}=\frac{\eta_{l}}{\sqrt{l}}\)
7:\(x_{t+1}=\prod\limits_{f,diag(\eta_{l}^{+})}\left(x_{t}-\eta_{l}\cdot m_{t}\right)\)
8:endfor
```  
**Algorithm 3** The AdaBound algorithm. Compared to other methods, AdaBound has two major advantages. It is uncertain whether there exists a fixed turning point to distinguish the simple ADAM algorithm and the SGD. The advantage of the AdaBound is the fact that it addresses this problem with a continuous transforming procedure, rather than with a \"hard\" switch. The AdaBound introduces an extra hyperparameter to perform the switching time, which is not very easy to fine-tune. Moreover, it has a higher convergence speed than the stochastic gradient descent ones. Finally, it overcomes the poor generalization ability of the adaptive models, as it uses dynamic limits on the LER, aiming towards higher classification accuracy. The selection of the appropriate hyperparameters to be employed in the proposed method, was based on the restrictions' settings and configurations, which should be based on the consideration of the different decision boundaries of the classification problem. For example, the obvious choice of classifiers with the smallest error in training data is considered as improper for generating a classification model. The performance based on a training dataset, even when cross-validation is used, may be misleading when first seen data vectors are used. In order for the proposed process to be effective, individual hyperparameters were chosen. They not only display a certain level of diversity, but they also use different operating functions, thus allowing different decision boundaries to be created and combined in such a way that can reduce the overall error. In general, the selection of features was based on a heuristic method which considers the way the proposed method faces each situation. For instance:  
* Are any parametric approaches employed? * What is the effect of the outliers?