## 4 Methodology
### Model Definition  
Unlike the previous deep neural network models, this model is based on a shallow neural network developed with the aid of knowledge distillation. Therefore, two models were defined, the teacher model and the student model. The teacher model is the large  
Figure 4: Sample images from the dataset. Figure 5: Proposed System setup. cumbersome model where knowledge will be distilled for the student model. The student model is the shallow network that will train under the supervision of the teacher model. The student model takes both plant images from the dataset and distilled knowledge from the teacher model as input. The student model training took place in two phases: In the first phase, it was trained on images and soft labels as a target from the teacher model, while, in the second phase, the student model will be trained on images and hard labels or ground truth as the target. First, we defined a deep neural network model with several hidden layers as the teacher model (TM). The model can be formulated mathematically below:  
Given an image, \(I\), with features (_h,w,c_), \(I\) (_h,w,c_) is an image with height, width, and color channel \(c\). The first convolutional layer takes this image and applies filter \(K\) on it for features selection:  
\[(I)v,j=\sum_{m=0}^{h-1}\cdot\sum_{n=0}^{w-1}\cdot\sum_{c=1}^{c}K(m,n,c)\cdot I( v+m,\,j+n,c)+b \tag{1}\]  
The RELU activation function is applied to the result obtained in Equation (1) to remove negative values and normalize the values:  
\[\text{A}=(\text{Max}(0,\text{x})) \tag{2}\]  
where A: is the output of the convolutional layer before the pooling layer. The output from convolution layer one is then passed into a pooling layer to reduce the feature map:  
\[\text{X}=\text{Maxpooling}(\text{A}) \tag{3}\]  
where X is the vector containing the convolve image features. The process was repeated for another three convolutional layers available in the model. The results of the final layer were passed through a flattening layer to convert the 2D array into a 1D vector. Vector W served as input to the fully connected layer. The equation for the operations taking place in the fully connected layer is a typical forward pass equation, with a softmax activation function applied to the logits of the final layer:  
\[Z=w^{T}\cdot X+b \tag{4}\]  
where \(w\) is the weight associated with each neuron. In the case of knowledge distillation, the teacher model (_TM_) will have a modification in its softmax function to generate soft labels, which will serve as distilled knowledge to the student. Therefore, _TM_ can be defined as:  
\[TM=yi(x\backslash t)=\frac{e^{\frac{zi(x)}{t}}}{\sum_{j}e^{\frac{zi(x)}{t}}} \tag{5}\]  
where \(t\) is a real number greater than 1. With this, the next step was to define a student model (_SM_) with two training phases to obtain distillation loss as well as student loss. In phase one, the _SM_ definition is similar to _TM_, and they only differ in the target labels. The _TM_ target are the hard labels from the dataset while the _SM_ target are the soft labels generated by _TM_. Therefore, _SM_ models are defined as:  
_SM_ training Phase I:  
\[TM(x\backslash t)=\frac{e^{\frac{zi(x)}{t}}}{\sum_{j}e^{\frac{zi(x)}{t}}} \tag{6}\]  
with temperature \(t\) greater than 1. \(SM\) training Phase II:  
\[\text{Student Loss}:\;SM(x\backslash t)=\frac{e^{\frac{z\langle x\rangle}{t}}}{ \sum_{j}e^{\frac{z\langle x\rangle}{t}}} \tag{7}\]  
with temperature \(t\) = 1. The general loss function can now be calculated as:  
\[\text{General Loss}:\;L(x,W)=\alpha*M(y,\sigma(SM,T=1))+\beta*M(\sigma(TM,t), \sigma(SM,t)) \tag{8}\]  
where \(x\) is the input;  
\(W\): are the student model parameters;  
\(y\): is hard label or ground truth;  
\(M\): is cross entropy loss function;  
\(\sigma\): is the softmax activation function;  
\(\alpha,\beta:\) are the constants or coefficients. In Equation (8), the overall loss function is computed, incorporating both distillation loss and student losses.