## 3 DNN-Based Demodulation Evaluation
### Evaluation Methodology  
Four different supervised DNN models were implemented and evaluated on the collected LED data. These supervised models included the VGG16 model, Resnet50 model, Inception V3 model and a simple three layer convolution model. The reason why the first three models were chosen was because these models have demonstrated good performance in many image classification problems. The first three supervised DNN models were utilized as a transfer model and had all their previous layers frozen and  
Figure 3: Four level data representation. \begin{table}
\begin{tabular}{c c c c} \hline \hline  & **RGB With Background** & **RGB Cropped/Resized** & **RGB Cropped** \\ \hline
2 Level & 97.37\% & 97.20\% & 97.10\% \\ \hline
3 Level & 96.90\% & 97.18\% & 96.98\% \\ \hline
4 Level & 90.41\% & 83.11\% & 83.17\% \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Average basic thresholding-based approach results. the last three dense layers removed. This was then followed by adding three new dense layers and two new dropout layers to the end of each of these models. For reference, these models were implemented with the Keras library, and the pre-trained weights were from when these models were trained on the ImageNet database [32]. Figure 4 illustrates the graphical representation of the VGG16 model, Resnet50 model and Inception V3 model architecture with the appended layers. Although the input of this network is a 128 \(\times\) 128 \(\times\) 3 frame, which represents the full frames, this same architecture was applied for the 32 \(\times\) 32 \(\times\) 3 frames. Therefore, only two types of frames were evaluated with the DNN models: the full-frame, which was a 128 \(\times\) 128 \(\times\) 3 image, and a cropped version, which was resized to 32 \(\times\) 32 \(\times\) 3. This is different from the basic thresholding-based approach in which cropped frames that were not resized to 32 \(\times\) 32 \(\times\) 3 were also evaluated. However, cropped frames used for the Inception V3 model were resized from the 32 \(\times\) 32 \(\times\) 3 cropped versions to 75 \(\times\) 75 \(\times\) 3 because that was the smallest image size that the Keras library allowed when using their Inception V3 model as a transfer model [32]. As mentioned earlier, only the appended layers of the transfer models were trainable because the previous layers for the VGG16 model, Resnet50 model and Inception V3 model were all frozen. As for the simple three layer convolutional model, it had all its layers set to trainable. Each time the model is compiled, the trainable layers received a new set of weights. The new weights affect the manner the model performs because of how the gradient adjusts the parameters of the model as the back propagates. This is why, if upon compile-time a \"good\" set of weights are acquired, the model can quickly learn the features, and if the weights are \"bad\" then it takes longer for the model to figure out the  
Figure 4: Appended layers for the deep neural networks. features. However, because the weights are randomly chosen at the compile-time, there is no guarantee of \"good\" weights. In order to circumvent this, each model was compiled and ran three times to obtain an average. For each model, both Softmax and Sigmoid activations were experimented with as the last activation function for the output layer. In addition, three different optimizers and three different learning rates were explored. The optimizers included RMSprop, SGD and Adam and for the learning rate 0.001, 0.0001 and 0.00001 were used. The F1 score was also taken to obtain a more intuitive understanding of the model's performance. Each model was trained for 10 epochs and dropout was set to 0.2, although other dropout values were used to reduce overfitting of some models. However, the results presented later are for a dropout set to 0.2. Each of the data levels had 500 frames set aside for validation and 500 frames set aside for testing; the rest were used for training. This was performed for the 4 Level, 3 Level and 2 Level balanced datasets.