# Improving Convolutional Neural Networks' Accuracy in Noisy Environments Using k-Nearest Neighbors
## 3 Experiments  
The four configurations (kNN, CNN + softmax, CNN+ kNN and CNN + \(\ell_{2}\) + kNN) have been evaluated with and without adding synthetic noise to the data by swapping the labels of pairs of randomly-chosen prototypes. The percentages of prototypes that change their label are 10%, 20%, 30% and 40% since these are the common values in this kind of experimentation [31]. It is important to remark that each CNN has been trained separately for each noise level, from 0% to 40%. Then, the weights obtained at each noise level were used as a basis for evaluating the different CNN configurations at that noise level. \begin{table}
\begin{tabular}{l c c c c c c c} \hline \hline
**Dataset** & **Layer 1** & **Layer 2** & **Layer 3** & **Layer 4** & **Layer 5** & **Layer 6** & **Layer 7** \\ \hline \multirow{3}{*}{USPS} & \multirow{3}{*}{Conv (32 \(\times\) 3 \(\times\) 3)} & Conv (32 \(\times\) 3 \(\times\) 3) & FC (128) & & & & \\  & & MaxPool (2 \(\times\) 2) & Drop (0.5) & & & & \\  & & Drop (0.25) & & & & & \\ \hline \multirow{3}{*}{MNIST} & \multirow{3}{*}{Conv (32 \(\times\) 3 \(\times\) 3)} & Conv (32 \(\times\) 3 \(\times\) 3) & FC (128) & & & & \\  & & MaxPool (2 \(\times\) 2) & Drop (0.5) & & & & \\  & & Drop (0.25) & & & & & \\ \hline \multirow{3}{*}{Landsat} & Conv (64 \(\times\) 1 \(\times\) 1) & Conv (64 \(\times\) 2 \(\times\) 2) & Conv (64 \(\times\) 2 \(\times\) 2) & & & \\  & UpSamp (2 \(\times\) 2) & UpSamp (2 \(\times\) 2) & MaxPool (2 \(\times\) 2) & FC (256) & FC (128) & & \\  & Drop (0.3) & Drop (0.3) & UpSamp (2 \(\times\) 2) & Drop (0.3) & Drop (0.3) & & \\ \hline \multirow{3}{*}{HOMUS} & Conv (256 \(\times\) 3 \(\times\) 3) & Conv (128 \(\times\) 3 \(\times\) 3) & Conv (128 \(\times\) 3 \(\times\) 3) & \multirow{3}{*}{\begin{tabular}{} \end{tabular} } & \multirow{3}{*}{\
\begin{tabular}{} \end{tabular} } & \multirow{3}{*}{
\begin{tabular}{} \end{tabular} } \\  & MaxPool (2 \(\times\) 2) & MaxPool (2 \(\times\) 2) & Drop (0.2) & Drop (0.2) & Drop (0.1) & Drop (0.1) & Drop (0.1) \\ \hline \multirow{3}{*}{NIST} & Conv (256 \(\times\) 3 \(\times\) 3) & Conv (128 \(\times\) 3 \(\times\) 3) & Conv (128 \(\times\) 3 \(\times\) 3) & Conv (64 \(\times\) 3 \(\times\) 3) & FC (512) & FC (256) & FC (128) \\  & MaxPool (2 \(\times\) 2) & MaxPool (2 \(\times\) 2) & Drop (0.2) & Drop (0.2) & Drop (0.1) & Drop (0.1) & Drop (0.1) \\ \cline{1-1}  & & & & & & \\ \cline{1-1}  & & & & & & & \\ \hline CIFAR10 & & & & & & & \\ \hline CIFAR100 & & & & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 2: CNN architectures evaluated with each image dataset. Conv(\(f\times w\times h\)) stands for a layer with \(f\) convolution operators with \(w\times h\) kernel; MaxPool(\(w\times h\)) stands for the max-pooling operator with \(w\times h\) kernel; UpSamp(\(w\times h\)) stands for an up-sampling operator with \(w\times h\) kernel; Drop(\(d\)) refers to dropout with ratio \(d\); and FC(\(n\)) mean a fully-connected layer with \(n\) neurons. Different numbers of neighbors \(k\) (from 1 to 50) have also been tested for the kNN algorithm. Additionally, in the hybrid approach, the neural codes of the last hidden layer have been \(\ell_{2}\)-normalized to compare their performance with the unnormalized values. Table 3 reports the detailed results with the image datasets evaluated. In addition, Figure 2 illustrates the accuracy curves as the level of induced label noise is increased. In an overall sense, using kNN over the raw input data obtains a lower average accuracy than the CNN-based methods, especially with the most challenging datasets. Analogously, accuracy is strongly affected by the level of induced label noise in the dataset. All figures depict a noticeable accuracy drop as this level is increased.