## 5 Experimental Results
### Experimental Results
\begin{table}
\begin{tabular}{c c c c c c c c c c c c c c c c c c c} \hline \hline
**ACTIVATION** & **CH** & **HE** & **LO** & **TR** & **RN** & **TB** & **LY** & **MA** & **LG** & **LA** & **CO** & **BG** & **LAR** & **RT** & **HP** & **AVG** \\ \hline MeLU (\(k=8\)) & 99.69 & 92.09 & 98.00 & 92.91 & 59.00 & 60.93 & 78.67 & 87.92 & 86.67 & 93.14 & 95.20 & 89.67 & 90.53 & 73.73 & 42.61 & 82.71 \\ \hline Leaky ReLU & 99.08 & 91.98 & 98.00 & 93.45 & 66.50 & 61.13 & 80.00 & 92.08 & 86.67 & 91.81 & 95.62 & 91.33 & 88.94 & 74.86 & 38.07 & 83.30 \\ \hline ELU & 98.77 & 93.95 & 97.00 & 92.36 & 56.00 & 59.69 & 81.60 & 90.83 & 78.33 & 85.90 & 95.78 & 93.00 & 90.45 & 71.55 & 40.91 & 81.74 \\ \hline MeLU (\(k=4\)) & 99.38 & 91.16 & 97.60 & 92.73 & 64.50 & 62.37 & 81.07 & 89.58 & 86.00 & 89.71 & 95.82 & 89.67 & 93.18 & 75.20 & 42.61 & 83.37 \\ \hline PReLU & 99.08 & 90.47 & 97.80 & 94.55 & 64.00 & 60.00 & 81.33 & 92.92 & 78.33 & 91.05 & 95.80 & 92.67 & 90.38 & 73.74 & 35.23 & 82.49 \\ \hline SReLU & 99.08 & 91.16 & 97.00 & 93.64 & 65.50 & 60.62 & 82.67 & 90.00 & 79.33 & 93.33 & 93.61 & 94.00 & 92.58 & 76.80 & 45.45 & 83.81 \\ \hline APLU & 99.08 & 92.33 & 97.60 & 91.82 & 63.50 & 62.27 & 77.33 & 90.00 & 82.00 & 92.38 & 96.00 & 91.33 & 90.98 & 76.58 & 34.66 & 82.52 \\ \hline ReLU & 99.69 & 93.60 & 98.20 & 93.27 & 69.50 & 61.44 & 80.80 & 85.00 & 85.33 & 88.57 & 95.50 & 93.00 & 91.44 & 73.68 & 40.34 & 83.29 \\ \hline Small GaLU & 98.46 & 91.63 & 97.80 & 91.35 & 64.50 & 59.79 & 80.53 & 89.58 & 77.33 & 92.76 & 95.70 & 91.67 & 91.97 & 72.63 & 44.32 & 82.66 \\ \hline GaLU & 98.46 & 94.07 & 97.40 & 92.36 & 65.00 & 59.07 & 81.07 & 92.08 & 75.67 & 93.71 & 95.68 & 88.67 & 91.74 & 75.81 & 39.20 & 82.66 \\ \hline Flexible MeLU & 97.54 & 94.19 & 96.60 & 94.91 & 59.00 & 62.68 & 77.07 & 90.00 & 89.00 & 91.81 & 95.94 & 92.67 & 89.92 & 72.15 & 38.64 & 82.80 \\ \hline TanELU & 97.85 & 93.14 & 97.00 & 92.36 & 61.00 & 61.44 & 72.80 & 89.17 & 77.33 & 91.62 & 95.28 & 89.67 & 90.23 & 72.84 & 43.75 & 81.69 \\ \hline
2D MeLU & 97.85 & 93.72 & 97.20 & 92.73 & 61.00 & 61.34 & 81.60 & 91.25 & 92.33 & 94.48 & 95.86 & 89.67 & 92.35 & 71.91 & 38.64 & 83.46 \\ \hline MeLU + GaLU & 98.15 & 93.72 & 98.20 & 93.64 & 60.00 & 60.82 & 77.60 & 92.08 & 81.00 & 93.14 & 95.54 & 92.33 & 89.47 & 75.60 & 47.16 & 83.23 \\ \hline Splash & 97.85 & 92.79 & 97.80 & 92.18 & 58.50 & 62.06 & 75.73 & 88.33 & 83.67 & 85.90 & 95.02 & 91.67 & 90.15 & 74.29 & 42.05 & 81.86 \\ \hline Symmetric GaLU & 99.08 & 92.79 & 97.20 & 92.91 & 60.50 & 60.00 & 78.93 & 88.33 & 79.33 & 91.62 & 95.52 & 92.67 & 91.67 & 73.91 & 40.34 & 82.32 \\ \hline Symmetric MeLU & 98.46 & 92.91 & 96.60 & 92.18 & 56.50 & 59.69 & 74.93 & 90.00 & 85.00 & 87.05 & 94.76 & 90.33 & 90.68 & 72.87 & 41.48 & 81.56 \\ \hline Soft Learnable v2 & 95.69 & 87.91 & 94.60 & 93.45 & 34.50 & 55.57 & 50.67 & 77.50 & 64.67 & 29.71 & 94.08 & 67.67 & 92.35 & 68.96 & 35.80 & 69.54 \\ \hline Soft Learnable & 98.15 & 92.91 & 97.00 & 91.82 & 47.50 & 54.33 & 62.13 & 86.67 & 95.67 & 65.90 & 95.04 & 84.33 & 90.38 & 71.08 & 40.34 & 78.21 \\ \hline PDELU & 98.77 & 93.60 & 96.40 & 92.18 & 59.00 & 58.25 & 76.80 & 87.92 & 87.67 & 89.33 & 95.36 & 90.33 & 91.74 & 75.24 & 42.05 & 82.30 \\ \hline Mish & 96.31 & 90.70 & 94.60 & 93.64 & 18.50 & 46.80 & 54.13 & 66.67 & 73.67 & 56.38 & 93.88 & 80.00 & 82.73 & 73.89 & 44.32 & 71.08 \\ \hline SRS & 71.08 & 59.19 & 45.00 & 51.64 & 29.50 & 31.44 & 57.60 & 61.25 & 61.00 & 45.33 & 86.88 & 57.00 & 67.50 & 39.74 & 19.32 & 52.23 \\ \hline Swish Learnable & 97.54 & 91.86 & 97.00 & 93.64 & 43.50 & 54.64 & 66.67 & 87.08 & 81.00 & 79.43 & 94.46 & 81.00 & 85.23 & 70.02 & 35.23 & 77.22 \\ \hline Swish & 98.77 & 92.56 & 96.80 & 93.64 & 63.50 & 58.97 & 80.80 & 90.00 & 89.31 & 94.68 & 93.33 & 91.74 & 75.24 & 39.77 & 83.46 \\ \hline ENS & 99.38 & 93.84 & 98.40 & 95.64 & 68.00 & 65.67 & 85.07 & 92.08 & 85.00 & 96.38 & 96.74 & 94.33 & 92.65 & 75.55 & 44.89 & 85.57 \\ \hline ENS\_G & 99.69 & 94.65 & 99.00 Examining Figure 5, which illustrates the average rank of the different methods used in Table 4, with ensembles in dark blue and stand-alone in light blue, it is clear that: (a) there is not a clear winner among the different AFs; (b) ensembles work better with respect to stand-alone approaches; (c) the methods named Sto_x work better with respect to other ensembles.