# Development of Smart and Lean Pick-and-Place System Using EfficientDet-Lite for Custom Dataset
## 3 Results
### Results of Variation of Batch Size on Average Precision  
A comparison of Average Precision across Efficient-Det Lite models is built in order to investigate which model performs the best with a small dataset. As shown in Table 14 below, Batch4 has the highest overall AP at 66.8%, and it has the lowest standard deviation at 6.23%. In contrast, Batch16 has the lowest overall AP at 57.4% and highest standard deviation at 7.87%. According to [PERSON] [25], this is because the large-batch methods tend to converge to sharp minimizers of the training and testing function and as is well known, sharp minima lead to poorer generalization. In contrast, small-batch methods consistently converge to flat minimizers due to the inherent noise in the gradient estimation. As compared to Batch16, both Batch4 and Batch8 have higher average APs of 66.1 \(\pm\) 0.7% and standard deviations of 6.245 \(\pm\) 0.015%.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**TFLite Model** & **Average Precision** & \begin{tabular}{c} **Batch4** \\ **(\%)** \\ \end{tabular} & \
\begin{tabular}{c} **Batch8** \\ **(\%)** \\ \end{tabular} &
\begin{tabular}{c} **Batch16** \\ **(\%)** \\ \end{tabular} \\ \hline \multirow{6}{*}{EfficientDet-Lite1} & Yellow cube & 72.1 & 64.4 & 66.3 \\  & Yellow cylinder & 47.9 & 53.2 & 40.2 \\  & Red cube & 66.2 & 71.4 & 73.2 \\  & Red cylinder & 62.7 & 64.8 & 56.6 \\  & Blue cube & 63.4 & 62.2 & 55.6 \\  & Blue cylinder & 59.1 & 59.9 & 54.4 \\ \hline \multirow{6}{*}{EfficientDet-Lite2} & Yellow cube & 56.9 & 73.7 & 55.5 \\  & Yellow cylinder & 72.5 & 50.3 & 40.7 \\  & Red cube & 68.6 & 70.4 & 58.2 \\  & Red cylinder & 68.1 & 64.0 & 53.9 \\  & Blue cube & 63.9 & 63.8 & 49.6 \\  & Blue cylinder & 71.4 & 59.3 & 50.7 \\ \hline \multirow{6}{*}{EfficientDet-Lite3} & Yellow cube & 73.5 & 70.8 & 67.7 \\  & Yellow cylinder & 62.4 & 59.0 & 49.2 \\  & Red cube & 72.3 & 75.8 & 70.7 \\  & Red cylinder & 68.4 & 63.0 & 55.5 \\  & Blue cube & 63.3 & 63.8 & 58.3 \\  & Blue cylinder & 70.8 & 70.3 & 56.8 \\ \hline \multirow{6}{*}{EfficientDet-Lite3} & Yellow cube & 72.5 & 69.9 & 59.3 \\  & Yellow cylinder & 57.6 & 57.6 & 54.7 \\  & Red cube & 73.8 & 71.8 & 66.0 \\  & Red cylinder & 66.0 & 68.9 & 57.4 \\  & Blue cube & 63.1 & 62.5 & 55.5 \\  & Blue cylinder & 68.4 & 67.2 & 53.3 \\ \hline \multirow{6}{*}{EfficientDet-Lite3} & \multicolumn{2}{c}{Average of APs} & 66.8 & 65.4 & 57.4 \\  & Standard deviation of APs & 6.23 & 6.26 & 7.87 \\ \hline \hline \end{tabular}
\end{table}
Table 14: Variation of batch size.  
Figure 10: Bar graph of detection scores with average and variance.  
From Table 15 and Figure 11 below, the overall results show that the red cube has the highest AP at 69.9% and the second lowest standard deviation at 4.48%. The red cylinder has the second highest AP at 62.4% with a standard deviation at 5.09%. In comparison to other classes, the yellow cylinder has the lowest AP at 53.8% and the highest standard deviation at 8.69%. This is consistent with our prior findings [26], which show that neutral-colored things such as yellow cubes have lower APs than strong-colored objects such as red cubes. As the objects for this pick-and-place action have a metallic surface, yellow objects suffer from surface reflection from external lighting.