# A Review of Recent Advances and Challenges in Grocery Label Detection and Recognition
## 7 Challenges and Opportunities  
The automatic detection and recognition of grocery items has many applications with high economic impact. Moreover, the ability to analyse product labels is becoming increasingly relevant mostly as a way to improve the customer experience, which is expected to translate to higher sales and fidelity, but also as a means to assist people with impairments. Nevertheless, the process is highly challenging due to the inherent characteristics of the scenario such as visual clutter or high density in shelves. Moreover, when focusing on the label, processing the information introduces additional challenges due to small text size or blurry appearance. The difficulties associated with these scenarios typically increase when we descend the hierarchy of classification. Fine-grained classification is more challenging due to intra-class variance and inter-class similarity. Grocery items of the same macro-category share similar visual characteristics in size, shape, colour and inscriptions. On the other hand, products of the same brand are frequently only recognised through slight packaging variations. This is the case of products that differ by flavour or quantity. When these distinctions only occur in small areas, there is no guarantee that images taken from a particular angle can capture the mentioned differences. Moreover, several environmental conditions, including lighting, background and occlusions, may significantly impact product recognition, particularly with micro-categories. Consequently, \begin{table}
\begin{tabular}{l c c c c} \hline \hline \multirow{2}{*}{**Methods**} & \multicolumn{2}{c}{**Detection**} & \multicolumn{2}{c}{**End-to-End**} & \multirow{2}{*}{**FPS**} \\ \cline{2-2} \cline{4-5}  & **F-Measure** & **None** & **Full** & \\ \hline FOTS [145] & 0.440 & 0.322 & 0.359 & - \\ TextDragon [147] & 0.803 & 0.488 & 0.748 & - \\ ABCNet+MS [148] & - & 0.695 & 0.784 & 6.9 \\ ABCNet V2 [149] & 0.870 & 0.704 & 0.781 & 10 \\ ARTS-RT [150] & 0.803 & 0.659 & 0.781 & 28.0 \\ ARTS-S [150] & 0.865 & 0.771 & 0.851 & 10.5 \\ Mask TextSpotter [151] & 0.613 & 0.529 & 0.718 & 4.8 \\ Mask TextSpotter V3 [152] & - & 0.712 & 0.784 & - \\ Unconstrained [153] & 0.864 & 0.707 & - & - \\ MANGO [154] & - & 0.729 & 0.836 & 4.3 \\ TESTR-Bezier [155] & 0.880 & 0.716 & 0.833 & 5.5 \\ TESTR-Polygon [155] & 0.869 & 0.733 & 0.839 & 5.3 \\ AE TextSpotter [156] & - & - & - & - \\ SwinTextSpotter [157] & 0.880 & 0.743 & 0.841 & - \\ \hline \hline \end{tabular}
\end{table} 
Table 10: Comparison of end-to-end text recognition in Total-Tex dataset. For more details of the networks, please consult the respective papers. \begin{table}
\begin{tabular}{l c c c} \hline \hline \multirow{2}{*}{**Methods**} & \multicolumn{3}{c}{**ICDAR15 - End-to-End**} \\ \cline{2-4}  & **S** & **W** & **G** \\ \hline FOTS [145] & 0.811 & 0.759 & 0.608 \\ TextDragon [147] & 0.825 & 0.783 & 0.651 \\ ABCNet+MS [148] & - & - & - \\ ABCNet V2 [149] & 0.827 & 0.785 & 0.730 \\ ARTS [150] & 0.815 & 0.773 & 0.687 \\ Mask TextSpotter [151] & 0.830 & 0.777 & 0.735 \\ Mask TextSpotter V3 [152] & 0.833 & 0.781 & 0.742 \\ Unconstrained [153] & 0.855 & 0.819 & 0.699 \\ MANGO [154] & 0.818 & 0.789 & 0.673 \\ TESTR [155] & 0.852 & 0.794 & 0.736 \\ AE TextSpotter [156] & - & - & - \\ SwinTextSpotter [157] & 0.839 & 0.773 & 0.705 \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Comparison of end-to-end text recognition in ICDAR2015. The acronyms “S”, “W” and “G” represent the “Strong”, “Weak” and “Generic” lexicon, respectively. the recognition model should be able to identify the nuances of packages and be agnostic to environmental factors. The items typically follow an established layout in a grocery store. Those who belong to a general category are placed together on the racks. As a result, along with local features, context may serve as a guideline for identifying similar products on shelves. The grocery market is constantly evolving with products launched frequently and their appearance may change over time to appeal to consumers.