# A Review of Recent Advances and Challenges in Grocery Label Detection and Recognition
## 6 Product Label Analysis
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Methods** & **System** & \multicolumn{2}{c}{**GroZi-3.2K**} & \multicolumn{2}{c}{**SKU-110K**} \\  & & **AP** & **AR** & **AP** & **AR** \\ \hline [PERSON] et al. [94] & Region matching & 0.447 & - & - & - \\ [PERSON] et al. [44] & Matching & 0.739 & - & - & - \\ [PERSON] et al. [4] & Matching & 0.735 & 0.827 & - & - \\ [PERSON] et al. [97] & Matching & 0.842 & - & - & \\ [PERSON] et al. [98] & Multi-scale CNN-based & 0.478 & 0.581 & - & - \\ [PERSON] et al. [95] & Matching & - & - & 0.580 & 0.710 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Evaluation and comparison of grocery detection and recognition methods on GroZi-3.2K_mAP. similar strategy. The Unitail framework was initially used to calculate the cosine similarity between the query and gallery images based on visual features. Then, text similarity scores were computed to determine the best class between the highest and second-highest rank list. Consequently, the model compared the encoded features of both images and used the Hungarian algorithm [100] to compute the similarity between variable text sequences. The product matching system was not trained end-to-end, so errors were accumulated and propagated by the text detector and recogniser. To mitigate propagated errors, the model exploits intermediate feature vectors from the text recogniser. The following frameworks only use text information to identify grocery products. The system becomes very independent of the text that is provided, how visible it is in the frame and how it is formatted when only text information is employed for grocery classification. [PERSON] et al. [7] suggest the use of label texts to avoid training models on a large amount of image data. The proposed system verifies the information of the barcode reader with the product classifier in a self-checkout system through natural language processing (NLP) techniques. Textual information is extracted by the pre-trained Google Vision API. After several experiments, they concluded that combining word and character embeddings improves the model's accuracy. The classification based on these text embeddings was higher using a CNN-based model instead of LSTM or random multimodel deep learning (RMDL). The images in the source domain were in RGB format while in the target domain they are monochromatic. Given the domain shift, the authors evaluated transfer learning in NLP [101]. Comparing the performance of the CNN with Global Vectors (GloVe) embeddings trained on a large corpus with a pre-trained BERT [102] model available on [103], the results showed that the BERT was more robust for the domain change. The quality of the image and text description will determine how well the model performs. Products without text information or scarce information are frequently misclassified. The algorithm also performs poorly when there is partial or total text occlusion. Specifically for package label detection and transcription, the authors of [104] proposed the pre-trained SegLink [105] to detect text instances. Assuming that the texts are regular and horizontal, read from left to right, its transcription results from the combination of CNN and RNN networks, trained with Connectionist Temporal Classification (CTC) loss function. They compared two versions, the Bidirectional LSTM and the Bidirectional Gated Recurrent Unit (GRU). The models achieved similar recognition rates, although BGRU has the advantage of being a lightweight method. In addition to the limitation of recognising regular texts, dense background environments also pose challenges with the main difficulties for text transcription identified as the character size and the font. The results could be influenced by the CTC-based approach, since they are not as effective as attention-based methods. The model of [PERSON] et al. [106] applied the YOLOv5 [107] to detect a grocery product and the detected item passes through the text detector to obtain product information.