# Computer Vision Based Planogram Compliance Evaluation
###### Abstract  
Arranging products in stores according to planograms, optimized product arrangement maps, is an important sales enabler and necessary for keeping up with the highly competitive modern retail market. Key benefits of planograms include increased efficiency, maximized retail store space, increased customer satisfaction, visual appeal, and increased revenue. The planograms are realized into product arrangements by humans, a process that is prone to mistakes. Therefore, for optimal merchandising performance, the planogram compliance of the arrangements needs to be evaluated from time to time. We investigate utilizing a computer vision problem setting--retail product detection--to automate planogram compliance evaluation. Retail product detection comprises product detection and classification. The detected and classified products can be compared to the planogram in order to evaluate compliance. In this paper, we propose a novel retail product detection pipeline combining a Gaussian layer network product proposal generator and domain invariant hierarchical embedding (DIHE) classifier. We utilize the detection pipeline with RANSAC pose estimation for planogram compliance evaluation. As the existing metrics for evaluating the planogram compliance evaluation performance assume unrealistically that the test image matches the planogram, we propose a novel metric, called normalized planogram compliance error (\(E_{\text{PC}}\)), for benchmarking real-world setups. We evaluate the performance of our method with two datasets: the only open-source dataset with planogram evaluation data, GP-180, and our own dataset collected from a large Nordic retailer. Based on the evaluation, our method provides an improved planogram compliance evaluation pipeline, with accurate product location estimation when using real-life images that include entire shelves, unlike previous research that has only used images with few products. Our analysis also demonstrates that our method requires less processing time than the state-of-the-art compliance evaluation methods. computer vision; object detection; planograms +
Footnote †: journal: Applied  
memories, do not make mathematical errors, and are better at multitasking than humans, and therefore a computer-based compliance method is appealing. We encourage the reader who is less familiar with the latest deep learning-based methods of computer vision to look for background information from the field's foundational sources [2; 3]. _Object detection_ is a computer vision task that could provide automation for planogram compliance evaluation. Most object detection literature deals with _generic object detection_--recognizing everyday objects in everyday environments. This includes approaches such as feature pyramid networks (FPNs) [4], region-based convolutional neural networks (R-CNNs) [5], and you only look once (YOLO) [6], as well as challenges such as ImageNet [7] and MS COCO [8]. However, recognizing retail products in the environment of racks is a mix of generic object detection and _object instance detection_. Object instance detection means detecting a specific object instead of a general class, as is carried out in general object detection. In the retail product detection problem setting, the aim is to localize and classify products from images of shelves, cabinets, racks, or bays taken in retail store aisles. The recognition process is called the _retail product detection problem_[9], and it exhibits a number of difficulties in comparison to this more general problem that has made using computer vision for planogram compliance verification infeasible for quite some time. The two most fundamental difficulties of retail product detection--the small amount of available training data per class and the domain shift between training data and test data--were identified already by [10], who kick-started research into retail product detection in 2007.