Planogram compliance control via object detection, sequence alignment, and focused iterative search

Abstract

Smart retail stores are becoming the fact of our lives. Several computer vision and sensor based systems are working together to achieve such a complex and automated operation. Besides, the retail sector already has several open and challenging problems which can be solved with the help of pattern recognition and computer vision methods. One important problem to be tackled is the planogram compliance control. In this study, we propose a novel method to solve it. The proposed method is based on object detection, planogram compliance control, and focused and iterative search steps. The object detection step is formed by local feature extraction and implicit shape model formation. The planogram compliance control step is formed by sequence alignment via the modified Needleman-Wunsch algorithm. The focused and iterative search step aims to improve the performance of the object detection and planogram compliance control steps. We tested all these steps on two different datasets. The results show that our proposed method achieves a 0.992 F1 score in object detection and a 0.935 F1 score in planogram compliance control. We further analyzed the strengths and weaknesses of the proposed method from different perspectives. We finally summarized possible extensions to our work.