# Detection of Household Furniture Storage Space in Depth Images
###### Abstract  
Autonomous service robots assisting in homes and institutions should be able to store and retrieve items in household furniture. This paper presents a neural network-based computer vision method for detection of storage space within storage furniture. The method consists of automatic storage volume detection and annotation within 3D models of furniture, and automatic generation of a large number of depth images of storage furniture with assigned bounding boxes representing the storage space above the furniture shelves. These scenes are used for the training of a neural network. The proposed method enables storage space detection in depth images acquired by a real 3D camera. Depth images with annotations of storage space bounding boxes are also a contribution of this paper and are available for further research. The proposed approach represents a novel research topic, and the results show that it is possible to facilitate a network originally developed for object detection to detect empty or cluttered storage volumes. storage space detection; neural network; depth images 202  
and transferring its knowledge to a mobile robot agent with a visual sensor. Training a neural network requires a large dataset with properly labeled regions of interest that the robot should recognize. Since the aim of this study was to recognize household 3D storage volumes, the training dataset had to consist of 3D scenes with household storage furniture with annotated storage volumes. To our knowledge, none of the available 3D indoor scenes datasets have such annotations. Therefore, to achieve our goal, we needed to create these annotations on an appropriate dataset. Manual annotation of the ground truth is prone to human error and time consuming; thus, we propose a method for automated annotation of 3D storage volumes in synthetic depth images. The proposed procedure whose overview is given in Figure 1, consisting of two novel methods, detects storage volumes in depth images even if these volumes are cluttered by items. The first method starts from empty 3D mesh models of household furniture, such as bookshelves, cabinets, and tables, and automatically provides ground truth annotations of the bounding boxes of their storage volumes. Such models can be found in a number of publicly available 3D model databases, such as ShapeNet [1]. The 3D models with assigned ground truth bounding boxes representing storage volumes are used as input to the second method. This method generates realistic depth images of furniture with objects, such as boxes, bottles, books and lamps, representing realistic clutter commonly found in households and randomly placed on the shelves. This approach allows fast and fully automatic generation of a large number of synthetic depth images with information about the bounding boxes of storage volumes which can be used as the training dataset for neural networks. To demonstrate the feasibility of our approach, we trained the VoteNet [2] and PCAF3D [3] neural networks, originally developed for object recognition, to detect storage volumes. The trained network was finally evaluated on a test dataset consisting of synthetic and real depth images with assigned ground truth bounding boxes of storage volumes. The images acquired by a real 3D camera were manually annotated using annotation software which we developed for the purpose of the presented research. It should be noted that these manually annotated images were used only for testing purposes, and the neural network training was performed using exclusively automatically generated synthetic data. The contributions of this paper are:  
* An empty storage volume detection algorithm which provides automatic ground truth annotation of storage volumes in empty synthetic 3D furniture mesh models. * A method for generating depth images of realistic synthetic scenes containing storage furniture with randomly cluttered storage space.