# Detection of Household Furniture Storage Space in Depth Images
## 3 Methods
### Synthetic Scenes Generation  
In this section, we address the very specific task of training a neural network to predict bounding boxes of storage volumes. As far as we know, there is no publicly available dataset with storage volume annotations. Therefore, we designed a method which generates realistic synthetic datasets containing depth images of household furniture captured by a virtual camera. A depth image reproduces points from the point cloud that can be seen from a certain point of view, usually the point of view of the camera. A point cloud is a set of 3D points, where each point is assigned \(x\), \(y\), and \(z\) coordinates in Cartesian space. Each pixel of the depth image is assigned \(x\), \(y\), and \(z\) values, where \(x\) and \(y\) are image coordinates and \(z\) is the distance from the camera. In a real environment, 3D cameras provide depth images. In this work, during synthetic scene generation, depth images  
Figure 8: Examples of labeled ground truth annotations (purple bounding boxes) of storage volumes in selected furniture models: (**a**) Table, (**b**) Bookshelf A, (**c**) Bookshelf B.  
are rendered from the point clouds by positioning a virtual 3D camera on the 3D scene and capturing the depth image of the scene points from that viewpoint. The generated depth images are stored as point clouds with associated annotations of the bounding boxes representing storage space. These synthetic depth images resemble depth images captured by real 3D cameras. The detailed process of scene generation is explained below. The input to this algorithm is a 3D mesh of a piece of furniture, commonly found in homes: a bookshelf, table, desk, or cabinet. Each mesh contains information about the bounding boxes of the storage units, provided by the empty storage volume detection algorithm explained in Section 3.1. Models in publicly available 3D model datasets are not necessarily provided in realistic sizes; e.g., models are often scaled to unit size. In that case, one way to scale models to realistic sizes is to utilize datasets of real objects with associated bounding boxes. An example of such a dataset is SUNRGB-D [33] containing real scenes with annotated bounding boxes of various pieces of furniture, where the height of the object matches the height of its associated bounding box. For the purposes of this research, we have created a database of realistic heights of the objects belonging to the considered household furniture classes taken from the SUNRGB-D dataset. Each 3D model of furniture was thus scaled by the ratio of randomly chosen realistic furniture instance height to the original height of the 3D model. To create a realistic cluttered image, a predefined number of items were placed on the shelves of the furniture. Each item was placed in a random position on the shelf, in which it was not in the collision with other, already placed objects. These items are randomly selected 3D models of objects which are commonly placed on furniture, such as lamps, bottles, mugs, jars, and bowls or cuboids. Cuboid templates have a unit base and height of 0.6 m. Using the information about the bounding boxes of the storage volumes, the objects were placed at the bottoms of the bounding boxes which were aligned with the shelves, and their sizes were scaled by a random scaling factor drawn from a user-defined uniform distribution to fit within the storage volume. Since many objects found on the shelves in real scenarios are cuboids (books, boxes), the probability of cuboid objects being present on the shelves was set to 50%. Some examples of generated models of furniture with clutter on the shelves are shown in Figure 8(a). Some scenes include other furniture models--flower pots, lamps, sofas, and chairs--that are also commonly found in households but do not provide storage volumes.