Efficient 3 D Scene Labeling Using Fields of Trees-Supplementary Material Anonymous

This supplementary material contains additional experiments to the main paper. Instead of the Cornell-RGBDDataset we evaluate the very same system of ours on the NYU Depth V2 dataset. The main findings are identical to the ones that have already been stated in the main paper. However, these additional experiments underline the statistical significance of the results and may also allow future comparison with similar systems. 1. Experiments on NYU Depth V2 Dataset In addition the the experiments based on the CornellRGBD-Dataset from [1] we also evaluate our algorithm using the NYU Depth V2 dataset from [4]. In previous works the NYU Depth V1 and the extended and revised V2 datasets have mostly been used to evaluate 2D semantic segmentation systems with additional depth information. The typical performance metric in these evaluations, for example in [3] and [2], is the mean diagonal of the per pixel confusion matrix. Along with the V2 dataset a system for grouping segments and finding the outlines of objects is presented in [4], and it is evaluated by computing the 2D region overlap of predicted and true objects. All of these metrics are basically computed on 2D images and hence incompatible to our setup, where we compute labels for 3D scene segments. To the best of our knowledge, our work is the first to use the NYU Depth dataset in the scenario of labeling full 3D scenes, and a direct comparison to the state-of-theart on this dataset is not possible. Nevertheless, the results illustrate the performance of our system. We consider a subset of 40 office and home office sequences acquired with a Kinect and the same subset of 12 object classes as in [4]. Example images are shown in Figures 1 and 3. We evaluate our pipeline of KinectFusion, oversegmentation, feature extraction and scene labeling using Decision Tree Fields and Regression Tree Fields as presented in the main paper. Macro P Macro R Micro P/R Training Inference DTF 41.54 23.52 37.73 45-60min 50-300sec RTF 53.75 41.75 60.30 20-24h 30-120sec Table 1. Experimental evaluation of macroand micro-averaged precision and recall and typical timings for fine-grained scene labeling using different classification methods. Macro P Macro R Micro P/R Training Inference DTF 54.35 38.67 58.29 45-60min 50-300sec RTF 67.36 48.39 81.09 4-6h 20-120sec Table 2. Experimental evaluation of precision and recall for estimating the coarse scene layout using DTFs and RTFs. 1.1. Fine-grained Scene Labeling We compute the macroand micro-averaged precision and recall scores of our scene labeling system and, as we assign a label to each segment in the scene without rejections, the micro-averaged precision and recall are identical. A 5-fold cross validation was performed and the average precision and recall scores are used as the final result. The performances achieved with our system are shown in Table 1. As in the experiments on the Cornell-RGBDDataset presented in the main paper, we observe that RTFs consistently achieve higher scores than DTFs, and their inference step is more efficient while training takes longer. Compared to our experiments on the CornellRGBD-Dataset the inference times are overall slightly higher and the performance lower, which underlines the qualitative observation that the scenes in the NYU Depth V2 dataset are generally larger and more diverse, and the overall scene labeling task hence more challenging. 1.2. Estimating Scene Layout As discussed in the main paper, we also use our system for identifying the coarse scene layout and finding the ground plane. We reduce the set of labels to the four classes {floor, wall, table, clutter}, do not use knowledge about the ground plane in the feature extraction, and apply the same scene labeling framework as before. Sample results are shown in Figure 3. As argued in the main paper, we use the precision and