## 4 Methodology
### Location and Boundary Estimation Using RGBD Point Cloud  
Localization and boundary estimation is performed after the presence of an object is detected by YOLO image-based object detection. Figure 4 presents a point cloud processing pipeline that extracts the location and boundary of the object detected by YOLO. This pipeline is used to determine the locations and boundaries when general objects, such as a sofa, are detected. The pipeline begins with obtaining a set of RGBD point cloud data from a depth camera (Step 1). This is either passed to the pipeline as an ROS PointCloud2 object, or it is passed as a file path to a point-cloud (.pcd) data file, which can be loaded into memory. The next two steps are performed separately and in parallel. A set of point clouds representing the floor is removed (Step 2). Using plane segmentation of the PCL library [47], planar surfaces are identified and organized in the order of their size, with the largest planes identified first. If one or more planes are found to be within an absolute tolerance of the vertical (\(z\)-axis) basis vector, they were considered floors. The tolerance factor was experimentally set after trial-and-error to be 0.97 for real-world experiments and 0.99 for simulations. This process has the effect of reshaping the point cloud, converting it from one point per pixel (640 \(\times\) 480 in these experiments) to a single dimensional \"flat\" data structure. Because of this re-shaping, the cropping of the YOLO-identified bounding box (Step 3) must occur in parallel with (rather than after) the floor plane segmentation. With this process, the full point cloud is cropped to what YOLO has identified as the object. This provides a starting point for further segmentation. In addition, it cannot occur before plane segmentation since a large amount of the cloud is removed during bounding box cropping, often leaving too few floor points to qualify as a plane. Thus, these two processes are performed in parallel, and then, a union of the points is generated in Step 4 of the pipeline. Statistical outliers, which are points that statistically likely do not belong (i.e., noisy points), are removed in Step 5. A process of grouping together points that likely belong to the same object is performed in Step 6. This step is called Euclidean Clustering of PCL library [48], and it generally results in several point-clouds, each of which is indicative of an object. YOLO generally draws a tight bounding box around the object it identifies and the point-cloud that is cropped according to this bounding (Step 3). Thus, the largest object after Euclidean clustering is chosen as the object of interest. The point cloud corresponding to this object is projected onto a 2D plane below the object, and a convex hull calculation is performed in Steps 7 and 8. This hull is used as the size and shape of the target object. A principal component analysis (PCA) and oriented bounding box were initially used to calculate the object boundary, but the often occluded view and unstructured partial point-cloud representation of objects would not result in consistently well-oriented bounding boxes. The convex hull procedure was more reliable in this study, especially for real-world cases. Figure 5 shows the results of point cloud isolation and convex polygons for furniture and doors detected from the pipeline. The PCL pipeline for the isolation and localization of doors is depicted in Figure 6. The first six steps for the PCL pipeline for door recognition are identical to the pipeline for general objects shown in Figure 4. The difference between doors and general objects occurs in Steps 7 and 8. Unlike non-door objects that used a 2D projection and convex hull calculation, door objects used principal components analysis (PCA) in order to determine the object major axes. PCA is an eigen-analysis-based technique [49]. With the principal (eigen) components known, an oriented bounding box (OBB) is placed around the object.