SLIBO-Net: Floorplan Reconstruction via Slicing Box Representation with Local Geometry Regularization
Authors: Jheng-Wei Su
Kuei-Yu Tung
Chi-Han Peng
Peter Wonka
Hung-Kuo Chu
Venue: NeurIPS 2023 poster (2023)
Keywords: deep learning
layout reconstruction

Abstract: This paper focuses on improving the reconstruction of 2D floorplans from unstructured 3D point clouds. We identify opportunities for enhancement over the existing methods in three main areas: semantic quality, efficient representation, and local geometric details. To address these, we presents SLIBO-Net, an innovative approach to reconstructing 2D floorplans from unstructured 3D point clouds. We propose a novel transformer-based architecture that employs an efficient floorplan representation, providing improved room shape supervision and allowing for manageable token numbers. By incorporating geometric priors as a regularization mechanism and post-processing step, we enhance the capture of local geometric details. We also propose a scale-independent evaluation metric, correcting the discrepancy in error treatment between varying floorplan sizes. Our approach notably achieves a new state-of-the-art on the Structure3D dataset. The resultant floorplans exhibit enhanced semantic plausibility, substantially improving the overall quality and realism of the reconstructions. Our code and dataset are available online.

SLIBO-Net: Floorplan Reconstruction via Slicing Box Representation with Local Geometry Regularization
Jheng-Wee Wei Su', Kuei-Yu Tung', Chi-Han Peng", Peter Wonka Hung-Kuo Chu National Tsing Hua University Naational Yang Ming Chiao Tung University 3 *King Abdullah University of Science and Technology hkchuêcs, nthu. edu.tw
'Correspondence:
Abstract
This paper focuses on improving the reconstruction of 2D floorplans from unstruc- tured 3D point clouds. We identify opportunities for enhancement over the existing methods in three main areas: semantic quality, efficient representation, and local ge- ometric details. To address these, we presents SLIBO-Net, an innovative approach to reconstructing 2D floorplans from unstructured 3D point clouds. We propose a novel transformer- based architecture that employs an efficient floorplan represen- tation, providing improved room shape supervision and allowing for manageable token numbers. By incorporating geometric priors as a regularization mechanism and post-processing step, we enhance the capture of local geometric details. We also propose ocale-independent evaluation metric, correcting the discrepancy in error treatment between varying floorplan sizes. Our approach notably achieves a new oate-offtthe-rt on the Structured3D dataset. The resultant floorplans exhibit enhanced semantic plausibility, substantially improving the overall quality and realism of the reconstructions. Our code and dataset are available online"]
Introduction
Reconstructing 2D floorplans from unstructured 3D point clouds has received significant attention recent years as facilitates various applications scene understanding, robotics, VR/AR, etc. Prior works (24] & 9 22 [4] typically employ series of operations to address the problem at hand. The raw 3D point cloud is initially projected along the gravity axis (or z-axis) to generate 2D density map. This density map is then fed into deep neural network to either extract room masks 88 [2) or regress room corners 9 14] followed by an optimization process to extract room polygons. In contrast the two-stage framework, Yue et al. [24] introduced the RoomFormer model, an end-to-end trainable, transformer-type architecture that directly predicts room polygons using two-level queries. Although deep learning approaches have exhibited impressive performance in floorplan reconstruction, we have identified the following opportunities for improvement.
Semantic quality. The plausibility semantic quality of reconstructed floorplans heavily relies on the accuracy of estimated room corners and masks. Insufficient supervision of the room shape or fragile room representations result in artificial room shapes, demonstrated in Figure [). that are not plausible.
Efficient representation. One main challenge is designing representation that requires only a reasonable number of tokens the transformer architecture and can handle a variable number of rooms, each with variable number of corners. For example, the naive approach of predicting the layout room by room, corner by corner does not require too many tokens but too difficult to
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

(a) Artificiaa Corners
(b) Deasity Map
(c) Ground Truth
Figure 1: Limitations for current methods. (a) The lack of supervision on the overall room shape
can lead to the generation of artificial corners. (b) The density map of a large floorplan may have insufficient resolution to effectively capture local geometric details, such as wall thickness, which are clearly evident in the (c) ground truth layout.
train. To address this challenge, we employ a novel floorplan representation that slices the floorplan top-down manner (Figure 33. This representation offers two
into a of boxes by scanning key advantages. Firstly, it yields manageable number of slicing boxes, making it suitable for the transformer architecture as each slicing box can be represented by a query token. Secondly, these slicing boxes provide improved supervision of the room shape by utilizing Intersection over Union (IoU) loss [15]. Furthermore, this representation naturally leads to higher semantic quality, enhancing the overall accuracy and realism of the reconstructed floorplans.
Local details. We observed that deep learning approaches excel in predicting the overall room structure but struggle to capture local geometric details such as wall thickness due image resolution limitations (Figure b)). We incorporate prior knowledge about geometric details derived from the input 3D point cloud to address this limitation. This knowledge is encoded as cell complexes, which serve multiple purposes. It acts as a regularization mechanism during the training phase and is also utilized post-processing step enhance the accuracy of the reconstructed floorplans.
Evaluation. The metrics employed in previous studies are scale-dependent, resulting in differential treatment of errors between large and small floorplans. To illustrate this, consider starting with floorplan of specific size. Adding additional rooms to the floorplan alters its error measurement, leading to more lenient evaluation of incorrect corner predictions. This behavior counter-intuitive and primarily masks drawback of neural networks in scaling to larger inputs. Hence, we advocate for adopting a scale-independent metric that mitigates this issue.
In summary, we make the following contributions: 1) We propose a novel transformer architecture for floorplan reconstruction that only requires manageable number of tokens. 2) We propose integrating geometric priors as regularization during training and as a post-process to improve local geometric details. 3) We advocate the use of new metric that is scale independent and that better measures local details are correctly reconstructed. 4) We produce more semantically meaningful floorplans that do suffer from implausible geometric details.
Related Work
Optimization-based methods. Early work mostly assumed the task be based finding 3D planes from the point clouds using traditional techniques such as Hough transform [16] 2). plane sweeping/plane fitting 4
23
17
T2
18 21). and leveraging histograms of point cloud positions 77]
Higher-level structured objects such as rooms could then be assembled from the recovered planes. For larger buildings with many rooms, the optimization became increasingly difficult to solve. Researchers then developed graph-based data structures [10 5 [T] g, (22) to help organizing the reconstructed sub-models (e.g., corners, walls, objects, and rooms) and guide the optimization to follow certain semantic, topological constraints. In particular, Ikehata et al. [I]] used hierarchical tree graphs of reconstructed sub-models, such as objects, walls, and rooms, guide the 3D reconstructions. Bassier and Vergauwen 3 extracted highly-detailed BIM-compatible objects from point clouds by plane fitting and RANSAC. the same spirit of extracting more semantic and realistic 3D models, Ochmann et al. [9 proposed methods to extract parametric 3D building models from input point clouds. key advantage was that their reconstructed floorplans are volumetric i.e., having wall thickness. Our results shared the same idea. Finally, our cell complex-basee constraints was inspired

---

Ingra Point Cloud
(b) Segmanned Clusters
(c) Wall Clasters
(d) Wall Seguments
(c) Cell Complex
Figure 2: Encoding geometric priors into the cell complex. (a) Input 3D point cloud. (b) Segmentation based on normals and proximity. (c) Plane fitting and grouping. Aerial view of wall clusters with orientations. (d) Wall segment formation by collinear line grouping. (e) Cell complex from parametric lines. Non-axis aligned lines removed.
by [3]. in which they proposed optimization-baser approach to register multiple 3D scans of multi-story, multi-room buildings.
Deep learning methods. More recent methods began to leverage neural networks solve the floor- plan reconstruction problems. Earlier methods were hybrid methods that utilized both optimization and neural networks. Floor-SP g assembled room segments detected by Mask R-CNN M to form
full floorplans by sequentially solving shortest path problems. MonteFloor [2 later solved the room assembly problem by Monte-Carke Tree-Search (MCTS). FloorNet (14) took point cloud and RGB images as inputs, utilized a triple-branch neural network to generate pixel-wise predictions floorplan geometry and semantics, and then used integer programming generate the final floorplans in ccctor-graphics format.
Later, fully neural network beaserr-bbsed methods (which are end-to-end trainable) were being developed and delivered state-of-the-ar performances. HEAT [9] was method to reconstruct planar graphs with underlying geometric structures from 2D raster image inputs. It had two main applications building segmentation in aerial images and floorplan reconstruction from density maps distilled from point clouds. It utilized an end-to-end trainable edge classification neural network for the task. Recently, RoomFormer [24] proposed single-stage, end-to-end trainable neural network to directly predict floorplans from given point cloud. Their main innovation was to encode a floorplan as variable-size set polygons, which are variable-lenge sequences ordered vertices. This design enabled efficient handling by transformers. Our method is improvement upon it.
3 Methodology
3.1 Pre-processing
In the data preprocessing stage, we adopt a similar process as previous works 9 project the raw 3D point cloud along the gravity axis (or z-axis), resulting in 2D density map in the aerial view (or x-y plane). It is important note that while this density map provides adequate visual information for describing the overall structure (e.g., rooms and walls) of the floorplan, faces challenges in capturing geometric details like wall thickness due to the limitation of image resolutions.
22 to
Encoding geometric priors. We extract wall segments from 3D point cloud data geometric priors for floorplan reconstruction. We project these wall segments as parametric lines onto the x-y plane, forming "cell complex" representation. Figure Showss shows the steps computing cell complex from 3D point cloud. First, we segment the point cloud by normals and proximity (Figure (Figure[bb).. Next, we fit planes for each cluster and select those orthogonal to gravity as wall clusters (Figure 2c)) After that, we project points onto planes and group collinear line segments wall segments (Figure Ed). Finally, we convert wall segments into parametric lines and remove non-axis aligned lines to get the cell complex (Figure 2())). After obtaining the cell complex representation, we follow the strategy described in Section where we adjust our ground truth by rounding to the cell complex representation for training.

---

Figure 3: Floorplan representation. Illustrative examples highlighting floorplans (left) and their
respective slicing boxes representation (right).
3.2 Floorplan representation
One key challenge in work designing suitable data structure for floorplans. The repre- sentation should work with the transformer architecture and only require a reasonable number of tokens. One possible representation would be to represent each grid cell of the cell complex query token, but this would require large number of tokens, e.g., 2655 for our dataset. Another possible representation is encode floorplan predicting rooms as sequences of corners may lead to artificial room shapes as shown in Figure] aa). We propose to employ novel floorplan representation that slices the floorplan into set of boxes by scanning the floorplan along the vertical axis (or y-axis). We denote this box set as B = {b, ] where 6, ) and Nm is the number of boxes by in the m room. Examples of the slicing boxes representation are depicted in Figure 3 The major advantage of this representation lies in its ability to maintain manageable number of slicing boxes and query tokens ($ 50 in our dataset). However, each floorplan may consist of a varying number of rooms, with each room containing a different number of slicing boxes. This poses challenge in clustering the slicing boxes to their respective rooms. The unordered nature of the rooms, combined with the variable number of rooms and slicing boxes, makes it difficult to assign room IDs for the classification task. To tackle this challenge, we transform the original box classification problem into a 2D coordinate regression problem. Specifically, we employ another network to predict all the room centers from the density map. We compute the room center coordinate the bounding box center for each room in the floorplan, represented as C J.M where M denotes the number of rooms and c E [-1, 1] [-1, 1]. This allows us to cluster the slicing boxes based on the predicted room centers.
of room polygons. However, we observed that
network to predict the room center for each slicing box, and
3.3 Network architecture
Figure 4 fillustrates the SLIBO-Net architecture, comprising two primary components: the center transformer Teenter and the box transformer Thoz- The center transformer' Teenter is responsible for predicting the coordinates of all room centers based on the density map, while the box transformer Thor predicts both the slicing boxes and their corresponding room center coordinates. Subsequently, post- post-processing step is implemented to merge the estimated slicing boxes into a complete floorplan with distinct room layouts, utilizing the predicted room center points, Each component are elaborated in the next subsections.
Center transformer. First, feed 256 256 density map into ResNet-50 feature extractor and generate one 16 x 16 feature map. Following the encoder-decoder transformer architecture, we feed the extracted feature maps into the transformer encoder block. The output tokens of the transformer encoder block are then used for cross attention in the transformer decoder block. In order to query all the room centers from the density map, we use M learnable embeddings as the input tokens of the transformer decoder. The output tokens of the transformer decoder are further processed using two auxiliary MLP heads. Specifically, we use: (i) room center MLP head From that outputs the coordinates of rooms' bounding box center Croom {croom 1. Ji=1: and (ii) room prob MLP head Farob that outputs the probabilities proom = [proom M representing the probability of valid room center.
Box transformer. As for the box transformer Thaa we follow the same architecture of center transformer Teenter except the number of input tokens of the transformer decoder and the output MI.P heads. We query all the slicing boxes from the density map via N learnable embeddings. All the output tokens of the box transformer Too are further processed using two auxiliary MLP heads.

---

Transformer
Prsitional Facoding
Transtrmer
Encoder
25
Torddoreer Decoder
Fealure Backbone
toxr
Post-process
Transformer
Postional Unoooingg
256
Center
Density Map
mra Encodar
50
Tansformer
Decnder
Festure Dackbbone
Esxtrob
Figure 4: Network architecture. SLIBO-Net takes a 2D density map as imput, and outputs a real- scale floorplan as the final result. (a) The center transformer Teenter outputs set of room centers and the probabilities for them to be valid. (b) The box transformer Thoz predicts a set of slicing boxes, corresponding room centers, and their probabilities of being valid. Room centers and slicing boxes are further post- post-processed to form the final floorplan.
We use (i) box coord MLP head Froond that outputs six scalars at once. The first two scalars represent the corresponding room's 's center coordinates Chor = (c-oor N and the remain four scalars represent the upper-left and lower-right corners of slicing boxes = {b; J and (ii) box prob MLP head bour that outputs the probabilities Ipbox = {proz I boxes.
Ji=1 representing the probability of valid slicing
Bipartite matching. Note that our neural network would always predict 25 room centers (C room and 50 slicing boxes, which generally would be more than the ground-truth room centers C Troom and slicing boxes B, respectively. Thus, following DETR 6 conduct set-based bipartite matching between predictions and targets in the center transformer Teenter and the box transformer Thor- Since we use the same strategy both of the transformers, we explain this step using the center transformer Teenter as an example. Our goal is to optimize bipartite matching objective on a permutation function ((-) E E+ which maps M prediction indices to M targets:
o² = arg min Σ 1 (a()) (11d( grom
(a(()) + A2d2 (p: 20om
(1)
where (:--) represents L1 distance between predicted and target room center coordinates, d2() represents the focal loss between predicted and target room center probabilities, and ((-) is an indicator function. Finally, we use the Hungarian Matching algorithm calculate the minimal cost o² that maps M predicted room centers to M target room centers and N predicted slicing boxes to N target slicing boxes.
3.4 Post-processing
Here, we describe the post-processing steps in Figure Saureeru reconstruct the floorplan using the slicing boxes B and room centers Croom obtained in the previous step by the SLIBO-Net.
The first step in the post-processing phase is to cluster the slicing boxes into rooms. Given the predicted room centers Croom from the center transformer Toenter and the predicted slicing boxes Chor corresponded rooms' centers Cboz as the input, we can cluster the slicing boxes B into different rooms by assigning the Cboz to the nearest room centers Croom from the center transformer Teenter-
After clustering the boxes into different rooms, fill the gaps between the bottom and top of any two adjacent boxes in the same room with a box whose height is the vertical distance between them and whose width is the intersection length of them.
Finally, we union together the boxes for each room to form room area A,. We then overlay A, on the cell complex, which allows to calculate the intersection ratio for each room per cell. By assigning each cell to the room with the largest intersection ratio, we can obtain a room representation by tracing the boundary formed by the cell groups. For non-connected components within a single room, we choose the component with the largest area. cell remains empty if no room has more than 50% overlap.
5

---

post-process
dasiarin
Gup filin
Rounding
Fille
Result
Figure 5: Illustration of the post-process pust-processing. We use cell complex and SLIBO-Net's output get the floorplan. Same-color boxes are one room. (a) Crosses show slicing boxes' rooms' centers, dots show predicted room centers. (b) Box fills gap between boxes, red dashed box shows its size. (c) Colored boundaries are final result by boxes. Cells in green box belong to red room if they intersect more with red room and intersection ratio is over 50%.
3.5 Loss functions
We first elaborate the loss functions for the center transformer' Teenter based on the optimal permutation a" from the bipartite matching. For the center transformer Toenters we use the index set (i) I M) representing the best matching prediction indices. For the coordinate regression, use the smooth Lı loss LLsmoohh (:) which is defined as follow:
||X - 11 + 0.5(X - Y)? || - Y - 0.5, otherwise
if - Y|1 <1
Lymooth
(2)
Y
where is the prediction and Y is the label.
Room center loss calculates the smooth L1 loss with ground truth room center croom,
(i) Crenter (L (ctoom, (
(3)
Room prob loss calculates the focal loss for room probability of being valid:
rprob - (o
pr log pf r.om 1 ())=
(p: (p. (pfomm log(1 - promm)
(4)
Thus, the final loss Crtotal of the center transformer Toenter is defined as below:
Lytotal
Σ
A A + X((
(5)
where A and A2 are the hyperparameters for weighting the loss functions.
Next, describe the loss functions compute for the box transformer' Thnar based the optimal permutation o* from the bipartite matching. For the box transformer Thoas we use the index set (i: oi;a"()) (1) & N} representing the best matching prediction indices.
Box room center loss calculates smooth L, loss for each slicing box room center with ground truth room center boa.
(6) beeeter (1) (chor box Z
(6)
Box prob loss calculates the focal loss for each slicing box probability being valid:
bprob - -or (Os() (1-pphr bary" log,p, box Lio (1)> (p. bear log(1 p,
(7)
Box corner loss calculates the smooth L2 loss for all the corners of each slicing box:
coorner Wa(s) cl boomner (o) (beorner bg(t)
(8)
where boorner represented all 4 corners of a slicing box. We assume a conversion from the original upper-left and lower-right representation of the slicing boxes 6,. We set Wa(s to if and only the zeorner corner bo(9) equals any corner in the floorplan; otherwise, is 0.

---

Table Quantitative comparison of floorplan reconstruction on Structured3D best results are in bold font. means using cell complex to round the floorplan.
25 dataset. The
Roomt
Cornert
Anglet
Delaa
Method
Prec.
Rec.
F1
Prec.
Rec.
F1
Prec.
Rec.
F1
Prec.
Rec.
F1
Floor-SP
0.89
0.88
0.885
0.81
0.73
0.768
0.80
0.72
0.758
0.01
0.01
0.01
MonteFloor
22
0.956
0.944
0.950
0.885
0.772
0.825
0.863
0.754
0.805
0.022
0.018
0.02
HEAT 9
0.969
0,940
0.954
0.817
0.832
0.824
0.776
0.790
0.783
0.041
0,042
0.041
Roomerrmer
24
0.977
0.965
0.971
0.889
0.851
0,870
0.826
0.791
0.808
0.063
0.06
0.062
RoomFormer
24
0.991
0.976
0.983
0.851
0.826
0.838
0.840
0.816
0.828
0.011
0.01
0.01
0.991
0.978
0.984
0.889
0.854
0.878
0.812
0.844
0.011
0.009
0.01
Table 2: Quantitative comparison floorplan reconstruction on Structured3D dataset with real-scale measurements (0.1m). The best results in bold font.
Room
Comer00.1
Angle@0.11
Delta@0.1 $
Method
Prec.
Rec.
Prec.
Rec.
F1
Prec.
Rec.
F1
Prec.
Rec.
F1
HEAT 9
0.969
0.94
0.954
0.434
0.436
0.435
0.417
0.419
0.418
0.017
0.017
0.017
KnomFormer
24
0.977
0.965
0.971
0.637
0.612
0.624
0.608
0.585
0.596
0.029
0.027
0.028
Ours
0.991
0.978
0.984
0.652
0.602
0.626
0.648
0.598
0.622
0.004
0.004
0.004
Box IoU loss calculates the GloU loss
I Lygoo for all slicing boxs with ground truth slicing box 5,:
L (1) (b., oo())
(9)
biou
Thus, the final loss Lototal of the box transformer Thoz defined as below:
N
Litotal Aefeeenter + S + A Acoobrr + de A
((1)
(10)
where A(a,....)) are the hyperparameters for weighting the loss functions.
Experiments
4.1 Experimental settings
Dataset. We conducted experiments on large-scale indoor synthetic dataset called Struc-
tured3ID (25). Following the approach in 9 24], we divided the complete dataset into 2991 training samples, 250 validation samples, and 241 test samples, taking into account the ignore list specified in the HEAT g codebase. By employing the pre-processing method described in Section [3.]] we obtained density map of size 256 x 256 for each sample, with pixel values normalized to the range of [0, 1].
Baselines. We compare our method with the following competing methods: Floor-SP g scores
taken from (24). MonteFloor (22) scores taken from (24) HEAT g
scores based on their official pre-trained weights, RoomFormer 24 scores based on their official pre-trained weights.
Evaluation metrics. We follow the Room/Comeer/Angle evaluation method from MonteFloor and HEAT to measure the quality of floorplan reconstruction. However, the existing metrics suffer from a scale-variant nature as they calculate results based on a normalized 256 x 256 pixel space, without considering the actual size of each floorplan. Consequently, this leads to a more lenient evaluation for floorplans with more rooms larger sizes. Therefore, we propose the use of real-scale measurements, such as meters, as the threshold for Corner/Angle evaluation. For instance, Corner@0.1 denotes the measurement with threshold of 0.1 meter.
We found that the current corner evaluation solely focuses on the distance to the ground truth corner, disregarding the shape connectivity between corners. This approach can lead to scenarios where there are multiple "correct" corners but inadequate connectivity (Figure [()), resulting in a low Angle score. Hence, we additionally introduce the Delta metric that measures the global structure of the
7

---

Table Ablation study on loss function of box transformer. The best results are in bold font.
Roomt
Comert
Anglet
Method
Prec.
Rec.
F1
Prec.
Rec.
F1
Prec.
Rec.
F1
w/o Box IoU loss
0,981
0.964
0.972
0.756
0,774
0.765
0.740
0.758
0.749
w/o corner mask
0.988
0.975
0.981
0.882
0.821
0.850
0.871
0.811
0.84
Ours
0.991
0.978
0.984
0.889
0.821
0.854
0.878
0.812
0.844
Table 4: Ablation study on post-processing
The best results are in bold font.
Roomt
Cornrt
Anglet
Method
Prec.
Rec.
Prec.
Rec.
F1
Prec.
Rec.
F1
Baseline
0,991
0.978
0,984
0.889
0.821
0.854
0.878
0.812
0.844
cluster w/ GT room corner
0.994
0.987
0.99
0.894
0.831
0.861
0.884
0.822
0.852
w/o gap filling w/o cell complex rounding
0.991
0.977
0.984
0.887
0.815
0.849
0.875
0.805
0.839
0.97
0,956
0,963
0.745
0.745
0.745
0.72
0.72
0.72
w/ gap filling + rounding
0.991
0.978
0.984
0.889
0.821
0.854
0.878
0.812
0.844
floorplan by computing the discrepancy between Corner and Angle scores. Thus, lower Delta score indicates better shape for the resulting floorplans.
Implementation details. We implemented our model in PyTorch and trained our center transformer Teenter on NVIDIA V100s for days. The box transformer Thoz was trained on 8 NVIDIA V100s for 1.9 days. We the Adam optimizer with =009 and by=0.999. The learning rates of two transformers and the ResNet-500 are 2.5c-4, and le-5, and we train the center transformer for 5000 epochs, box transformer for 16000 epochs with batch size 128 and 123 for each GPU, We empirically
set Y A2 1 in Equation 5 À = 1.4g = 1, = 5., A = in Equation 10 7 2 in
Equation and M 25, N 50 Section 3.3
4.2 Floorplan reconstruction performance
Quantitative comparison. As shown in Table 1. our SLIBO-Net outperforms all the baselines the overall F1 score. Compared to RoomFormer, which has the best Fl score among all baselines, our model improves the Fl score of Room by 1.3% and Angle by 3.6%, while maintaining a comparable precision of Corner. Since RoomFormer fails to generate good room shapes (i.e., low Angle score), we round its results by our cell complex, denoted as RoomFormer', Compared to RoomFormer', our model achieves better F1 scores all metrics. It is worth highlighting that our model exhibits substantial performance boost of 5.2% in the Delta F1 score when compared to RoomFormer. This advancement indicates our model's s ability to generate more plausible and realistic floorplans.
Real-scale evaluation. As shown in Table 2 our model attains the highest scores across most of the metrics, including Room, Corner, Angle, and Delta, with particular emphasis the Delta scores. This improvement can be attributed to the incorporation of cell complex regularization, which effectively enhances our model's capabilities under more stringent evaluation conditions.
Qualitative comparison. The qualitative results are shown in Figure 6 Compared HEAT, our model can reconstruct the correct floorplan structure and preserve the wall thickness while HEAT fails to reconstruct the walls. Compared to RoomFormer, our model preserves all structural details without any self-intersection, whereas RoomFormer generates some self-intersections that are semantically implausible. Please refer the supplementary for additional comparisons.
4.3 Ablation Studies
The effectiveness of loss functions. In this experiment, we evaluate the necessity of two loss functions. First, we remove the Box loU loss (Fquation 9 in our total loss function (Fquation T0) of the box transformer. Second, we eliminate the corner mask by setting Wa(() to I in Equation

---

Table Quantitative comparison and computational efficiency by varying number of tokens in box transformer. The best results are in bold font.
Time (sec./ppoch)
Roomt
Cornert
Anglet
# tokens
Training
Inference
Prec.
Rec.
F1
Prec.
Rec.
F1
Prec.
Rec.
F1
38.36
6.923
0.991
0.978
0.984
0.889
0.821
0.854
0.878
0.812
0.844
100
42.84
7.243
0,974
0.959
0.967
0.846
0.808
0.827
0.831
0.792
0.811
200
50.92
7.620
0,966
0,950
0,958
0.827
0.782
0.805
0.805
0.775
0.790
300
59.97
7.850
0.962
0.950
0.956
0.812
0.765
0.789
0.793
0.769
0.781
Table 6: Quantitative comparision on the mean-shift clustering method. Rows 4 to 7 present mean-shift clustering results using various bandwidth parameters. The best results are in bold font.
Roomt
Cornrrt
Anglet
Method
Prec.
Rec.
Fl
Prec.
Rec.
F1
Prec.
Rec.
F1
Ours
0.991
0.978
0.984
0.889
0.821
0.854
0.878
0.812
0.844
Bandwidth=1
0.979
0.986
0.982
0.881
0.821
0.850
0.870
0.811
0.840
Bandwidth=1.2
0,982
0.983
0.983
0.883
0,821
0.851
0.872
0.811
0.841
Bandwidth=1125
0,986
0.982
0.984
0.887
0.823
0.854
0.876
0.813
0.844
Bandwidth=1.3
0.986
0.979
0.982
0.885
0.819
0.851
0.874
0.810
0.841
As shown in Table we obtain the best performance with the Box IoU loss and the corner mask. The Box loU loss significantly boosts our performance, which agrees with previous work (20). Our slicing box representation is compatible with this Box IoU loss, unlike the corner representation. We attribute this Box IoU loss as one of the key factor for surpassing other methods in score.
The necessity of each step in post processing. In this experiment, we examine two aspects of our floorplan reconstruction method. First, we investigate how the aceuracy of room center estimation affects the performance. Second, evaluate the impact of gap filling and cell complex rounding in our post-proce post-processing step in Section The upper block of Table shows that using the ground truth room center for clustering the slicing boxes slightly improves the performance. The lower block of Table 4 shows that applying gap filling and cell complex rounding achieves the best performance. We observe that cell complex rounding significantly enhances the performance, indicating that cell complex regularization is a crucial component in our post processing.
The impact of the number of tokens. In this experiment, we examine how the number of tokens in the box transformer influences the computational efficiency and the final accuracy, as shown in Table [ We observe that both the training time and inference time increase significantly as we increase the number of tokens. On the other hand, the accuracy also drops as we use more tokens. The main reason is that more tokens require more training time and introduce more dummy tokens, which makes harder to reach the same accuracy within the same training time. These results confirm the importance of using reasonable number of tokens to represent the floorplan, as discussed in Section I
The effectiveness of center transformer. In this experiment, we evaluate the necessity of our center transformer. We compared our results with another method that directly employs mean-shift clustering of the room centers estimated by the box transformer. As shown in Table the conventional clustering approach experiences numerical fluctuations because of the varying bandwidth parameter. Overall, our method surpasses the performance of the mean-shift clustering approach.
Conclusions
propose SLIBO-Net, a transformer-based method for floorplan reconstruction from 3D point clouds. Our method uses a novel floorplan representation that slices the floorplan into boxes, which reduces the tokens and improves the room shape supervision. We further use cell complex derived from the 3D point cloud as regularization and post-procest to enhance the local details. We advocate

---

3D point cloud
Density map
HEAT
Roomformer
24
Ours
Ground truth
Figure 6: Qualitative evaluations on Structured3D, Visual comparisons with competing methods.
Results
Ground Truth
Figure 7: Failure cases. Left: our method struggles in reconstructing non-Manhatte layouts. Right: ground truth. One possible approach would be to extend the cell complex and post-pine processing to include non Mon-MMahattan lines.
scale-independent metric that measures the local details better. Our method produces more meaningful and realistic floorplans and achieves state-of oftee-of-the-ar performance various evaluation metrics.
Limitations. Our method is limited by its inability to handle non-Manhattan layouts (see Fig- ure 7). Moreover, our network relies on a two- wwo-branch framework that constrains the reconstruction performance by the room center transformer.
Future work. Our method has hyper- parameters our loss functions that have been thoroughly tuned. We intend to optimize these parameters to improve the performance. Additionally, we would like integrate the cell complex into the box representation to avoid the need for the rounding step. However, this would require major change in the data format for our current method, so needs further investigation. Finally, we to make the post-processing more capable of handling non-Manhattan layouts.
This work significantly improves the efficiency and accuracy of floorplan reconstruction, which can benefit various applications in scene understanding, robotics, VR/AR, and construction industry. However, it also poses some risks, the reconstruction is biased through training data and other factors.
Broader impact.
Acknowledgments and Disclosure of Funding
The project was funded part by the National Science and Technology Council of Taiwan (110-2221- E-007-061-MY3,
We gratefully acknowledge Taiwan Web Service Corporation (TWSC) for their generous contribution cloud GPU computing resources to support research.
10

---

References
[1] Waleed Abdulla. Mask r-can for object detection and instance segmentation keras and tensorflow. https://github. http:///w.....///tttttttttttt///
RCNN, 2017.
[2] Antonio Adan and Daniel Huber. 3d reconstruction of interior wall surfaces under occlusion and clutter. In 2011 International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission, pages 275-281, 201 1. doi: 10. 109/3D1MPVT2201 1.42.
[3] Maarten Bassier and Maarten Vergauwen. Unsupervised reconstruction of building information modeling wall objects from point cloud data. Automation in Construction, 120: 103338, 2020. ISSN 0926-5805. doi: https://doi.org/// http:///..o.oo11........ 1016/0166...... awtp///.ttoonn2220 2020. 103338.
[4] Angela Budroni and Jan Bochm. Automated 3d reconstruction of interiors from point clouds. International Journal of Architectural Computing, 8:55-73 73,7,, 01 2010. doi: 10. 10.1260/1478 07260/1478-0771... 0.....7......55..
Ricardo Cabral and Yasutaka Furukawa. Piecewise planar and compact floorplan reconstruction from images. 2014 IEEE Conference on Computer Vision and Pattern Recognition, pages 628-635, 2014. doi: 10.1 10.1109/CVPR.200 2014.5666.
[6] Nicolas Carion, Francisco Massa, Gabriel Synnaeve. Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers, In ECCV. pages 213-229. Springer, 2020.
[7] Kunal Chelani, Chitturi Sidhartha, and Venu Madhav Govindu. Towards automated floorplan generation. In Proceedings of the 11th Indian Conference on Computer Vision, Graphics and Image Processing, ICVGIP 2018, New York, NY, USA, 2020. Association for Computing Machinery. ISBN 0781450366151 10. 00.144//22333333292444
doi:
URL hwt.:htttps//tc
org/10..11000............111111. 2144/329335333333 32219333333333300 3293430
[8] Jiacheng Chen, Chen Liu, Jiaye Wu, and Yasutaka Furukawa. Floor-sp: Inverse cad for floorplans by sequential room-wise shortest path. In The IEEE International Conference on Computer Vision (ICCV), 2019.
[9] Jiacheng Chen, Yiming Qian, and Yasutaka Furukawa. Heat: Holistic edge attention transformer for structured reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3866-3875, June 2022.
[10] Yasutaka Furukawa. Brian Curless, Steven M. Seitz, and Richard Szeliski. Reconstructing building interiors from images. 2009 IEEE 12th International Conference on Computer Vision, pages 80-87, 2009, doi: 10. (009/CCCV22009555544555
[11] S. Ikehata, H. Yang, and Y. Furukawa. Structured indoor modeling. In 2015 IEEE International Conference on Computer Vision (ICCV), pages 1323-1331, Los Alamitos, CA, USA, dec 2015. IEFE Computer Society. doi: 10.1 10.100/CCV..015..10
[12] Minglei Li, Peter Wonka, and Liangliang Nan. Manhattan-world urban reconstruction from point clouds. In ECCV, 2016.
[13] Gahyeon Lim and Nakju Doh. Automatic reconstruction of multi-level indoor spaces from point cloud and trajectory. Sensors (Basel), 2021.
[14] Chen Liu, Jiaye Wu, and Yasutaka Furukawa. Flooret: A unified framework for floorplan reconstruction from 3d scans. Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, and Yair Weiss, editors, Computer Vision - -Esv 2018, pages 203-219, Cham, 2018. Springer International Publishing. ISBN 978-3-030-01231-1.
[15] Jiachen Liu. Yuan Xue, Jose Duarte, Krishnendra Shekhawat, Zihan Zhou, and Xiaolei Huang. End-to-end graph-constrained vectorized floorplan generation with panoptic refinement. In ECCV, 2022.
[16] Josep Lladós, Jaime López- Krahe, and Enric Martí. A system understand hand drawn floor plans using subgraph isomorphism and hough transform. Mach. Vision Appl., 10(3):150 158. aug 1997. ISSN 0932-8092. doi: 10. 1010000/00010000000000
[17] Aron Monszpart, Nicolas Mellado, Gabriel J. Brostow, and Niloy Mitra. Rapter: Rebuilding man-made scenes with regular arrangements of planes. ACM Trans. Graph., 34(4), jul 2015. ISSN 0730-0301. doi:
10. 10.1145/2766995. URL https: tww./ooooooooooooooo.ooo http::///ooooooo//// 1ttp//.0.115//2769999993
[18] Liangliang Nan and Peter Wonka. Polyfit: Polygonal surface reconstruction from point clouds. In Proceedings of the IEEE International Conference Computer Vision, pages 2353-236 2017.
11

---

[19] Sebastian Ochmann, Richard Vock, Raoul Wessel, and Reinhard Klein. Automatic reconstruction of parametric building models from indoor point clouds. Computers & Graphics. 54:94-103, 2016. ISSN 0097-8493, doi: https://doi. wwtp::/i..g/100.......00 wwt. ..000 Special Issue on CADIGraphics 2015.
[20] Hamid Rezatofighi, Nathan Tsoi, JunYoung Gwak, Amir Sadeghian, lan Reid, and Silvio Savarese. Generalized intersection over union: A metric and loss for bounding box regression. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 658-666, 2019.
[21] Victor Sanchez and Avideh Zakhor. Planar 3d modeling building interiors from point cloud data. In 2012 19th IEEE International Conference on Image Processing, pages 1777-1780, 2012. doi: 10.1109/ (CIP.2012.646722)
[22] Sinisa Stekovic, Mahdi Rad, Friedrich Fraundorfer, and Vincent Lepetit. MonteFloor: Extending MCTS for Reconstructing Accurate Large-Scale Floor Plans. In International Conference on Computer Vision (ICCV 2021), Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV, pages 10034-16044 Online, Canada, October 2021. URL https::t/p/hh //all www.sscciccaaaaeeoooo hatpe/hhllll322222
[23] Xuchan Xiong, Antonio Adan, Burcu Akinci, and Daniel Huber. Automatic creation of semantically rich 3d building models from laser scanner data. Automation Construction, 31:325-337, 2013. ISSN 0926-5805. doi: https://doi.org/10... org/10. wttp://o//1101011.ottaoooooo 2012 10.006. TRRLhttss://www..n http://wwwwwccccdiiiicccocc
com/ scweececeeeeeeeeeeeeeo/cceoccces att.icceeeepp.0/0225555101100000
[24] Yuanwen Yue, Theodora Kontogianni, Konrad Schindler, Francis Engelmann. Connecting the Dots: Floorplan Reconstruction Using Two-Level Queries. In IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
[25] Jia Zheng, Junfei Zhang. Jing Li, Rui Tang, Shenghua Gao, and Zihan Zhou. Structured3d: A large photo- realistic dataset for structured 3d modeling. In Proceedings of The Luropean Conference Computer Vision (ECCV), 2020.
12


Reviews:
[{"review_id": "RThGgBVEi0", "reviewer": "Reviewer_mWNJ", "summary": "The authors propose a new method for floorplan reconstruction from point clouds. Specifically, the method formulates the problem from predicting polygons for each room to predicting sliced boxes and their corresponding room centers by imposing the Manhattan grid assumption on the floorplans. The authors also use the prior information--cell complex, derived by extending the extracted wall segments from the input point clouds, to improve the results. The qualitative and quantitative results demonstrate that the proposed method outperforms the prior work by a significant margin.", "questions": "1. It is unclear how the cell complex is used for regularization during training. None of the branches have a regularization term, and cell complex seems to be used only in post-processing.\n\n2. The seems to be some redundancy in representing the box losses: why is computing loss on four corners required? Aren't top-left and bottom-right corners enough (i.e., xmin, ymin, xmax, ymax)?\n\n3. It would be interesting to know how important the center transformer is. For example, the room assignment can be done using clustering algorithms on the predicted room centers. \n\n4. How are the extracted wall segments for the cell complex aligned with the ground truth room shapes? Do the misalignments affect the accuracy?\n\n5. The visual information in 3D is potentially helpful. Several works have reconstructed room shapes from panorama views, such as [1]. How are these approaches compared to the proposed method? For example, can one reconstruct the plan room-by-room instead of the whole floorplan?\n\n[1] PSMNet: Position-aware Stereo Merging Network for Room Layout Estimation by Wang et al.", "limitations": "Yes, the authors have addressed the limitations.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The proposed method effectively improves the results using the Manhattan assumption and the prior knowledge, cell complex, obtained from the given point clouds. It is also a clever approach to use predicted room centers to cluster the boxes instead of predicting complex, non-rectangular polygons. \n\n2. The figures are clear and helpful in understanding the methods and details.\n\n3. The results outperform prior works by a significant margin, especially in the qualitative results' geometric details, such as wall gaps. ", "weaknesses": "1. As the authors mentioned, the main limitation of this method is the Manhattan formulation. \n\n2. Extending the local planes across the whole space might be unnecessary to form the cell complex. The extended planes will intersect at many empty locations. For example, the second line from the bottom in Figure 2(e) creates extra cell units on the bottom right, but it is unrelated to the room on the bottom right. Aren't the extracted wall segments already useful for post-processing? Or have the author tried using local grids (e.g., a plane is extended until it intersects with another plane) instead of global ones for the cell complex?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W2**: *Extending the local planes across the whole space might be unnecessary to form the cell complex. The extended planes will intersect at many empty locations.* \n\n**Reply**: Extending local planes across the whole space has several advantages. Firstly, such unlimited expansion can propagate global geometric knowledge across different areas of the scene, capturing details not present in the initial data. Secondly, it can bridge the gaps in areas sparse in structural details, enhancing the accuracy of floorplan reconstructions (refer to Figure 1 in the attached pdf). Thirdly, this method aids in reconstructing extensive floorplans and multiple buildings by leveraging shared geometric insights. In addition, implementing limited local plane extension is not straightfoward due to the complex issue of plane intersection priorities. \n\n**Q1**: *It is unclear how the cell complex is used for regularization during training. None of the branches have a regularization term, and cell complex seems to be used only in post-processing.*\n\n**Reply**: The cell complex serves as a key modifier of the ground truth used in training and thus directly influences the objective function. We adopt the strategy outlined in Section 3.4 of the main paper, adjusting our ground truth by rounding to the cell complex prior to training. Consequently, this actively prompts our predictions to align with the cell complex. This integral use of the cell complex is pivotal in enhancing both the accuracy and robustness of our floorplan reconstruction technique.\n\n**Q2**: *The seems to be some redundancy in representing the box losses: why is computing loss on four corners required? Aren't top-left and bottom-right corners enough (i.e., xmin, ymin, xmax, ymax)?*\n\n**Reply**: Rather than calculating the box corner loss based on all four corners, we determine our box corner loss solely on those corners of the box that correspond to the ground truth layout's corners. This correspondence is denoted by the $w_{\\sigma (i)}$ mask term in equation 8 of the main paper. Such an approach guarantees that our box corner loss exclusively fine-tunes the layout corners.\n\n**Q3**: *It would be interesting to know how important the center transformer is. For example, the room assignment can be done using clustering algorithms on the predicted room centers.*\n\n**Reply**: To highlight the importance of the center transformer, we compared our results with another method that directly employs mean-shift clustering to the room centers estimated by the box transformer. As shown in Table 1, the conventional clustering approach experiences numerical fluctuations because of the varying bandwidth parameter. Overall, our method surpasses the performance of the mean-shift clustering approach.\n\n**Table 1** : **Quantitative comparision on the mean-shift clustering method.** Rows 4 to 7 present mean-shift clustering results using various bandwidth parameters. The best results are emphasized in bold.\n\n|Method||Room$\\uparrow$|||Corner$\\uparrow$|||Angle$\\uparrow$||\n|-|-|-|-|-|-|-|-|-|-|\n||Prec.|Rec.|F1|Prec.|Rec.|F1|Prec.|Rec.|F1|\n|Our method|**0.991**|0.978|**0.984**|**0.889**|0.821|**0.854**|**0.878**|0.812|**0.844**|\n|Bandwidth=1|0.979|**0.986**|0.982|0.881|0.821|0.850|0.870|0.811|0.840|\n|Bandwidth=1.2|0.982|0.983|0.983|0.883|0.821|0.851|0.872|0.811|0.841|\n|Bandwidth=1.25|0.986|0.982|**0.984**|0.887|**0.823**|**0.854**|0.876|**0.813**|**0.844**|\n|Bandwidth=1.3|0.986|0.979|0.982|0.885|0.819|0.851|0.874|0.810|0.841|\n\n**Q4**: *How are the extracted wall segments for the cell complex aligned with the ground truth room shapes? Do the misalignments affect the accuracy?*\n\n**Reply**: In many cases, the extracted wall segments align with the ground truth. Nonetheless, some discrepancies between these segments and the ground truth do exist, influencing the \"rounded ground truth\" discussed in the response to Q1. The differences between the ground truth and its rounded counterpart is reported in Table 2. Notably, the accuracy of the rounded ground truth surpasses that of all existing methods. As a result, our predictions, which learns from the rounded ground truth, still outperforms other approaches.\n\n**Table 2** : **Quantitative comparison on the rounded ground truth.** The best results are emphasized in bold.\n\n|Method||Room$\\uparrow$|||Corner$\\uparrow$|||Angle$\\uparrow$|||Delta$\\downarrow$||\n|-|-|-|-|-|-|-|-|-|-|-|-|-|\n||Prec.|Rec.|F1|Prec.|Rec.|F1|Prec.|Rec.|F1|Prec.|Rec.|F1|\n|RoomFormer|0.977|0.965|0.971|0.889|0.851|0.870|0.826|0.791|0.808|0.063|0.06|0.062|\n|Ours|0.991|0.978|0.984|0.889|0.821|0.854|0.878|0.812|0.844|0.011|0.009|0.01|\n|Rounded GT|**0.998**|**0.993**|**0.995**|**0.965**|**0.905**|**0.934**|**0.957**|**0.899**|**0.927**|**0.008**|**0.006**|**0.007**|\n\n**Q5**: *The visual information in 3D is potentially helpful. Several works have reconstructed room shapes from panorama views, such as [1]. How are these approaches compared to the proposed method? For example, can one reconstruct the plan room-by-room instead of the whole floorplan?*\n\n**Reply**: Reconstruction methods based on panorama images are generally investigated in a separate branch of work, e.g. [1, 2]. One of the main challenges of these methods is to register different rooms into a coherent floorplan. The registration process is often more difficult than the reconstruction of individual rooms, due to the lack of reliable features and correspondences across rooms. Therefore, the current bottleneck of registration limits the applicability of these methods to constitute full floorplans.\n\n[1] PSMNet: Position-aware Stereo Merging Network for Room Layout Estimation by Wang et al.\\\n[2] GPR-Net: Multi-View Layout Estimation via a Geometry-Aware Panorama Registration Network by Su et al."}, {"review_id": "E25su9tOTq", "reviewer": "Reviewer_EcBt", "summary": "This paper focuses on 2D floorplan reconstruction from 3D point clouds. The authors represent 2D floorplans as sliced ​​boxes and apply a transformer network to predict box characteristics, as well as post-processing steps to build the final floorplan. The authors conduct experiments on Structure3D and show better performance in both quantitative and qualitative terms compared to state-of-the-art methods.", "questions": "Please see *Weaknesses", "limitations": "I found the proposed method a bit clumsy, with two parallel transformers instead of a single one. Inference time was not evaluated compared to other methods, so it is not known how the preprocessing and postprocessing steps affect the latency of this method.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- I like the idea of ​​introducing geometric priors (slicing boxes) into the design of neural network architectures. In my opinion, a combination of learning methods and pure geometric reasoning should be a better way to solve 3D related tasks. I'm happy to see better performance compared to other state-of-the-art method.\n\n- In addition to the evaluation on synthetic Structure3D, the authors also introduce a novel real-world dataset (Gibsonlayout) in the supplement to further demonstrate the effectiveness of the proposed method compared to other state-of-the-art methods.\n\n- The proposed method is technically sound. The authors also conduct ablation studies to verify the effectiveness of the design choices.", "weaknesses": "- I still don't understand why two parallel transformers are used instead of a single transformer for room center and slice box prediction. Perhaps it is better to show quantitatively that joint prediction is not a viable solution. Otherwise I find the proposed method a bit clumsy.\n\n- It also better to conduct evaluation of inference time compared to other methods. I'm wondering how long the pre- and post-processing will take and the cost of running two transformers. If it takes significantly longer than others, it may not be desirable given the amount of improvement compared to other methods on Structure3D.\n\n- Maybe an unnecessary requirement, it's also better to experiment on SceneCAD as other methods too. Although I know the authors propose a new dataset in the supplement, it should stand alone in the main text. Given the many heuristics introduced in pre- and post-processing, I want to make sure this is not tuned specifically for Structure3D and the proposed dataset.\n\n- Minor issue, page 7, line 211, missing citation after Structure3D.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1**: *I still don't understand why two parallel transformers are used instead of a single transformer for room center and slice box prediction. Perhaps it is better to show quantitatively that joint prediction is not a viable solution. Otherwise I find the proposed method a bit clumsy.*\n\n**Reply**: Our method employs two transformers: the center transformer and the box transformer. The center transformer predicts a room corner for each input query, while the box transformer forecasts the boxes and their associated room centers. To consolidate these two transformers into a single branch, one could segment the input query into two subsets, directing each to a distinct transformer. The first subset would yield results akin to the center transformer, and the second subset to the box transformer. Nonetheless, this approach requires the transformer to handle diverse queries concurrently (room-wise tokens versus box-wise tokens), leading to inferior results compared to the original dual-transformer design, as evidenced in Table 3.\n\n**Table 3** : **Quantitative comparison on the single-transformer architecture.** The best results are emphasized in bold.\n\n|Method||Room$\\uparrow$|||Corner$\\uparrow$|||Angle$\\uparrow$||\n|-|-|-|-|-|-|-|-|-|-|\n||Prec.|Rec.|F1|Prec.|Rec.|F1|Prec.|Rec.|F1|\n|Our method|**0.991**|**0.978**|**0.984**|**0.889**|**0.821**|**0.854**|**0.878**|**0.812**|**0.844**|\n|Single transformer|0.970|0.966|0.968|0.853|0.796|0.823|0.840|0.784|0.811|\n\nAnother possibility to build a single-transformer architecture is to directly employ a clustering algorithm on the room centers deduced from the box transformer. However, as discussed in the response to **Q3** of reviewer **mWNJ**, this alternative yields slightly inferior outcomes. As a result, we adopted the two-stage approach as our primary method.\n\n**W2**: *It also better to conduct evaluation of inference time compared to other methods. I'm wondering how long the pre- and post-processing will take and the cost of running two transformers. If it takes significantly longer than others, it may not be desirable given the amount of improvement compared to other methods on Structure3D.*\n\n**Reply**: We report the pre- and post-processing runtime in Table 4 and 5. In general, the computation time of all methods including ours is still very short compared to the time it takes to scan a complete floorplan with its corresponding processing. We conduct our experiments on a machine with an Intel i7-9700K @ 3.60GHz CPU, NVIDIA V100 GPU and 16 GB RAM. We also show the inference time for the two transformers in Table 4. Our method uses native transformers instead of deformable DETR [2], which makes our inference time slightly slower than other fully-neural methods, but still comparable. Overall, we believe that our method is worth the extra time, as it achieves significant improvement over other methods in terms of plausibility and accuracy. \nAs we mainly focused on quality, we did not spend any time optimizing the pre-processing time as it is quite short overall.\n\n**Table 4** : **Inference time analysis on Structured3D dataset.** The unit of the time is Second.\n\n|Method|Pre-processing|Inference|Post-processing|Total|\n|-|-|-|-|-|\n|HEAT|1.87|0.11|-|1.98|\n|RoomFormer|0.57|0.01|-|0.59|\n|Ours|7.55|0.06|0.11|7.72|\n\n**Table 5** : **Timing analysis on pre-processing.** The unit of the time is Second.\n\n|Steps|Density Map|Clustering|Plane Fitting|Line Grouping|Non-axis-aligned Removal|Cell Complex|Total|\n|-|-|-|-|-|-|-|-|\n|Timing|0.57|3.21|0.36|3.22|0.08|0.11|7.55|\n    \n**W3**: *Maybe an unnecessary requirement, it's also better to experiment on SceneCAD as other methods too. Although I know the authors propose a new dataset in the supplement, it should stand alone in the main text. Given the many heuristics introduced in pre- and post-processing, I want to make sure this is not tuned specifically for Structure3D and the proposed dataset.*\n\n**Reply**: We decided not to conduct experiments on the SceneCAD dataset due to the following 3 reasons. 1) Poor layout annotation quality. As depicted in Figure 2 of the attached pdf, a substantial number of layouts in the SceneCAD dataset are flawed and do not align with the density maps. These inconsistencies render the dataset noisy and less trustworthy. 2) Not suitable for the problem. SceneCAD offers only one room per floorplan and not complete floorplans. 3) Small size. SceneCAD only has 127 rooms. Gibsonlayout has 7,563 rooms with superior layout annotations. These annotations were derived from the panoramas in the Gibson Environment dataset, details of which are available in the supplementary material.\n\n**W4**: *Minor issue, page 7, line 211, missing citation after Structure3D.*\n\n**Reply**: Thanks for the reminder. We will add the missing citation in the revision.\n\n[1] Semantic Instance Segmentation with a Discriminative Loss Function by Brabandere et al.\\\n[2] Deformable DETR: Deformable Transformers for End-to-End Object Detection by Zhu et al."}, {"review_id": "OWofZxNEwN", "reviewer": "Reviewer_84jG", "summary": "The paper introduces SLIBO-Net, a novel approach for reconstructing 2D floorplans from unstructured 3D point clouds. By employing a transformer-based architecture and an efficient floorplan representation, the method improves room shape supervision and manages token numbers effectively. The proposed approach achieves state-of-the-art results on the Structure3D dataset, generating floorplans with enhanced semantic plausibility and realistic reconstructions.", "questions": "The description, design, and usage of the cell complex as presented in the paper may not be immediately intuitive or clear. It requires multiple readings to fully grasp how it is utilized in the final stages of the proposed method. To improve the clarity of this aspect, it would be advantageous for the authors to provide more explicit explanations and visual aids. For instance, incorporating a diagram that illustrates how and when the cell complex is employed would greatly enhance understanding and ensure a smoother comprehension of its role in the overall pipeline.\n\nThe authors' motivation in the proposed approach is heavily influenced by the transformer-based representation and the desire for a reasonable number of tokens. However, the impact of the selected number of tokens on the final performance, as well as the size and speed of the network, remains unclear. Conducting an ablation study that explores the relationship between the number of tokens and these factors would greatly enhance the effectiveness of the work. By systematically analyzing the effects of varying token counts, the authors can provide valuable insights into the trade-offs between representation size, computational efficiency, and model performance. Including such an ablation study would further strengthen the paper and provide a more comprehensive understanding of the proposed approach.", "limitations": "The authors have done good job discussing the most obvious limitations. ", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Technical Contribution: The authors introduce a novel approach for reconstructing 2D floorplans, tackling the challenges associated with semantic quality, efficient representation, and local geometric details. The proposed solutions are well-designed, showcasing a clear advancement over existing methods.\n\nPresentation: The paper is well-structured and clearly presents the motivations, methodology, and experimental results. The technical concepts and terminologies are adequately defined and explained.\n\nEvaluation: The authors conducted a thorough experimental evaluation, demonstrating the effectiveness of their proposed method. The comparisons with existing approaches and the state-of-the-art highlight the benefits of the proposed approach. The proposed scale-independent evaluation metric addresses a common limitation and ensures a fair assessment of performance across different floorplan sizes.\n\nContributions: The paper provides several notable contributions, including a novel transformer architecture for floorplan reconstruction, the integration of geometric priors for improved local geometric details, the adoption of a scale-independent evaluation metric, and the generation of semantically meaningful floorplans without self-intersections. These contributions offer valuable insights for future research.\n\nFurthermore, the supplementary section of the paper introduces a valuable contribution in the form of a new labeled dataset called Gibsonlayout, which augments the Gibson dataset with annotated 2D floor-plans. The potential release of this dataset to the research community would be highly beneficial for advancing the field and promoting further research on the problem at hand. The availability of such a dataset would facilitate comparative evaluations, enable benchmarking, and encourage the development of more robust and accurate solutions.", "weaknesses": "The authors have effectively addressed a range of limitations in their paper, including the difficulties associated with handling non-Manhattan scenes and the seemingly arbitrary choice of the many hyperparameters. However, it is notable that the significant improvement in performance can be largely attributed to the Box IoU function, as acknowledged by the authors themselves. From a practical standpoint, this suggests that the additional technical contributions and associated complexity may have limited value for real-world applications.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1**: *It is notable that the significant improvement in performance can be largely attributed to the Box IoU function, as acknowledged by the authors themselves. From a practical standpoint, this suggests that the additional technical contributions and associated complexity may have limited value for real-world applications.*\n\n**Reply**: A possible direction for future work is to extend our box IoU function to a quad IoU function or a constrained quad IoU function, which could handle more challenging real-world non-Manhattan cases. This would allow our method to deal with more complex and irregular wall shapes, and improve the accuracy of our floorplan reconstruction. However, we would still consider our current solution very applicable to real-world applications.\n\n**Q1**: *The description, design, and usage of the cell complex as presented in the paper may not be immediately intuitive or clear. It requires multiple readings to fully grasp how it is utilized in the final stages of the proposed method. To improve the clarity of this aspect, it would be advantageous for the authors to provide more explicit explanations and visual aids. For instance, incorporating a diagram that illustrates how and when the cell complex is employed would greatly enhance understanding and ensure a smoother comprehension of its role in the overall pipeline.*\n\n**Reply**:  The cell complex plays a dual role in our method. First, it modifies the ground truth that we use for training, which derives the target slicing boxes. Second, it acts as a regularizer in the post-processing step, where we round the predicted slicing boxes using our cell complex. A complete system overview is illustrated in Figure 3 in the attached pdf.\n\n**Q2**: *The authors' motivation in the proposed approach is heavily influenced by the transformer-based representation and the desire for a reasonable number of tokens. However, the impact of the selected number of tokens on the final performance, as well as the size and speed of the network, remains unclear. Conducting an ablation study that explores the relationship between the number of tokens and these factors would greatly enhance the effectiveness of the work. By systematically analyzing the effects of varying token counts, the authors can provide valuable insights into the trade-offs between representation size, computational efficiency, and model performance. Including such an ablation study would further strengthen the paper and provide a more comprehensive understanding of the proposed approach.*\n\n**Reply**: The choice of prediction representation determines the maximum number of tokens, as it is not feasible to vary the number of tokens dynamically during the training of the transformer. The main challenge we saw was integrating the cell grid into the pipeline. Predicting a token per cell would already require 2655 tokens and predicting tokens per edge or corner would require even more. It requires 600 tokens for shared corners and edges (HEAT [1]), 800 tokens for non-shared corners (Roomformer [2]), and 75 tokens for slicing boxes and room centers (ours). To better demonstrate the benefits of using fewer tokens, we provide a comparison between different numbers of tokens and analyze their training and inference time of our box transformer in Table 6. The results show that we can achieve the best computational efficiency with the least tokens. Due to the time limitation, we have not conducted a comparison of the model performance. We will add this comparison in the revision.\n\n**Table 6** : **Computational efficiency of training time by varying number of tokens.**\n\n|Number of tokens|Training time|Inference time|\n|-|-|-|\n|50|38.36s / epoch|6.923s / epoch|\n|100|42.84s / epoch|7.243s / epoch|\n|200|50.92s / epoch|7.620s / epoch|\n|300|59.97s / epoch|7.850s / epoch|\n|1000|290.26s / epoch| 8.172s / epoch|\n|2655|1063.99s / epoch| 10.07s / epoch|\n\n[1] HEAT: Holistic Edge Attention Transformer for Structured Reconstruction by Chen et al.\\\n[2] Connecting the Dots: Floorplan Reconstruction Using Two-Level Queries by Yue et al."}, {"review_id": "sOfuazDBaF", "reviewer": "Reviewer_qV7s", "summary": "The paper presents a learning-based framework on floorplan reconstruction from point cloud. To tackle with the complex polygons emerge widely in floorplan, the polygon is decomposed to non-overlapping, vertical-slicing boxes as regression target of the networks. A bbox transformer and a room transformer are designed for bbox regression and room regression, respectively. A few post-processing steps are needed to intersect the two branches and integrate the bboxes into rooms. Extensive experiments are conducted on Structured3D dataset to verify its performance.", "questions": "Overall I think the paper is novel and technically sound, with a clear pipeline for future research to follow. So currently I lean toward accepting this paper. However, I still expect authors’ feedback on my concerns listed in weaknesses. For question 1, I think a possible choice is to learn an assignment matrix during training between bboxes and rooms to encourage the matching process to be learned in a data-driven manner instead of only relying on center distances.", "limitations": "Authors have discussed their limitations on non-Manhattan floorplan reconstruction. It is indeed a challenging problem and may need some specified design to be proposed in the future.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "(1) The idea of dividing complex rooms into bboxes is novel and can effectively reduce the difficulty of learning the polygon corners from scratch.\n(2) The overall pipeline is clear and straightforward, and well presented.\n(3) The key challenges of the problem are well displayed in the introduction section, making it easier for readers to follow and future researchers to investigate on.", "weaknesses": "(1) Bbox assignment: the paper assigns the slicing bboxes to rooms by nearest distance search. However, this may not always be the case when some irregular polygons (the center living room shown in Figure 7 left). Some bboxes may be incorrectly matched with other rooms when there exists some neighboring rooms whose center is more adjacent. I wonder if there are more well-suited ways for bbox integration.\n(2) Bbox corner regression: to represent a bbox, the paper learns to regress both the center and two corners (4 scalars). But there exists redundancy between their coordinates. Regressing height and width is enough if the bbox center is given. Please demonstrate the reason to regress both corners other than learning (h, w).\n(3) A very recent work on floorplan generation (https://arxiv.org/abs/2207.13268) also employs the bbox (room) iou loss to enforce the correct structural regularity. Although it is a different problem, it is good to cite and discuss this paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1**: *Bbox assignment: the paper assigns the slicing bboxes to rooms by nearest distance search. However, this may not always be the case when some irregular polygons (the center living room shown in Figure 7 left). Some bboxes may be incorrectly matched with other rooms when there exists some neighboring rooms whose center is more adjacent. I wonder if there are more well-suited ways for bbox integration.*\n\n**Reply**: Each slicing box predicted by the box transformer has two corners and a room center. The room center indicates the center of the room that the box belongs to, not the center of the box itself. Similarly, each room predicted by the center transformer has its own center. We match the boxes and rooms by finding the nearest distance between the room centers predicted by the box transformer and the room centers predicted by the center transformer.\n    \nFor example, for the red room on the right side of the Results in Figure 7 (We assume this is the “living room” mentioned in the comment), it should contain a room center predicted at the red dot (please refer to Figure 4 in the attached pdf). As for the green room, it should contain a room center predicted at the green dot. Although the blue box is closer to the room center of red room (red dot), the room center of blue box (blue cross) should be predicted closer to the room center of green room (green dot) than the room center of red room (red dot), so the blue box could be matched with the green room instead of the red room.\n\n**W2**: *Bbox corner regression: to represent a bbox, the paper learns to regress both the center and two corners (4 scalars). But there exists redundancy between their coordinates. Regressing height and width is enough if the bbox center is given. Please demonstrate the reason to regress both corners other than learning (h, w).*\n\n**Reply**: We use four corners of each slicing box as the input for our box corner loss, denoted as $b_i^{corner}$. As we discuss in the response to W1, each slicing box predicted by the box transformer has two corners and a corresponding room center. The raw output of each slicing box, denoted as $b_i$, represents the upper-left and lower-right corners of the slicing box. This means that we convert the format of the predicted slicing box before we compute the box corner loss, as shown in equation 8 of the main paper. Moreover, we multiply the loss with a mask term $w_{\\sigma (i)}$ to ensure that we only compute the loss on the predicted corners whose corresponding target corners are the corners in the ground truth layout.\n\n**W3**: *A very recent work on floorplan generation [1] also employs the bbox (room) iou loss to enforce the correct structural regularity. Although it is a different problem, it is good to cite and discuss this paper.*\n\n**Reply**: Thank you for providing us with the related recent work. We will discuss and cite this paper in the revision.\n\n**Q1**: *Overall I think the paper is novel and technically sound, with a clear pipeline for future research to follow. So currently I lean toward accepting this paper. However, I still expect authors’ feedback on my concerns listed in weaknesses. For question 1, I think a possible choice is to learn an assignment matrix during training between bboxes and rooms to encourage the matching process to be learned in a data-driven manner instead of only relying on center distances.*\n\n**Reply**: There are different design choices to tackle the problem of clustering the boxes. The main challenge of learning an assignment matrix between the boxes and rooms lies in the unordered nature of the rooms, combined with the variable number of rooms and slicing boxes. This makes it difficult to assign room IDs during training, as mentioned in Section 3.2 of the main paper. As illustrated in the reply to W1, our method should be robust enough to address this issue.\n\n[1] End-to-end Graph-constrained Vectorized Floorplan Generation with Panoptic Refinement by Liu et al."}]

Author rebuttal:
We are very grateful to all the reviewers for their valuable and insightful comments. To address all the questions, we have prepared a pdf document that contains figures to better illustrate our responses. Due to the format limitation of this pdf document, we will also explain all the figures below:

**Figure 1** (replying to the question **W2** of reviewer **mWNJ**): We show the point cloud and the corresponding cell complex of one case. The point cloud reveals that some of the occluded areas have very few points, which prevents the conversion of the local geometries to wall segments and the recovery of the correct layout structure by other methods. Our unlimited extended cell complex can propagate the global structure knowledge from the areas with dense point clouds to those occluded areas and provide more geometrical guidance for the subsequent steps of our pipeline.

**Figure 2** (replying to the question **W3** of reviewer **EcBt**): We present the flaws in the SceneCAD annotations, where the structures that are Manhattan are often marked as non-Manhattan. The middle column is the ground truth annotation for SceneCAD dataset, and the right column is the ground truth that we believe it should be. This kind of negligent labeling is very widespread on SceneCAD, lowering the quality of evaluation on SceneCAD.

**Figure 3** (replying to the question **Q1** of reviewer **84jG**): We demonstrate our system pipeline to clarify how the cell complex is employed in our method. The cell complex acts as a regularization mechanism during the training phase and is also utilized in the post-processing step to enhance the accuracy of the reconstructed floorplans.

**Figure 4** (replying to the question **W1** of reviewer **qV7s**): This figure illustrates the post-processing clustering process. Boxes are grouped into the same room based on a two-step assessment:

1. Their predicted room centers, as determined by the box transformer.
2. Proximity to the room centers estimated by the center transformer.

This indirect association method helps resolve situations where two boxes may be spatially near each other but do not actually belong to the same room.

Meta-review: All the reviewers are initially positive about the paper. Rebuttals addressed most of the concerns and all the reviewers keep the original ratings. The area chair agrees with the positive assessment of the paper and recommends acceptance.
Decision: Accept (poster)