Parsing Floor Plan Images
Samuel Dodge
Arizona State University

Jiu XuBjörn Stenger
Rakuten Institute of TechnologyAbstract
This paper introduces a method for analyzing floor
plan images using wall segmentation, object detection,and optical character recognition. We introduce a challenging new real-estate floor plan dataset, R-FP, evaluate different wall segmentation methods, and propose
fully convolutional networks (FCN) for this task. We
explore architectures with different pixel-stride values
and more compact ones with skipped pooling layers.
An FCN-2s with a 2-pixel stride layer achieves stateof-the-art performance, obtaining a mean Intersectionover-Union score of 89.9% on R-FP, and 94.4% on thepublic CVC-FP data set. Using OCR and object detection, we estimate room sizes. Finally, we show applications in automatic 3D model building and interactive
furniture fitting.

1

Introduction

Architectural floor plans are scaled drawings ofapartments or building layouts. They contain structural and semantic information, e.g. room types andsizes, and the location of doors, windows, and fixtures. Floor plans are a common tool for real estate agents in selling or renting out a space. Parsing
such images automatically has a number of applications, such as similarity search [1], CAD model generation, and 3D model creation for rendering and interactive walkthroughs [6, 8, 10]. Floor plan analysis
has been an active research topic in the area of document processing, but has mainly focused on processing
high-resolution scans. In this paper we instead focus
on readily available floor plan images from real estatewebsites. These plans were created in a number of different drawing styles and are at lower resolution thanstandard architectural documents. Previous methods
that rely on image binarization fail on most of theseimages. We propose a learning-based approach to segment walls and detect objects. Deep networks havebeen shown to perform well on semantic segmentation
tasks [2, 12, 14]. Specifically, we train fully convolutional networks (FCN), explore different network architectures, and evaluate their performance, comparing
them with baseline methods. The main contributions
are: (1) a single method using an FCN for segmenting
walls in different drawing styles with state-of-the-art
performance, (2) a new dataset of 500 labeled floor
plan images from a real-estate website, and (3) applications in automatic 3D model generation and interactive furniture fitting. Having extracted the sizes ofwalls from OCR, we are able to place furniture items
into the model at the correct scale.

2

Prior work

Early systems were designed for converting 2D floor
plans to 3D interactively, e.g. [6, 9, 10]. Two survey
papers provide a good review of methods for generating models from architectural plans [7, 15]. Many solutions share a common pipeline which includes binariza-Figure 1: Parsing a floor plan image: The goal isto obtain a model by segmenting walls, recognizing objects,
and estimating dimensions using OCR.

tion, structure and text separation, symbol recognitionand vectorization. Within the ScanPlan project, Maćeet al. detect walls and doors using the Hough transform [11]. Wall polygons are partitioned iteratively
into rooms, assuming convex room shapes. Ahmed
et al. [1] process high-resolution images by segmenting
lines according to their thickness, followed by geometrical reasoning to segment rooms. Doors are detectedusing SURF descriptors. In our case, the input images are of lower resolution and standard binarization
fails on many images. De las Heras et al. [4] proposed segmenting walls using the assumption of them
being a repetitive element, modeled by straight parallel lines. This method performs well on high-resolution
images in different graphical styles. Similarly, Gimenez
et al. [8] binarize the floor plan images and detectwalls by looking for parallel lines separated by a distance within a predefined range. Wall thickness is determined automatically by clustering the distance values. Unfortunately, the assumption of consistent wall
appearance does not hold in many of our input images. In [3] De las Heras et al. propose classifying image patches using a bag-of-words (BoW) model. This
BoW model is tuned to each particular graphical stylein the CVC-FP data set, and is trained on each subsetof wall types with different parameters. Our method,by contrast, is learned only once for all graphical styles
in the training set.

3Parsing floor plans

We combine three methods to extract geometric andsemantic information: Wall segmentation, object detection, and optical character recognition (OCR).
Wall segmentation. We use fully convolutional
networks (FCN) for segmenting wall pixels, following [14]. We train models with different pixel-stridevalue and compare FCN architectures for different final stride layers to find the best stride value. Starting with a FCN-32s with a 32-pixel stride, initializedwith VGG-16 parameters [14], in sequential training
each model is initialized with the parameters of the3.1

Figure 2: FCN with different stride values: Addinglayers with reduced stride increases performance according
to all three measures.

previous one. Following [14] all models are trainedby stochastic gradient descent with momentum (value0.9, weight decay of 5−4 ) and a batch size of 1. The
learning rates are set to 10−8 , 10−9 , 10−10 , 10−10 , and10−11 for FCN-32s to FCN-2s, respectively. These values are chosen by running two training procedures withdifferent learning rates for 10 epochs and chosing therate that reduces the error more rapidly. To introduce a layer with smaller stride (by a factor of two)
we need to fuse two streams within the network: apooling layer, and a deconvolutional layer. For thiswe ensure that the sizes and outputs of the layers isconsistent. For overviews of the network architectures,
please see the supplementary video. The FCNs arecompared with these baseline methods: The first baseline uses a global threshold (GT) with the thresholdvalue determined using Otsu’s method. We additionally use morphological closing to remove small linesthat are not walls (GT+MC). Finally, we remove text
regions from the image using the Google Vision API(GT+MC+TR). We also implement a patch-based approach to wall detection, including a BoW approachsuggested in [3]. We sample 10 × 10 pixel patches fromthe image such that the patches have at least one dark
pixel (other pixels are assumed to be non-wall). We
take the PCA of the training patches (1000 per image)and take the principal components that describe 95%of the variance of the data for the CVC-FP data set
and 50 components for the R-FP dataset. We compare
three binary classifiers to determine which patches arewall or non-wall: random forest (RF), linear support
vector machine (SVM), and BoW. During testing wesample the image at a stride of 3 pixels. The final prediction for each pixel is the mean of the predictions
from the overlapping boxes that contain the pixel.
Object detection. We use the Faster R-CNN
framework from [13] for object detection and recognition, using a light-weight ZF network [16] for feature
map computation. A subset of the R-FP images wasannotated for training and testing, with 6 different object classes (doors, sliding doors, kitchen stoves, bathtubs, sinks, and toilets). The model is trained on 144
images for 150 epochs with ZF networks using top2000-score Region Proposal Networks.
Optical character recognition. Our input images
may contain both English and Japanese text. We use
the Google Vision API for text detection and character
recognition, which handles multiple languages.

Experimental Framework

Data sets. For evaluation, we collected a newdataset, R-FP, of 500 floor plan images from a public real estate website. These images have different
sizes, with side lengths in the range of 156-1427 pixels.
The images were created as a tool for real-estate agentsand show some degree of artistic freedom, such as theuse of different color and shading schemes and decorative elements. Compared with other public datasets,
this dataset exhibits a significant amount of variation.
Fig. 3 shows example images. We also evaluate on thepublicly available CVC floor plan data set [5]. Thisset includes 122 high-resolution images in four different drawing styles.
Metrics. As evaluation metrics we use the mean
pixel accuracy and mean Intersection-over-Union(IoU), as previously introduced in [14] as well as theJaccard Index (JI) for wall pixels as proposed in [3].

4Results

Wall segmentation using FCN with differentstrides. Figure 2 shows the increase in mean accuracy, mean IoU, and Jaccard index when progressively
adding new layers with smaller stride values. Note thata stride value of 2 is lower than in previous work on semantic segmentation, where a stride value of 8 was suggested [14]. In our layout prediction task, fusing evenshallower layers achieves further improvement. Theselayers contain features representing local details suchas corners and edges, which are important for parsing
floor maps.
Wall segmentation on R-FP dataset. Wall segmentation results on the new R-FP dataset are shownin Table 1. The patch-based random forest classifier
slightly outperforms the baseline methods in terms ofmean IoU. FCN-2s networks, trained either in stages orat once, perform best. Figure 3 shows sample resultsby different segmentation methods. FCN-2s at-oncetraining results in comparable performance as FCN2s trained in stages, while reducing the training time
(20h for 400 iterations compared with 35h for staged
training, both on a GeForce 1080 GPU). The FCN-2s
network segments one image in approximately 0.14s.
Skip-layer architectures. For further investigating
the importance of fusion by different pooling layers, wetest the performance of two alternative network architectures, both skipping particular upsampling streams.
In the first, FCN-2s-skip-2-4, pool2 and pool4 layers areskipped, which means two 8× and one 2× interpolation layers are placed before the final output layer. In
the second architecture, FCN-2s-skip-2-3-4, additionally pool3 layer is skipped for training, which means one
16× and one 2× interpolation layers are placed before
the final output layer. The results in Table 1 show thatskipping pool2 and pool4 layers results in similar performance to FCN-2s staged training, however, skipping
all layers pool2, pool3, and pool4 layer leads to poorer
performance. The performance of training with skip
layers infers some feature redundancy in stream fusion. FCN-2s-skip-2-4 has a reduced training time of
18h and requires approximately 0.12s for segmenting
one image.
Evaluation on CVC-FP dataset. Wall segmentation results on the public CVC-FP dataset are shown

InputGT-MC-TEXT

Patches-RF

FCN-8sFCN-2sground truth

Figure 3: Wall segmentation on images from the new R-FP data set. Global thresholding (GT-MC-TEXT) does
not handle decorative lines and symbols. The patch based segmentation (Patches-RF) cannot handle different drawing styles
with the same parameter settings. Segmentations by FCN-8s [14] are more blurred than those by the proposed FCN-2s. The
bottom row shows difficult example on which all methods perform poorly.

Mean acc.

Mean IoU

JIMethod

Global Thresh. (GT)
GT+MC
GT+MC+TR

90.1
90.1
90.3

73.7
73.7
76.9

53.0
53.0
58.3

Patches - RF
Patches - SVM
Patches - BoW

89.7
82.1
89.5

77.6
72.3
75.8

FCN-2sFCN-2s at-once

94.0
92.9

FCN-2s-skip-2-4
FCN-2s-skip-2-3-4

93.6
91.4

MethodMean acc.

Mean IoU

JIGT
GT+MC
GT+MC+TR

78.7
95.1
95.3

69.6
80.1
85.0

41.5
61.9
71.2

59.3
49.6
56.3

Patches - RF
Patches - SVM
Patches - BoW

92.8
92.6
93.0

89.3
89.0
87.9

79.2
78.5
76.6

89.7
89.9

80.9
81.2

FCN-2s

97.3

94.4

89.2

89.7
87.3

81.0
76.6

Table 1: Wall segmentation results on the new realestate floor plan dataset (R-FP)

in Table 2. FCN-2s is also the top performing methodon this data set. Our patch-based baseline does notachieve as good performance as reported by De Las
Heras et al. [3], however the main difference is that
we perform training with a single set of parameters, whereas [3] choose different parameters for different image subsets. Another difference is the evaluation protocol. We perform 5-fold cross validation,
whereas [3] perform leave-one-out cross validation forcertain subsets of the dataset.
Cross-dataset performance. Table 3 shows theperformance of the FCN-2s model trained on one
dataset and tested on the other. Performance is poor,
largely owing to the differences in the drawing styles.
The R-FP dataset has many different types of wall images. Furthermore, the R-FP dataset is of floorplans inJapan and the CVC-FP dataset is of floorplans fromEurope. The CVC-FP dataset consists of high qual-

Table 2: Wall segmentation results on the public
CVC-FP dataset [5]Training

Test

Mean acc.

Mean IoU

JI

CVC
R + CVC

RR

82.7
94.0

76.1
90.5

56.0
82.5

R
R + CVC

CVC
CVC

84.2
96.0

81.7
92.9

64.7
86.3

Table 3: Cross-dataset generalization. An FCN-2s
model trained on one data set generalizes poorly when testing on the other. Training on the union of the training setsboosts performance.

ity architectural floorplans, whereas the R-FP dataset
consists of floorplans from an internet real-estate website. We also train a FCN-2s on the combined training
data, resulting in significantly improved results on bothtest sets, in particular achieving better performance on
R-FP than the model trained on R-FP alone.
Object detection performance. The average precision evaluated on 25 test images is 96.0% for doors,
35.9% for sliding doors, 76.2% for kitchen ovens, 95.8%
for bath tubs, 69.2% for sinks, and 70.8% for toilets,

5 Conclusion

Figure 4: Room size estimation example: room sizes
read with OCR in the input image (left) are propagated tounlabeled rooms (right).

We show that an FCN with stride of 2 achieves better performance than a stride of 8 as in [14]. Thesmaller stride is effective because some walls can beeasily described by low level features. The FCN model
combines features from subsequent layers that describe
contextual information and texture information withthese low level features. Secondly, the smaller stride
allows us to predict more precise wall locations. Walls
are often very thin structures that cannot be precisely
predicted at larger strides.
From the floor plan we extract a parsed representation of wall locations, objects, and size information.
We show two example applications using this information, 3D model creation and furniture fitting.

References

Figure 5: 3D model creation:Wall segmentation allows
automatic 3D model creation from floor plans (left). OCRof room measurements allows inserting 3D furniture models
scaled to the scene (right).

for an IoU value of 50%. Using 300 region proposals,
the average detection time is about 50ms on a GeForce
1080 GPU.
Text detection performance. We labeled 20 images from the R-FP dataset with text locations andvalues. For our applications (3D model generation andfurniture fitting), the most important text is the roomsize. The OCR is able to detect and recognize 74.1% of
the room size annotations. Note that many floor plans
contain multiple size annotations.

4.1

Applications: 3D model creation and furniture fitting

Using the wall segmentation from Section 3, we cancreate an approximate 3D model of home layout solely
from the floorplan. A true 3D reconstruction of thelayout requires a-priori knowledge of the wall height.
In absence of this information, we arbitrarily choose aheight. To create the 3D model, we consider the wallsegmentation result as an elevation map, where all of
the walls are of uniform height. Figure 5 (left) shows
an example of fully automatic reconstruction.
Secondly using the segmentation information andtext information we can infer the area of rooms, aswell as the length of walls (Figure 4). Rooms are delimited by walls, (sliding) doors and windows. For eachsegmented room, we query the text information for theJapanese room measurement unit (Jo). We computethe relationship between the room size in physical units
and pixels to compute the pixel density (in pixel/J).
This pixel density can be used to compute an estimateof the area of rooms that are not labeled with physicalunits, and well as can be used to compute wall length.
In addition, with the room size information we can estimate the fit of furniture in a room (Figure 5, right).
Please see the supplemental video for further results.

[1] S. Ahmed, M. Liwicki, M. Weber, and A. Dengel. Automatic room detection and room labeling from architectural
floor plans. In Int. Workshop on Document Analysis Systems, pages 339–343, 2012.
[2] V. Badrinarayanan, A. Kendall, and R. Cipolla. SegNet: Adeep convolutional encoder-decoder architecture for sceneegmentation. In TPAMI, 2017.
[3] L.-P. De Las Heras, D. Fernández, E. Valveny, J. Lladós,and G. Sánchez. Statistical segmentation and structural
recognition for floor plan interpretation. In IJDAR, pages
1–17, 2013.
[4] L.-P. De Las Heras, D. Fernández, E. Valveny, J. Lladós,and G. Sánchez. Unsupervised wall detector in architectural floor plans. In ICDAR, 2013.
[5] L.-P. De Las Heras, O. R. Terrades, S. Robles, and G. Sánchez.
CVC-FP and SGT: a new database for structural floor plananalysis and its groundtruthing tool. In IJDAR, 2015.
http://dag.cvc.uab.es/resources/floorplans/.
[6] P. Dosch, K. Tombre, C. Ah-Soon, and G. Masini. A
complete system for the analysis of architectural drawings.
In Int. J. Document Analysis and Recognition, volume 9,
pages 102–116, 2000.
[7] L. Gimenez, J.-L. Hippolyte, S. Robert, F. Suard, andK. Zreik. Review: reconstruction of 3D building information models from 2D scanned plans. In J. Build. Eng.,
volume 0, pages 48–56, 2015.
[8] L. Gimenez, S. Robert, F. Suard, and K. Zreik. Automaticreconstruction of 3D building models from scanned 2D floor
plans. In Automation in Construction, volume 63, pages
48–56, 2016.
[9] H. Goto, K. H. Law, and G. Brickey. Knowledge-based
creation of an architectural 3-D model from 2-D drawings.
Technical Report TR-59, Stanford University, 1991.
[10] R. Lewis and C. Séquin. Generation of 3D building models
from 2D architectural plans. In Computer-Aided Design,
volume 30, pages 765–779, 1998.
[11] S. Macé, H. Locteau, E. Valveny, and S. A. Tabbone. Asystem to detect rooms in architectural floor plan images.
In Int. Workshop on Document Analysis Systems, 2010.
[12] H. Noh, S. Hong, and B. Han. Learning deconvolution network for semantic segmentation. In CVPR, 2015.
[13] S. Ren, K. He, R. Girshick, and J. Sun. Faster R-CNN: Towards real-time object detection with region proposal networks. In NIPS, 2015.
[14] E. Shelhamer, J. Long, and T. Darrell. Fully convolutionalmodels for semantic segmentation. In TPAMI, 2016.
[15] X. Yin, P. Wonka, and A. Razdan. Generating 3D building
models from architectural drawings: A survey. In IEEE
CGA, pages 20–30, 2009.
[16] M. D. Zeiler and R. Fergus. Visualizing and understanding
convolutional networks. In ECCV, 2014.