SceneExpander: Real-Time Scene Synthesis for Interactive Floor Plan Editing

Authors: shao-kui  zhang
junkai  huang
liang  yue
jia-tong  zhang
jia-hong  liu
song-hai  zhang
yu-kun  lai
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-03-01 07:39:28

Abstract: Scene synthesis has gained significant attention recently, and interactive scene synthesis focuses on yielding scenes according to user preferences. Existing literature either generates floor plans or scenes according to the floor plans. This paper generates scenes over floor plans in real time. Given an initial scene, the only interaction a user needs is changing the room shapes. Our framework splits/merges rooms and adds/rearranges/removes objects for each transient moment during interactions. A systematic pipeline achieves our framework by compressing objects' arrangements over modified room shapes in a transient moment, thus enabling real-time performances. We also propose elastic boxes that indicate how objects should be arranged according to their continuously changed contexts, such as room shapes and other objects. Through a few interactions, a floor plan filled with object layouts is generated concerning user preferences on floor plans and object layouts according to floor plans. Experiments show that our framework is efficient at user interactions and plausible for synthesizing 3D scenes.

Main paper: 
 introduction: 
 Research has emphasized the importance of synthesizing 3D scenes [18] . The plausibility of generating 3D scenes has significantly improved in the past decades [19] . Scene synthesis usually selects a set of objects and plausibly arranges them in a given floor plan [26] . The object arrangements should match the learned spatial relation priors and the strict constraints of the floor plan [5, 27, 29] , e.g., the number of objects should fit the floor plan's capacity.To synthesize scenes with user preferences, another branch of scene synthesis emphasizes interactively crafting scenes, i.e., controlling the process of scene synthesis. Through cursors [28] (e.g., using a mouse), texts [23] , panels [25] , etc., users can specify selections/translations/rotations for a few objects, and the frameworks arrange involved objects subsequently in each interactive session. For example, Clutterpalette [24] requires a user to select a position in the 3D scene for each interactive session. An object appears concerning existing objects and supports in the selected position.However, existing literature mainly interprets user preferences as "selections and transformations" of objects, which is practical for users already satisfied with the provided floor plan. On the contrary, if users have not decided on a specific floor plan, they may repeatedly choose/modify a floor plan, interactively synthesize an object layout and decide to change the floor plan again, even if they only want to make minor tweaks to the floor plan. On the other hand, users may not know if the floor plan is ideal until they see how the objects can be arranged accordingly. To interactively synthesize a single floor plan with object layouts, a user has to conduct the synthesis process time after time for different floor plans. Interior designers typically face considerable floor plans. Therefore, the 1 Please refer to a supplementary video for an overview of our framework. The source code is publicly available at https://github.com/Shao-Kui/3DScenePlatform#sceneexpander. challenge is to concurrently control floor plans and objects during the synthesis process, i.e., in the early stages of interior design, both the floor plan and objects' layouts can be provided simultaneously.This paper introduces an interactive scene synthesis framework, where the interactive units are no longer objects but "room shapes" (floor plans). As shown in Figure 1 , a user edits the floor plan for each interactive session, resulting in an expanded/cropped floor plan. Our framework judges the new floor plan and infers (a) rearrangement of objects, (b) splitting/merging rooms and (c) adding/removing objects. This entire inference process runs in realtime. Objects and rooms are dynamically and gradually changed by continuously adjusting the floor plan. Objects are continuously arranged along with changing floor plans at every transient moment, as shown in Figure 2 . Users can stop when they are satisfied with the synthesized object layouts.Our framework is executed iteratively for every transient moment. An algorithm pipeline is proposed to catch the changes in floor plans and decide if we should divide rooms, how we should divide rooms, how we should rearrange objects, and what objects should be added/removed (Section 3). The pipeline includes generating division plans, which help divide existing rooms so the floor plan is gradually enriched with more rooms. Our framework proposes multiple division plans for each transient moment, so it subsequently evaluates them and chooses the best or none of them (Section 4). If a room is enlarged, shrunk or divided, the pipeline may add more objects, remove objects that no longer fit the room, or arrange existing objects. We propose "Elastic Boxes" representing groups of objects for layout adjustment, as shown in Figure 2 , to address the above operations (Section 5).To our knowledge, we are the first to investigate interactively synthesizing scenes over editing floor plans. Our work makes the following contributions:• We present an interactive scene synthesis framework that adds/rearranges/removes objects and splits/merges rooms over continuous changes of floor plans. • We propose a system pipeline that detects room shape changes, executes room divisions, dispatches object groups and arranges objects in a transient moment of editing floor plans. • We propose elastic boxes, which help spatially fit object groups to the continuous floor plan changes, thus plausibly arranging objects at any moment.Note that our framework still arranges objects similar to typical synthesis frameworks, e.g., [24, 28] . Our framework does not generate floor plans [3, 6, 8, 13, 21 ] (Section 2.3). Instead, the floor plans are input from users and are interactively controllable. 
related work 2.1 interactive scene synthesis: 
 Scene synthesis has been developed significantly in recent decades [26] . Researchers first investigate automatic scene synthesis, i.e., automatic selection and arrangement of objects in a room [5, 16, 20, 22, 27] . However, it is hard for this technique to generate scenes according to user preferences, e.g., a bedroom containing a double bed adjacent to the window. Therefore, interactive scene synthesis aims to control the scene synthesis process [28] . Yu et al. [24] introduce a framework for placing small objects into scenes concerning supports, co-occurrence, etc. Zhang et al. [25] also place small objects but are concerned with the up/ down/ left/ right/ front/ back between objects. Merrell et al. [12] introduce a framework where users can specify constraints. The framework will generate several suggestions to be chosen. Zhang et al. [28] introduce a real-time framework for inserting objects into scenes. An object pops up appropriately for each transient moment while moving the cursor (e.g., with a mouse). Zhang et al. [30] allow editing a group of objects simultaneously, following the cursor. Additional attributes control objects in the same group.Besides, literature introduces editing scenes passively, i.e., making users unaware of arranging objects. Liang et al. [10] rearrange objects in a workspace so the new layout can consume less commuting time. Zhang et al. [31] expand this to multi-users. They also calculate the optimal workspace layout and work allocation plan based on the individual work ability of employees, which can generate a more comprehensive scheduling plan. Facing virtual reality, Dong et al. [4] rearrange objects when users wear a Head Mounted Display to ensure they do not collide with obstacles. Liu et al. [11] propose a framework for arranging desktop items by studying user behaviors on the desktop, considering constraints such as object stacking and compactness.The literature above allows editing objects with fixed floor plans. If a user is unsatisfied with the floor plan, e.g., needing spaces for more objects or new rooms, the user has to rearrange objects in the next floor plan from scratch. In our framework, an interactive unit is no longer an object or a group of objects. Users edit floor plans to indirectly manipulate objects, where the object layouts are synthesized according to the user-specified floor plans.Existing methods also consider editing patterns inside 3D objects, e.g., rungs in the monkey bars. Funkhouser et al. [7] enable the creation of new 3D objects using existing objects. Schulz et al. [17] assemble components into a new object. Their method calculates the connections and alignments. Bokeloh et al. [1, 2] deform shapes while preserving object structures. They first search for discrete variations, such as component repetitions in an object. Then, the discrete variations are added or deleted when editing the objects. In contrast, Ovsjanikov et al. [14] explore 3D objects by continuously editing their components. A template model is extracted, deforming its components. Different objects are retrieved accordingly.The above literature inspires us. We create new scenes using existing objects, where the components are groups of objects. However, indoor scenes usually do not need repeated patterns. Objects are inserted according to their functions.Scene synthesis often assumes a floor plan is given, where objects are arranged. In contrast, other literature generates floor plans for scene synthesis [6] . Wu et al. [21] formulate neural networks to locate rooms, allocate areas for rooms, and assign room types. Nauata et al. [13] generate floor plans with a room graph, where each vertex indicates a room type. Hu et al. [8] generate floor plans with user constraints. They first generate graphs representing the relations of rooms. Then, the graph is decoded and aligned to yield floor plans. Deitke et al. [3] combine scene synthesis and floor plan generation into an automatic end-to-end framework.The above methods can generate plausible floor plans, but they are end-to-end approaches, where a complete floor plan is generated given an input such as a graph. In contrast, our framework does not generate complete floor plans but explores continuous room divisions, i.e., a division is calculated for each transient moment. Hence, our framework seeks for controllable room division over editing floor plans. In other words, a floor plan is not "generated" but "adjusted" to adhere to user input. 
framework pipeline: 
 Figure 3 shows how our framework is executed for every transient moment, a minimal period detected by the computer. For example, hundreds of transient movements occur when a user moves a room shape's edge. The only input is a modified room shape. Our framework generates several division plans according to the changed room shape. A division plan indicates how to divide the room, and a room shape has multiple plans. Two evaluations subsequently decide whether the framework should divide the room by evaluating the current whole floor plan and the existing elastic boxes. Section 4 discusses generating and evaluating division plans. Whether a room should be divided has two branches. Each transient movement executes one of the branches and waits for the next transient moment, i.e., the next user interaction.If the framework does not divide the room, the room shape is changed according to the interaction. Our framework then detects the elastic boxes that should be adjusted due to the modified room shape. For example, if an elastic box leans against a modified wall, the elastic box will be adjusted, i.e., translated according to the wall. Its adjacent elastic boxes are also involved in adjustments. This detection is recurred until no adjacent elastic boxes exist. "Shrinking" the room shape may remove one or more elastic boxes. After adjusting the elastic boxes, their objects are translated or rotated following their layout strategies, which suggest (1) object-object transformations, e.g., transforming a sofa concerning a coffee table, and (2) object-edge transformations, e.g., shrinking a living room may cause a sofa and a TV cabinet getting close to their coffee table. The object-object and object-edge transformations are maintained in a database. When all existing objects have been adjusted for this transient movement, our framework further attempts to add new elastic boxes by trying to find a free space. A new elastic box is The modified room only changes its shape and the objects involved. We detect the elastic boxes that must be adjusted according to the new room shape. The objects inside elastic boxes are arranged according to their layout strategies (Section 5), which are stored in a database. (Yes): The modified room is split. The elastic boxes involved have been reassigned to the two new rooms. Then, the elastic boxes and their objects are adjusted and arranged.added and rotated according to a dependent wall if a free space exists.If the framework should divide the room, a division plan is picked, and a new room is derived from the original room. Our framework examines each elastic box and reassigns it to one room, referring to room types and the ease of spatially moving the elastic box to another room. Every room has a type, e.g., bedroom, bathroom, etc. The two evaluations above also score the division plans, and the framework adopts the one with the highest score. A division plan contains the type for the new room indicating the initial elastic box, such as a "coffee table set" for living rooms. Since the room shapes are modified, the elastic boxes are adjusted, and their objects are arranged. Section 5 discusses the elastic boxes' mechanism and how we arrange elastic boxes' objects in detail.Our framework supports merging two rooms into one if a room is shrunk. The merging logic aligns with the "Divide" branch in Figure 3 . If, during the evaluation process, the framework decides that the room should be merged with another one, our framework will check each elastic box in the rooms to be merged, using the same evaluations above. The objects' arrangements follow how we utilize the elastic boxes above. We illustrate our framework to make our technical details self-contained, focusing on enlarging and dividing. Note that we allow users to choose their favoured room types, which can easily be added by giving tags to each room. 
room divisions: 
 Our framework splits the room by evaluating several division plans and choosing the best one. A division plan consists of two room shapes, r 1 and r 2 , with room types 𝑡 1 and 𝑡 2 . r 1 and r 2 are divided from the original room. In each transient moment, only one room directly adjacent to the moving edge can be divided.We generate division plans using a sampling method, as shown in Figure 4a , where an "inner edge" divides a room into two, and our method proposes a series of inner edges. Each inner edge differs from the previous division plan with a constant distance. The room adjacent to the moving edge is a new room that will be assigned a new room type, and the other room will keep the original type.To evaluate a division plan, we define an evaluation function 𝜙 (r, 𝑡) to measure how a room shape r and a room type 𝑡 are suitable, as shown in Equation 1, where 𝛽 (r) represents the number of the room r's edges. 𝛼 (r) is the ratio of r's area to the area of its rectangular bounding box. The sum of 𝛽 (•) and 𝛼 (•) evaluates the regularity of room shape r. A high value of it indicates a regular room, such as a rectangle in Figure 4b . The exponential and logarithmic are applied, so the values' differences are significant.EQUATION𝑃 𝑟 (r, 𝑡) is the room area distribution function extracted from the RPLAN dataset [21] . For each room type 𝑡, the distribution varies. 𝛿 (r) is the minimal edge length of the room's bounding box, while 𝜖 (𝑡) is a constant representing the desired minimal length of a room type 𝑡. 𝑃 𝑟 (•) and min(1, 𝛿 (r) 𝜖 (𝑡 ) ) ensure the room has proper space to accommodate the functionality of its type.We also define another evaluation function, 𝜓 (c), to control the number of each type of room, as shown in Equation 2. Each entry of c is the number of a room type in the current editing floor plan, and 𝑛 is the number of room types, i.e., c is an 𝑛-dimensional vector.EQUATIONS = {(ĉ, 𝑣)} is the distribution of room types extracted from RPLAN [21] . ĉ is a vector of room types similar to c. 𝑣 is the frequency of ĉ in the dataset. P and Q are two positive constant vectors that measure if a new room should be added with a specific room type. Room types such as "living rooms" that are necessary but should not be many have large P 𝑘 and Q 𝑘 . Necessary room types, such as "bedrooms", can be many and have a large P 𝑘 but a small Q 𝑘 . Room types such as "storage" are unnecessary and have small P 𝑘 and Q 𝑘 . Then for each division plan {(r 1 , 𝑡 1 ), (r 2 , 𝑡 2 )}, its overall evaluation 𝜎 (r 1 , 𝑡 1 , r 2 , 𝑡 2 ) is shown in Equation 3.EQUATION)where c is the new room type count vector after new rooms are introduced, i.e., an entry of c increases. 𝛿 (r 1 , r 2 ) is the number of elastic boxes removed, which will be explained in section 5. If a division plan eliminates too many elastic boxes, 𝛿 (•) will be large. 𝐶 𝑡 and 𝐶 𝑑 are the trade-offs between the two evaluations. We also evaluate the original room, i.e., 𝜙 (r, 𝑡) -𝐶 𝑡 ×𝜓 (c), which is compared with the evaluations yielded by various division plans. Suppose any of the division plans have a higher evaluation. In that case, the division plan with the highest evaluation is accepted, and the original room is divided, i.e., the "Yes" branch in Figure 3 and Section 3. Otherwise, the room is not divided, i.e., the "No" branch.If a room is shrunk, we will determine the connectivity of the room r with the room from which it was divided. If they are connected and the evaluation of the merged room is higher than the evaluation of the two individual rooms, the two rooms are merged. 
elastic boxes: 
 An elastic box contains several objects, as shown in Figure 5 . We use each object's "layout strategy" to calculate its transformation at every transient moment. A "layout strategy" is (1) an object's relative transformations concerning another object in the same elastic box (e.g., the yellow arrows in Figure 5 ) or ( 2) an object's relationship to a specific boundary of the elastic box (e.g., the red arrows in Figure 5 ). Each elastic box contains four boundaries. Each boundary may be adjacent to a room's edge (wall). Whenever the four boundaries are determined, the elastic box's objects are arranged according to our layout strategies, e.g., Figures 5(a-d ).An elastic box's semantics consist of its objects' semantics, indicating its function in an indoor scene. We utilize the Spatial And-Or Graph of Qi et al. [16] to infer elastic boxes given room types. Qi et al. [16] build a hierarchy to infer objects (children) from room types (parent). Our framework adds elastic boxes in between.As shown in Figure 3 , our framework contains three operations on elastic boxes. The first operation, "Evaluation Based on Elastic Boxes," evaluates division plans given existing elastic boxes. We attempt to assign elastic boxes from the original room to one of the new rooms. If the room's semantics fit the elastic box, we will pull the elastic box into the new room, where we first test the intersection of elastic boxes and rooms. If an elastic box and a room disjoint, the elastic box should not be moved into the room. Otherwise, we will check the intersection between the room's edges and the box's boundaries. The elastic box should be moved along the normal direction of the boundaries outside the room. Figure 6 illustrates more details. If the distance to move the elastic box is large, it should not be moved into the new room. The elastic box is removed if an elastic box should not be moved into both new rooms. Each elastic box from the original room is assigned to a new room or removed through this process.The second operation is "Adjusting Elastic Boxes, " which changes existing elastic boxes' boundaries and removes the elastic boxes that are too small after room shape modified. If an edge (wall) is moved, we enumerate each boundary adjacent to it, where the boundaries When we try to assign the elastic box to the new room (Red), we should pull the box along the normal direction of the boundaries (Blue Arrows). We check the intersections between the elastic box's boundaries and room edges, and edges 𝑒 1 and 𝑒 2 intersect with the box's boundaries. For example, we achieve the boundary 𝑏 1 facing the same direction as 𝑒 1 , and calculate the distance between 𝑒 1 and 𝑏 1 . The box is moved along the normal direction of 𝑏 1 for the distance.are added to a pending set. For each boundary, we also check its opposite boundary's adjacencies, added to the set, as shown in Figure 7 . If an edge is adjacent to a boundary, the boundary will not be moved, i.e., keeping leaning against the edge. If a boundary is adjacent to nothing, it will follow the edge (wall) dragged by the user. Then, all the boundaries in the set are adjusted according to their lengths and the distance caused by the user (Δ𝑑 in Figure 7 ).The third operation is "Adding New Elastic Boxes, " which detects free spaces that may hold a new elastic box, adds a new elastic box, and rotates it concerning a wall. The elastic box database initially proposes an elastic box whose semantics fit the current room. Our framework places the newly added elastic box next to each room corner and checks if it conflicts with other objects. If no conflict occurs at a corner, the box is added. Figure 8 shows an example. 
experiments 6.1 implementation: 
 We developed a web-based platform to hold the proposed framework, as shown in Figure 9 . The platform supports editing or splitting room shapes. Users can freely move an edge or add breakpoints to it. At every transient moment, our platform detects a modified room shape, and our framework is executed so that objects are arranged. Our framework and platform are developed using Three.js.The platform supports editing individual objects or objects of elastic boxes, such as typical industrial solutions or other interactive scene synthesis frameworks (see Section 6.2). Users can add, delete, move, rotate or scale individual objects. Therefore, we can combine our framework with other interactive frameworks, enabling interactive convenience and customization flexibility.We utilize 3D-Front to extract the elastic boxes. Since 3D-Front mainly contains bedrooms and living rooms, we invite interior designers to create objects and their arrangements for other room types, such as bathrooms. The elastic boxes are structurally stored in our database. We can tune the strategies, e.g., changing the relative positions required between objects/walls.This experiment verifies our framework's usability and efficiency by comparisons. To our knowledge of interactive scene synthesis, there is no existing framework with exactly the same "interactive sessions" [30] . Our interactive session involves adding breakpoints and pulling edges (walls), indirectly arranging objects. Thus, we compare our framework with four baselines that interactively synthesize 3D scenes with their unique interactive sessions. We ensure the five frameworks (four baselines and ours) have the same input and output. The four baselines are:Table 1 : Time Consumption. Each row refers to a framework. Each column refers to times consumed by a particular operation. Each cell contains an average time recorded and its standard deviation in brackets. Clutterpalette [24] : Clutterpalette synthesizes a 3D scene by iteratively clicking positions, as shown in Figure 10a . A user clicks a position in each interactive session, and a set of objects appears on a UI based on the position's context. Selecting an object refines the recommended objects on the UI. When finding one, the user adds an object and continues another interactive session.MageAdd [28] : MageAdd synthesizes a 3D scene by exploring potential objects that may appear in the scene, as shown in Fig- ure 10b . In each interactive session, a user freely moves the cursor, and various objects may appear at the position pointed by the cursor. If the user finds a suitable object, the user can add the object, and the scene's contexts will change for more potential objects.SceneDirector [30] : SceneDirector synthesizes a 3D scene by simultaneously editing multiple objects, as shown in Figure 10c . In each interactive session, a user selects an object, referred to as a dominant object according to [30] . Subsequently, a set of subordinate objects appears around the dominant one. When the user moves the dominant object, other objects follow it.Industrial Solution: This denotes no "intelligent methods" involved, as shown in Figure 10d . Users follow typical industrial software such as Kujiale [9] and Planner 5D [15] . A user searches objects, adds objects, transforms existing objects or removes objects. Other "intelligent methods" sometimes suggest implausible layouts, so we enable industrial operations for all frameworks.(a) Clutterpalette [24] (b) MageAdd [28] (c) SceneDirector [30] (d) Industrial Software. We invite 30 participants. Each participant uses the 5 frameworks to generate 5 floor plans with object layouts. Since our framework arranges objects over editing floor plans, to ensure the 5 floor plans yielded from the same participant have the same room shapes, a participant first uses our framework to edit an initial floor plan (with objects arranged by our framework). Then, she/he uses each baseline to synthesize a 3D scene using the same initial floor plan. All frameworks are implemented on the same 3D scene platform, as shown in Figure 9 . The participants include professional interior designers, university students, freelancers, office workers, etc. All the participants were taught how to use our platform. A teaching video and a guide manual are available. They can freely try different frameworks until they are familiar with them. Each participant requires up to 1 hour to get familiar with the frameworks and approximately 1 hour to finish 5 floor plans. A technical staff member is nearby to answer technical questions and provide in-person instructions.The participants include 16 males and 14 females, with an average age of 24.73 years old, with the youngest age 18 and the oldest 36. The participants have a variety of professional backgrounds. 16 of them are from architecture and interior design. The participants were asked to interactively generate a floor plan until satisfied. For each generation, a participant can use only one of the above frameworks, while the "industrial Solution" has no intelligent method involved. We recorded the times consumed for the interactive synthetic processes. The time spent editing floor plans was not counted for the baselines. Because our framework was first executed, participants were more familiar with the baselines than ours, i.e., they will be more skilled in using the subsequent ones. Therefore, the time consumption was more advantageous to the baselines than ours. For different participants, the orders for using the four baselines were random.Table 1 shows the average time spent on different user interactions, including (1) navigating the scene where users adjust views to interact with scenes from different perspectives. (2) adding objects where users drag searched results into scenes or duplicate objects.(3) removing objects. (4) translating objects. (5) rotating objects. (6) rescaling objects. (7) the frameworks' interactive sessions, such as simultaneously editing multiple objects [30] . (8) other time consumption, including elaborations, considerations, misoperations, etc, which are not trackable by our system. We add timers to every unit operation to ensure each consumed time is correctly recorded, so the systematic errors are negligible. Note that we still allow participants to use operations from "industrial solutions" to give them further preferences, and counting on this time, our framework still shows significant interactive time savings.Each cell in Table 1 contains an average time and a standard deviation in brackets. Time is recorded in seconds. According to Table 1 , our framework significantly reduces the time required to craft a 3D scene. A Kruskal-Wallis H-Test shows significant statistical differences between our framework's total time and the baselines' total time, with the p-value close to 0.This experiment can also be treated as a usability study of recent interactive scene synthesis frameworks. For example, though SceneDirector [30] helps quickly arrange a set of objects concurrently, it does not arrange other objects irrelevant to the set. Thus, users need to add other objects manually. In contrast, though MageAdd [28] can be slow, it saves time in adding objects. Further discussions of the above frameworks are beyond the scope of this paper. See a supplementary video for a qualitative comparison.This experiment verifies our framework's plausibility and satisfaction. We invited the same participants in Section 6.2 to rate our framework and the baselines based on "interaction" and "firstperson score". The interaction refers to how participants feel a sense of smoothness, fluency and ease while interacting with the frameworks, with 0 being "poor feeling" and 5 being "excellent experience". The first-person score refers to how participants feel a sense of plausibility, aesthetics, and preferences in the scene they crafted, with 0 being "terrible" and 5 being "fine art".Another 13 participants are invited to rate a third-party score on the results generated from Section 6.2. The newly invited participants do not intersect with those in Section 6.2. Each participant is presented with 50 random questions. Each question contains a 3D scene generated by one of the five frameworks. The third-party scores refer to how newly recruited participants feel a sense of plausibility and aesthetics in the scene presented. The new participants include professional interior designers and university students knowledgeable about arts or interior design. The participants do not know the frameworks. The participants include 6 male and 7 female participants. Their average age is 24.46 years old, ranging from 20 to 35. 5 are from architecture and interior design.Table 2 shows the results. The "interaction" indicates that our framework maintains interactive smoothness, fluency and ease but significantly improves the interactive efficiency (Section 6.2). The "first-person score" indicates that our framework can generate results meeting user preferences since interactive scene synthesis aims to synthesize scenes incorporating user preferences [28] . The "third-party score" indicates that our framework can generate a plausible and aesthetic scene as a synthetic framework. Thus, the objects' arrangements from their elastic boxes are reasonable. 
conclusion: 
 This paper introduces an interactive framework for scene synthesis. Users can continuously edit room shapes in each interactive session, so objects are arranged accordingly. Nevertheless, several drawbacks remain for future improvements.First, as a graphical system works, our framework suffers from loading many objects concurrently. Our current optimization is caching and multi-threading, which keeps the framework smoothly operating when loading geometries and textures. However, our framework can not "foresee" elastic boxes to appear. If the framework keeps loading a few elastic boxes (e.g., loading complex models or encountering bad network traffic), nothing shall appear, even though our framework decides to add more objects. We should calculate elastic boxes' occupations and objects' arrangements to improve our framework upon adding new elastic boxes. In contrast, our framework must load all the 3D resources before the occupations and arrangements.Second, hard decorations still need to be addressed. Similar to other interactive synthetic frameworks [24, 28, 30] , our focus is still on objects and floor plans are not interactively synthesized. A complete "home decoration" still needs connections between rooms, which may conversely affect how we arrange objects, e.g., objects should not block a door between two rooms. Pipes and wiring are also currently not addressed in typical interactive/automatic scene synthesis frameworks. 
: 
 Figure 1 : We introduce a framework for synthesizing plausible scenes over interactively edited floor plans. Given an initial scene, a user can select an edge (wall) of it (a). The user can then expand (or shrink) the room shape, and our framework automatically adds (or removes) objects and continuously arranges objects (b). Users can add breakpoints (Red Box) to edit diverse floor plans (c). If a room is sufficiently large, our framework splits the room, adds new objects and arranges the existing objects (c, d, and e). Through a few operations, a floor plan that matches the user preference and contains plausible object arrangements is synthesized (f). Please see the supplementary video for interactive demos. 
 
 Appendix:  


Decision: The paper presents a novel method to synthesise 3D scene interactively. 

Even if reviewers don not have strong concerns about contributions and experimental evaluations, one reviewer has highlighted that the topic might be more suitable for Graphic and HCI venues (the paper has indeed only 28 cited papers and only a couple of them are from multimedia-related venues). The text should be improved clarifying missing information such as user study demographic and clarifying the experimental design. Two reviewers have however increase their score from borderline/weak reject to borderline accept after rebuttal phase.

The paper can be a good candidate for a poster presentation. 
Please, in the camera ready address the comments from reviewers (i..e, missing information on user study, clarifying experimental design).

Reviews: review: paper_summary: paper_summary: The authors proposed a floor plan editing framework. This work can achieve the interaction for building a floor plan including using the elastic boxes and modifying the floor plan using its floor plan modification pipeline. It can achieve a better satisfactory rates from users than SOTA works.
strength_weakness: strength_weakness: The paper proposed work is a good representation of designing the floor plan and could be helpful to in-room design. Their work also can achieve the interaction with users to be more flexible and customized. The design of the self-adjustable elastic box is interesting and should be useful for basic designs.
limitations: limitations: 1. This paper is more suitable for Graphic or HCI area conferences, not very close to the multimedia area
2. The lack of detailed explanation of the framework pipeline, for example, This work indicates the framework can merge the two rooms while one room is shrunk, as indicated in figure 3, and the elastic box to use in designs. Should add more experiments based on these designs. 
3. The experiments combined with user studies, but the represented results only compared the time consumption and two rating scores for plausible and satisfactory.  Suggest considering more questions and evaluation scenarios for user study.
4. The discussion and explanation of Table 2 is not clear enough. Suggest discussing more about results and discussion from different scenarios.
5. Section 6.1, named "Results and setup", assumes this subsection should include results and setting up details. I can only find some brief implementation details.
paper_summary: paper_summary: This paper proposes a new interactive scene synthesis method over floor plan editing for interactive scene synthesis tasks. Considering that traditional interactive scene synthesis methods mainly focus on the user's interaction with scene objects, while ignoring the user's editing needs for the original floor plan, this paper pioneered a scene synthesis interaction method based on floor plan editing, allowing users to directly edit the initial floor plan, and proposed the concept of elastic boxes to achieve the chain change of related objects during floor plan editing.
strength_weakness: strength_weakness: 1. Compared with the traditional scene interaction synthesis method, this method introduces floor plan editing for the first time, and introduces the concept of elastic boxes on the basis of plan editing to achieve real-time control of floor plan editing and scene object changes at the same time.
2. This method makes up for the lack of traditional methods in user floor plan editing needs, making the entire design process more in line with user needs. The experiments listed in this article can well support this contribution.
3. Effective scene interaction synthesis methods can effectively improve the efficiency of interior design. The method proposed in this paper not only enables users to customize the original floor plan, but also designs elastic boxes and algorithms to adjust and control the transformation of corresponding objects, greatly improving the editing efficiency of the original floor plan.
limitations: limitations: 1. Indirectly arranging objects in the scene during the plan editing process improves efficiency to a certain extent, but it also loses some customization flexibility compared to the previous object-based editing methods, such as the placement customization of a single object and the free splitting and combination of multiple objects.
2. I have some questions about the settings of the elastic boxes: 1) Can the initial elastic boxes be edited in the subsequent process, such as adding, deleting, and rearranging internal objects; 2) Can the layout strategy in the elastic box be customized in the subsequent editing process?
3. Some limitations of doors and windows are mentioned at the end of the paper.
paper_summary: paper_summary: This paper presents a scene synthesize system, named SceneExpander, with novel interactive editing support. Given an initial scene layout, the system allows users to expand, shrink, and adjust the scene layout with real-time updates of the generated layout map and rendered furniture. The proposed system is implemented as a real-time interactive system. Through real-world user studies, this paper showed that SceneExpander outperforms several related works in editing time consumption and result qualities.
strength_weakness: strength_weakness: - The proposed system provides natural and real-time feedback to the scene designer, resulting in significant total time consumption compared to related works
- The proposed system has a complete end-to-end implementation with great potential for real-world application impacts.
limitations: limitations: - Although promising results are confirming the impacts of the proposed new user interactions, the drag-and-drop design on its own is not new. This drag-and-drop interaction design is commonly used in the 2D designing process, the author should look into the difficulties of providing precise control of the edges in 3D space.
  - Leveraging contextual views, such as top-view orthogonal projection, should help ease the issue.
- If possible, the evaluation setup section should include basic demographic information of the user study participants, such as gender. The added information will help in judging whether the participants' feedback on generation quality would be biased.
- There are several minor issues related to the paper's writing and formatting:
  - Page 1, line 37: This paper generates scenes over floor plans in real time. -> The system proposed in this paper generates scenes over floor plans in real-time
  - Figure 3 should be moved to the previous page as a complementary visualization for the descriptions of system workflow in section 3.
paper_summary: paper_summary: This paper introduces an interactive framework for scene synthesis. The framework proposed by the author supports users to continuously edit the room shape in each interaction session, and supports the generation of object objects according to the room shape for corresponding arrangement.
strength_weakness: strength_weakness: 1.From the experiment, it seems that the author's platform has been set up very completely. 2.The illustrations in the paper are designed to be very intuitive and easy to understand. 3.The paper provides a good review of the literature in the relevant field and demonstrates the background and importance of the research. 4.The paper designed interactive behaviors for multiple scenarios, and conducted detailed comparisons and analyzes over time when compared with baseline methods.
limitations: limitations: 1.It is an interesting topic to allow users to modify the layout independently and choose the layout they like. However, I do not think it is novel enough and I have some doubts about the motivations proposed by the author, that is, lines 135 to 149, and I am not sure whether the author has relevant demand basis.
2.The article mentioned that the room reasoning process has been ongoing, but often the actual room layout should also consider other related factors such as pipes and wiring. I wonder if this reasoning process takes this into account. In addition, users often make independent choices when considering room layout, that is, they want a certain space to be a bedroom or a kitchen. I don’t seem to see this room selection being taken into account in the editing settings. If without those considerations, this work seems meaningless and need to be optimized.
3.I am curious about how the size of the elastic box is set. As mentioned in Figure 8, the author's framework places the newly added elastic boxes next to the corners of each room. Although this strategy can ensure that there is no obstruction between the elastic boxes, taking the flex box in Figure 8 as an example, will the frame setting in which the flex box where the sofa is located not intersect with other elastic boxes hinder more possible arrangements of the sofa itself in this layout?
4.I am not very clear about the author’s experimental design in the 6.2. If participants are asked to first use the author's framework to edit the initial floor plan, and then let other baseline methods use the initial floor plan to synthesize the scene, will it affect the participants' familiarity with the author's framework and the familiarity with other baseline methods on this initial floor plan? My problem may come from my failure to understand the author's experimental design. Please describe and explain it in more detail.