ScenePhotographer: Object-Oriented Photography for Residential Scenes

Authors: shao-kui  zhang
jinghuan  chen
zhike  peng
ziyang  chen
yong-liang  yang
song-hai  zhang
hanxi  zhu
xuebin  chen
  scenephotogra
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-03-01 07:41:46

Abstract: Humans understand digital 3D scenes by observing them from reasonably placed virtual cameras. Selecting camera views is fundamental for 3D scene applications but is typically manual. Existing literature on selecting views is based on regular or polygonal room shapes without focusing on the objects in the scene, resulting in poorly composed views concerning objects. This paper introduces ScenePhotographer, an object-oriented framework for automatic view selection in residential scenes. Potential object-oriented views are yielded by a learning-based method, which clusters objects into groups according to objects' functional and spatial relationships. We propose four criteria to evaluate the views and recommend the best batch, including room information, visibility, composition balance, and line dynamics. Each criterion measures the view according to its corresponding photography rule. Experiments on various room types and layouts demonstrate that our method can generate views focusing on coherent objects while preserving aesthetics, leading to more visually pleasing results.

Main paper: 
 introduction: 
 Research topics related to virtual 3D scenes have significantly progressed in the last decades, such as scene synthesis [13, 38, 39, 43] , scene reconstruction [10, 24, 29] , etc. A virtual camera view comprising camera position, orientation, field of view (FoV), and aspect ratio must be carefully selected to better render scenes for human observation and perception.For instance, in terms of interior 3D scene synthesis, selecting an informative view is one of the most fundamental ways to understand scenes. Typically, researchers fix the view in a predefined top-down direction to showcase and assess the layout results [16, 37, 40] . However, top-down views may not be intuitive for scene understanding, especially the objects therein. As such, interior decoration companies often show their products through eye-level photos. Therefore, automatic view selection contributes to intuitive scene perception for humans.In addition, rendered scene images can serve as the training data for computer vision tasks, such as semantic segmentation [45] . The selection of views affects the performance of the machine learning model trained on the view-based dataset. Some methods generate training data by selecting views [17, 19] , demonstrating that good views contribute to high-quality datasets for deep learning. For furniture retailers, selecting informative and aesthetic views for furniture may leave a lasting impression on potential customers. However, manually taking photos of each product is time-consuming, and it can be difficult for non-experts to specify a desired view. This highlights the commercial significance of automatic view selection. 1 Please refer to a supplementary video for an overview of our method.Nevertheless, only limited approaches to automatic view selection have been proposed. Existing literature primarily focuses on opting views of 3D models [4, 6, 12] , with few considerations on object relationships. Moreover, generating views for a 3D model depends on the shape's regularity (e.g., continuity, symmetry), which is unavailable for complex scenes. For scene-level view generation, Zhang et al. [44] introduced a framework that automatically selects informative and aesthetic views for 3D scenes based on interior photography rules. It provides reasonable views based on the shapes of the walls. However, this work needs to consider objects in the scene, such as their positions, types, and interrelations. Thus, it is limited to an intuitive understanding of objects in the scene.This paper proposes an object-oriented method for automatic view generation in interior scenes. As shown in Fig. 1 , our method proposes views for individual object groups in the scene where the objects in the same group are functionally relevant. We consider functional relevance between objects (e.g., a bed and a wardrobe), ensuring that a set of semantically related objects are displayed in the appropriate positions under the same view. As a result, the views generated by our method better exhibit objects and their relations while preserving scene aesthetics.To generate an object-oriented view, we first group the furniture objects suitable for concurrent observation using an agglomerative clustering algorithm (Section 3). Aiming to divide various objects in indoor scenes functionally, we consider the spatial distance between objects and their quantified "category" differences as the basis for classification. Subsequently, our method generates potential views based on the clustered object groups (Section 4). By applying residential photography rules to the grouping results, such as One-Point Perspective (OPP), Two-Point Perspective (TPP), and Golden Section [20] , we can generate a series of appealing views.However, not all of them are aesthetic and practical. Hence, we further assess and filter the generated views by four criteria from two aspects. First, views are measured by how many object parts are captured (Section 5.1) and how broad the horizon is (Section 5.2) to reveal more scene information. Second, views are measured by how balanced objects are distributed on the canvas (Section 5.3) and how much tension is created by visible objects (Section 5.4) to preserve scene aesthetics.Extensive experiments on various room types and layouts were conducted to evaluate our method. We compare the views generated by our method, those generated by SceneViewer [44] , and those manually created by professional photographers by gathering feedback from human participants (Section 6.2). We also conduct an ablation study to assess the validity of the four criteria (Section 6.3). Moreover, our dataset is collected from an interior design company with thousands of exclusive stores, where our method has been successfully used in real applications for proposing views.Our technical contributions are as follows:• We present an object-oriented method for automatically recommending views in residential scenes. • We propose an algorithm for grouping functional objects based on their spatial distance and semantic category. • We formulate a set of evaluation criteria to assess the informativeness and aesthetics of views. 
related works: 
 View Selection for Objects. View Selection for 3D objects was addressed through several approaches. View entropy was proposed [35] to recommend visual-pleasing and informative views of a single CAD model (e.g., a teapot) or an isolated group of related models (e.g., a desk with a reading lamp and a chair). A benchmark for evaluation of best view selection was introduced [11] , but it concentrated on selecting views for a single 3D model presented in triangular meshes. Various methods were proposed for 3D volume [5, 22, 27, 32] . Tao et al. [33] offered a method for selecting views of streamlines. Vázquez proposed a novel method to automatically select an informative view of 3D objects through depth-based view stability analysis [34] . Bruce et al. presented an algorithm based on heuristic compositional rules to find the views with suitable compositions, but the method was toward 3D objects [18] . There is an informative survey [4] for additional studies regarding the view selection for 3D models.View Selection for Scenes. In view selection at the scene level, several methods were proposed for an optimal view of 3D scenes. Early work created a graphical interface to find the desired position of the virtual camera given the frame constraints [3] . A photo assistant was achieved to optimize camera parameters from an initial view to match users' demands on composition [2] . Ji et al. [23] formulated metric functions for the information and aesthetics of views and optimized a set of aesthetically pleasing views through a simulated annealing algorithm. Camera control in cinematography was also explored [7] . Daniyal et al. [8] proposed a method to select the best view of several cameras in the real world by comparing video content scores and human activities.Furthermore, a recent work applied the rules of residential photography in automatic view selection of the whole scene [44] . However, the rules only concern the shape of walls, which is inconsistent with objects' layouts. In contrast, our method focuses on the functional groups in the room. We distinguish the related objects in the scene and apply rules of residential photography to the groups, automatically recommending aesthetic object-oriented views.3D Scenes. View selection appears in topics related to 3D scenes. For scene synthesis, it is frequent to select a view to visualize the synthesized results [41] [42] [43] . For example, user studies need the views to assess the results. For computer vision, view selection generates high-quality images utilizing the views, thus yielding training datasets [17] . Handa et al. [19] generated a training dataset for computer vision through different views from 3D scenes. Luo et al. [25] evaluated 3D scenes using views derived from 3D scenes. Our method automatically recommends views for various scenes, facilitating the related computer graphic and vision studies. 
object clustering: 
 In order to find high-quality views at the object level, we first divide indoor objects into groups suitable for observation in the same view. In interior scenes, functionally related furniture objects, such as dining tables and chairs, sofas and coffee tables, etc., may be located close to each other. An agglomerative clustering algorithm is adopted to build a hierarchical organization of objects [1, 28] . However, we alter the definition of "distance" between classes. Suppose 𝑖 and 𝑗 represent two objects.⃗ ⃗⃗⃗ ⃗ 𝑝 𝑖 and ⃗ ⃗⃗⃗⃗ ⃗ 𝑝 𝑗 are their positions. 𝑡 𝑖 and 𝑡 𝑗 are their types/categories. The distance between object i and object j can be formulated as in Equation 1, where ∥ ⃗ ⃗⃗⃗ ⃗ 𝑝 𝑖 -⃗ ⃗⃗⃗⃗ ⃗ 𝑝 𝑗 ∥ is the spatial distance, and 𝛾 (•) is the "distance of type".𝐷 (𝑖, 𝑗) = ∥ ⃗ ⃗⃗⃗ ⃗ 𝑝 𝑖 - ⃗ ⃗⃗⃗⃗ ⃗ 𝑝 𝑗 ∥ + 𝛾 (𝑡 𝑖 , 𝑡 𝑗 )(1) Distance of type is based on the probability of two object types appearing in the same functional group. The greater the likelihood that they are considered to be in the same group, the closer they are to the clustering. For example, a dining table and a dining chair are likelier to be in the same group, so their 𝛾 (𝑡 𝑖 , 𝑡 𝑗 ) is lower than that between the dining table and a bunk bed.We utilize the 3D-Front dataset [14, 15] to obtain the co-occurrence between categories. First, we annotate the objects in the dataset and identify the objects that can be grouped. This allows us to create a dataset of grouped objects, as shown in Fig. 2 , where each item contains a group of objects serving the same function.Using the data above, we calculate the co-occurrence of categories in the same group. T (𝑡 𝑖 , 𝑡 𝑗 ) represents the co-occurrence of 𝑡 𝑖 and 𝑡 𝑗 . We formulate the distance of type as Equation 2. If an object appears once, we ignore the occurrence with itself.𝛾 (𝑡 𝑖 , 𝑡 𝑗 ) = -ln T (𝑡 𝑖 , 𝑡 𝑗 )(2) We use the negative logarithm to correlate the distance with the co-occurrence frequency and ensure that it has a similar magnitude to the spatial distance. The "distance of type" heatmap is shown in Fig. 3 . A darker colour indicates a closer relationship between the two categories. Some grouping results are shown in Fig. 4 . ⃗⃗ 𝜁 is the camera position, which is a three-dimensional vector.⃗ ⃗ ⃗ 𝛽 is the camera direction, a three-dimensional normalized vector, i.e., ∥ ⃗ ⃗ ⃗ 𝛽 ∥ = 1. 𝜃 and 𝑟 are the camera horizontal field of view (HFoV) and aspect ratio, respectively. 𝑟 = 𝑊 /𝐻 is the ratio of image plane width 𝑊 and height 𝐻 . For simple notations, we omit the subscripts indexing to individual views.We explore object groups' front and side views to achieve the OPP and TPP. The front view ensures that most objects are perpendicular or parallel to ⃗ ⃗ ⃗ 𝛽, so the rendered image has a single visual vanishing point [20] .⃗⃗⃗⃗⃗ 𝜌 𝑖 represents the center of the object group Ω 𝑖 's bounding box. We generate four candidate views for each Ω 𝑖 , i.e., front, back, left and right. The four directions determine ⃗ ⃗ ⃗ 𝛽.⃗⃗⃗⃗⃗ 𝜌 𝑖 and moves back along -⃗ ⃗ ⃗ 𝛽. The camera goes back until it touches an object or a wall. Fig. 5 shows a front view.The side view makes objects and the camera form a certain angle, e.g., 45 • , so the rendered image has two vanishing points [20] . In the wall-based view selection, the best position of a camera is located at one of the trisection points of a wall towards the opposite wall [20] . One motivation is that the front wall should occupy a larger area in the canvas than the side wall [44] . In Fig. 6 , ⃗ ⃗ ⃗ 𝛽 is parallel to the EQUATIONThe aspect ratio 𝑟 is usually a design decision in interior photography. It determines the proportion of the ceiling and floor in the image, generally chosen between 𝑟 1 = 4 : 3 and 𝑟 2 = 16 : 9. If a photographer wants to focus on furniture, 16 : 9 is picked. 4 : 3 allows more ceiling and floor when other parameters are fixed. Fig. 8 shows the difference. Note that other aspect ratios can be used for the trade-off of including more ceilings/grounds [26] , but this is beyond the scope of this paper.According to the advice of professional photographers, one way to determine 𝑟 is by calculating the width-to-height ratio of the wall in front of the camera and choosing 𝑟 1 or 𝑟 2 based on closeness.Λ ⃗ ⃗ ⃗ 𝜁 , ⃗ ⃗ ⃗ 𝛽,𝜃 𝑘represents the visible span of wall Λ 𝑘 given three parameters. 𝐻 represents the room's height, and 𝑛 𝑘 represents the normal vector of Λ 𝑘 . Based on the above advice, 𝑟 is defined in Equation 4, where EQUATIONEQUATION5 View Evaluation 
room information: 
 Room information 𝑆 𝑜 measures how many objects can be seen and how complete each object is. Existing works [35, 36, 44] count the number of object primitives in a view (i.e., check whether the view captures an object primitive). However, the geometric complexity of objects is different. A large object can be geometrically simple with few primitives, but it occupies a prominent part of the view. Thus, we measure the room information incorporating object volume. Similar to HAISOR [31] , each object 𝑜 𝑖 ∈ O in our method is recognized as several parts 𝑎 𝑖,𝑗 ∈ 𝑜 𝑖 , e.g., handrails, sliders etc. O = {𝑜 1 , 𝑜 2 , ..., 𝑜 𝑁 𝑜 } contains the objects in the room. Calculating detailed part geometry is too heavy, so we use a bounding box to represent each part. Precisely, the room information 𝑆 𝑜 is measured by Equation 6, taking as input a view c and the objects in room O.EQUATION𝑓 𝑜 (c, 𝑎 𝑖,𝑗 ) returns the number of bounding box's vertices in 𝑎 𝑖,𝑗 seen by view c. 𝑓 𝑜 tests each point and counts the number of points passing the tests. It first tests if the point is projected on the camera frustum's horizontal and vertical visual planes, which are determined by the FoV 𝜃 and aspect ratio 𝑟 . Then, it tests whether occlusions exist between the camera position ⃗⃗ 𝜁 and the part 𝑎 𝑖,𝑗 . 𝑉 (𝑎 𝑖,𝑗 ) calculates the volume of the given part 𝑎 𝑖,𝑗 , i.e., how large an object part is. Therefore, if an object is sufficiently large and mostly visible, the object will largely contribute to the room information. The denominator in Equation 6is the distance between the view position ⃗⃗ 𝜁 and the part position P (𝑎 𝑖,𝑗 ), indicating that objects closer to the camera appear larger and vice versa. Generally, if a view captures sufficient objects and the objects are sufficiently large on the image plane, the view's room information will be high.Fig. 9a illustrates the intuition. With more volumes of large objects captured, an informative view can be selected.According to [21] , a view should be set as far as possible, introducing a sense of room "depth". In the rendering process, the view casts rays to its image plane, where the "depth" is determined by each ray's first hit of the scene. Ideally, most rays would hit walls and objects, while some may travel a sufficiently long distance until a far corridor, window, or door is hit. The sense of room depth needs to be larger under this criterion.The visibility 𝑆 𝑣 is measured by Equation 7. It is formulated as an integration over the image plane, where 𝑤 and ℎ define a ray originates from the camera position ⃗⃗ 𝜁 and passes through a position (𝑤, ℎ) on the image plane.𝑓 𝑣 (•) is the ray-casting operation in computer graphics. Therefore, 𝑓 𝑣 (•) returns the closest ray-scene intersection point, and ∥ • ∥ is the distance. q is the hyper-parameter greater than 1. It penalizes the casted positions too close to the camera while rewarding the rest.Fig. 9b shows the intuition of visibility. A visible view can be selected with more rays penetrating deeper into a room.𝑆 𝑣 ( ⃗⃗ 𝜁 , O ∪ R) = ∫ 𝑤,ℎ (∥ ⃗⃗ 𝜁 -𝑓 𝑣 ( ⃗⃗ 𝜁 , 𝑤, ℎ, O ∪ R)∥) q d𝑤dℎ (7)Composition balance measures how objects deviate from the view center. According to the law of symmetry in gestalt psychology, a view with objects evenly distributed on both sides enhances the coordination of the user's perception [30] . An extreme case with total imbalance is when all objects are captured on the left side of the image. Since most objects in residential scenes are near the ground, we mainly measure the horizontal deviations [26] . The composition balance 𝑆 𝑏 is measured by Equation 8, where objects are not split into parts. It is a weighted sum concerning object volume 𝑉 (𝑜 𝑖 ), i.e., the imbalance of a large object affects 𝑆 𝑏 more than that of a small object. 𝑓 𝑏 (c, 𝑜 𝑖 ) measures the signed distance from an object to the image center as shown in Equation 9, where I (𝑜 𝑖 ) returns the projected positions of object 𝑜 𝑖 on the image, 𝑥 takes the horizontal entry, and 𝑊 is the image width. 𝑓 𝑏 (c, 𝑜 𝑖 ) is further scaled by the distance between the view position ⃗⃗ 𝜁 and the object position P (𝑜 𝑖 ), indicating that distant objects have less impact on 𝑆 𝑏 . The final exponential operation 𝑒𝑥𝑝 (-(•)) ensures that 𝑆 𝑏 decreases as objects become imbalanced since a perfectly balanced image yields 0 of a weighted sum over 𝑓 𝑏 (c, 𝑜 𝑖 ).EQUATION)EQUATIONFig. 9c shows the intuition of composition balance. A view is balanced if objects are equally distributed between the vertical visual plane concerning their volume. Line dynamics is one of the most significant mechanisms to create a sense of space in views [9] . The impact of a view is enhanced when its borders are broken at oblique angles by sufficient lines, which refer to objects' outer contours in 10. It counts the number of lines intersecting the vanishing points, while each line is assigned a weight indicating its contribution to the expression.EQUATIONEQUATION𝛼 (𝜏 𝑘 ) addresses the expression in Equation 11. It first calculates the minimal angle between line 𝑙 𝑘 and the four 𝜋/4 lines. If line 𝑙 𝑘 aligns perfectly with a 𝜋/4 line, then cos(2(•)) = 1 indicating a total contribution to 𝑆 𝑙 . If line 𝑙 𝑘 is "upright", i.e., being perpendicular to the ground, then cos(2(•)) = 0 even if 𝑙 𝑘 is sufficiently close to the main vanishing point as shown in Equation 12. Since a view may have more than one vanishing point, we define v = (v 𝑥 , v 𝑦 ) as the main vanishing point with most lines intersecting. As lines may have multiple vanishing points (i.e., lines are not parallel), the view's vanishing points are the means of each line's vanishing points. Equation 12, therefore, calculates the distance from line 𝑙 𝑘 to v. If line 𝑙 𝑘 is sufficiently close to v concerning a threshold 𝛿, line 𝑙 𝑘 contributes to 𝑆 𝑙 . Otherwise, line 𝑙 𝑘 does not contribute even though Equation 11 returns a non-zero value.Fig. 9d shows the intuition of line dynamics. A deeply satisfying, resolved feeling is produced with more objects contributing their lines perpendicular to the image plane. 
experiments 6.1 setup and results: 
 The dataset is provided by Guangzhou 3D eXtremity Technology Company with over 1000 dealers and 2500 exclusive shops for residential home customization. The dataset includes floor plans, furniture objects, and their arrangements created by professional interior designers for commercial purposes. The threshold in our agglomerative clustering algorithm is set to 𝑆/3 + 4.5, where 𝑆 represents the larger value between the room width and length.𝛿 𝑎 in Equation 3 is 10 • . 𝛿 𝑙 in Equation 10 is 1.0×10 -3 . The weight of room information is 1.0. The weight of visibility is 0.5. The weight of composition balance is 10.0. The weight of line dynamics is 2.0. The criteria weights primarily uniformize various scales. All criteria are effective with a balanced scale. The parameters are not data sensitive because indoor scenes and furniture objects generally have the same scale of spatial dimensions.In order to improve the efficiency of the visibility test, we simplify the integration by summing up evenly spaced samples (15-pixel intervals in our implementation) on the canvas.As shown in Fig. 10 , our method generates aesthetically pleasing views focusing on coherent objects and can be applied to various room types. Our method is successfully integrated into the actual application of the interior design company. When users search for rooms or floorplans, the thumbnails are rendered according to our views. The interior designers also use our method to automatically generate views after designing a new scene. Our method also applies to 3D-Front [14] , one of the latest and most fundamental datasets for scene synthesis [31] . Please see the supplementary materials for the results generated from 3D-Front. A 3D-Front room with 6-10 objects consumes less than 10 seconds to generate views, which is much more efficient than manual view selection.We conducted a user study to evaluate the effectiveness of our method by comparing it with views generated by professional photographers and SceneViewer [44] . We invited 6 photographers with solid interior design and photography backgrounds to generate images manually. They were asked to adjust the visual camera to find the best views interactively. Overall, we obtain 92 such images.To evaluate the results, we invited two groups of participants to compare views generated by our method, photographers and SceneViewer. The first group, referred to as the "professional group", consists of professional photographers with backgrounds similar to those who created the manual views. The second group, referred to as the "third-party group", consists of university students knowledgeable about arts, photography, or interior design. Each newly recruited participant was presented with a questionnaire. Each Living Rooms, Dining Rooms and Kitchens.Bedrooms, Dressing Rooms and Kids Rooms. question displayed three images generated in different ways for a specific room, where the images were shown in random order. Participants compared three images and rated them based on aesthetics, informativeness, and overall quality. The aesthetics refers to how participants feel a sense of beauty in images, with 0 being "terrible taste" and 5 being "fine art". The informativeness refers to how participants receive sufficient details in images, with 0 being "no idea" and 5 being "rich details". The overall quality asks for a comprehensive rating, with 0 being the worst and 5 being the best.For each questionnaire, rooms were randomly assigned, and the order of the rooms was also random. For the professional group, 5 photographers were invited. Each photographer was assigned more than 500 questions (taking approximately 8 hours to finish), and we achieved 2933 scores for each image. For the third-party group, 31 knowledgeable participants were invited. Each one was assigned 45 questions, and we achieved 1395 scores.The comparison results are shown in Table 1 for the professional group and Table 2 for the third-party group. We noticed that the professional group is stricter than the third-party group in evaluating the views. However, both groups indicate that, though the photos taken by professional human photographers are better than ours, our method performs better than the prior art on 3D scene photography [44] and is competitive with photographers. Several Kruskal-Wallis Tests show that the differences between methods (e.g., Ours versus SceneViewer) are statistically significant, with the p-values being extremely close to 0. Fig. 11 shows a few qualitative comparisons. SceneViewer [44] largely depends on walls to yield views. Thus, it is hard for SceneViewer to address irregular shapes. Our object-oriented method helps locate plausible camera positions to avoid obstruction of walls. It directly focuses on object groups under photography constraints, e.g., selecting views with good object composition. Professional photographers may remove a wall or an object to acquire more space for better view specification.We conducted an ablation study to assess the validity of the four criteria. After completing the previous experiment, the participants Table 1 : Professional evaluation. The participants are professional interior photographers. Each cell contains an average score and a standard deviation. Each row refers to a baseline. "Human" refers to professional photographers in Section 6.2. were asked to compare the final results with views without considering one of the criteria. Each participant conducted a questionnaire, where each question had two unordered views from a randomly selected room, including one view taking into account all criteria and the other with one of the criteria ablated. We also asked each participant for ratings regarding each view's aesthetics, informativeness, and overall quality.The results are shown in Table 3 . The views with all criteria outperform those ablating one criterion, indicating that the proposed criteria are all necessary to yield plausible views. Kruskal-Wallis Tests show that the differences between "all criteria assembled" and "discarding a criterion" are statistically significant.We notice that removing the visibility largely affects the results as visibility requires sufficient space in front of the camera. Without considering visibility, a few objects will be placed out of space, resulting in a nearly empty view. We also find that ablating a specific criterion influences both aesthetics and informativeness. This is understandable because aesthetics and informativeness are often coupled. For instance, an aesthetic view can be attributed to wellselected objects, which is also informative. 
conclusions: 
 We present ScenePhotographer, a novel method for photographing digital 3D scenes based on object clustering, view proposal, and view selection in an object-oriented manner. Extensive evaluations demonstrate the effectiveness of our method for generating plausible views of 3D scenes. Our codes are publicly available 2 . The view selection criteria can be used as a general function to evaluate scene views quantitatively. We expect our work to further contribute to applications based on 3D scenes and their 2D images. While achieving quality results, our method has some limitations. First, our work does not explore all possible views as it is computationally prohibitive. Physical entities such as walls and furniture objects may not be easily moved. Hence, the space is often limited when placing the camera. However, virtual scenes can be easily moved or even removed if needed. Some professional photographers took photos with wall removed such that more spaces are available for a better camera position (i.e., c = ( ⃗⃗ 𝜁 , ⃗ ⃗ ⃗ 𝛽, 𝜃, 𝑟 ) in Section 4). Fig. 12 shows two photos taken by removing walls, which is particularly helpful for displaying small rooms. Also, by interviewing the professional photographers in Section 6.2, they suggested that we should prevent the camera from being too close to the objects, i.e., the distance from the camera to a particular object group should be above a threshold. This is because our method enlarges FoV when getting closer to the objects. However, a large FoV potentially causes distortions, i.e., furniture objects may be distorted in the image plane, as shown in Fig. 13 . Besides, a view should present a sense of space. Being too close to an object diminishes the surrounding space, thus making it difficult to display the whole scene. 
: 
 We present an object-oriented method for automatic view selection in residential scenes. Based on spatial and semantic proximity, we first partition the furniture objects in the room into several groups suitable for observation in the same view (Top-Left). After clustering, we probe one or multiple groups' front and side views to get the potential view set (Top-Middle). Ultimately, we formulate four criteria (Top-Right) to evaluate and select the optimal views (Bottom). 
 
 Appendix: https://github.com/Shao-Kui/3DScenePlatform#scenephotographer. 


Decision: The paper proposes an automatic camera view selection method for digital residential scenes. It extends a previous work [44] via object clustering and aesthetic criteria. The reviewers agree that the paper is well-written and the proposal is technically sound. Moreover, the authors have adequately responded to the reviewers' concerns in the rebuttal, including an extended discussion of the possible limitations. Although one of the reviewers is concerned with the paper's suitability for the scope of the conference, he/she has recognized that it might be interesting at least for part of the ACMMM audience. Thus, after a thorough evaluation by the reviewers and considering the authors' detailed rebuttal, I recommend accepting this paper as a poster presentation at ACM MM 2024.

Reviews: review: paper_summary: paper_summary: The paper presents a methods for automatic view selection when taking photograph of a 3d residential scene. It builds on top of a previous work [44] with two additions: (1) objects clustering so that objects with similar functionality will be seen from a similar view, (2) four additional aesthetic criteria for further selection from candidate views. 

The work seems to be one reasonably nice step further on top of [44]. My major concern is that the paper does not fit the conference so much, as it is pretty much unimodal.
strength_weakness: strength_weakness: + The idea of object clustering based on functionality is promising. It further considers the relationship among the objects in the scene from a semantic point of view, which might better match human's intention in certain applications (e.g., interior design).
+ The four proposed criteria seems to work properly, although it is hard to judge whether these are somewhat the optimal group of criteria.
limitations: limitations: As mentioned earlier, the main concern is the mismatch of focus of the paper versus the scope of the conference.
paper_summary: paper_summary: This paper introduces a method for automatic view generation in synthetic residential scenes. The method integrates functionality with photographic principles to produce informative and aesthetically pleasing views. The method consists of three core stages: firstly, object clustering is performed to identify groups of objects that can be captured from a single viewpoint and that correlate with each other. Secondly, using photographic rules and the identified object groups, potential views are proposed. Finally, each view is evaluated based on predefined quality metrics, and the best views are selected.
strength_weakness: strength_weakness: Overall, I find this paper to be very well-executed, with good results and a well-explained method. 

In the following, I list the strengths of the paper:
- Each component of the method is well justified and thoroughly motivated
- The approach is clearly and precisely explained, with simple equations supported by informative visuals.
- The method is simple. The paper provides enough implementation details (+ code is released).
- A comprehensive user study significantly demonstrates its superiority over state-of-the-art (SceneViewer).
- An ablation study validates the relevance of each „view evaluation metric.“
- The supplementary material contains extensive results, and the video is clear, providing a comprehensive overview.
- While each part in isolation might not be highly novel, their combination presents sufficient novelty, making the approach valuable for the community.
- I see great potential for generating improved synthetic datasets, where information is maximized and redundancy minimized.
limitations: limitations: The paper has only a few limitations that could be addressed in a revised version to enhance its comprehensiveness:

- Direct comparisons between ScenePhotographer, SceneViewer, and human-generated views are missing. Fig. 11 shows comparisons on different scenes, but it would be more insightful to see how ScenePhotographer improves the view within the same scene.
- It would be interesting to provide more insights about the professional camera choice and ScenePhotographer; how much do the parameters deviate from each other on the same scene? 
- It would be valuable to see how the „view evaluation metrics“ assess the views created by SceneViewer and professional photographers. Can the proposed metrics serve as a general evaluation function? For instance, do professional views not only receive higher scores in the user study but also achieve higher evaluation scores using these metrics?
- A related question is whether the method is limited by the potential view generation or by the quality of the view evaluation metrics.

minors:
- Fig. 9: An overlay between the image and mask could improve clarity. For example, it was challenging to clearly connect each dot to the corresponding object in Fig. 9(a)
- Fig. 3: Why does the diagonal of the heatmap not show a smaller „distance of type“? Is it because some objects do not appear multiple times in the same scene? 
- Fig. 3: It would be more informative to show the heatmap between 4-5 objects instead of the full heatmap without any labels or add the class labels.
paper_summary: paper_summary: This paper proposes an automatic selection method for camera views that are likely to be appealing in digital residential scenes. The proposed method generates multiple view proposals showing objects and their relationships, and selects the view with the maximum score calculated by four criteria: room information, visibility, composition balance, and line dynamics. Experiments show that the proposed method achieves better scores than the conventional method based on aesthetics, informativeness, and overall quality in user studies.
strength_weakness: strength_weakness: 1. The paper is well-written, and the overall idea is easy to follow.
2. The problem addressed is an important and interesting one. While much previous work generates views based on room shapes, this paper focuses on the functional and spatial relationships of objects to generate views.
3. The proposed method achieved the desired results on a dataset provided by an interior design company. Furthermore, the ablation study demonstrated the validity of the proposed four criteria.
limitations: limitations: 1. The proposed method requires floor plans, furniture objects, and their arrangements and type labels in digital residential scenes. This rich information is not necessarily available for every scene.
2. Object clustering in the proposed method relies on heat maps of distances of types built in advance, and the number of possible types is 35 according to Figure 3. The objects to which the proposed method can be applied are limited to those present in this heat map.
3. Experiments lack verification of the influence of various parameters of the proposed method, such as room information weights. The authors should explain how the values of these parameters were determined.
4. In Section 1, the authors highlight the importance of automatic view selection by pointing out the time-consuming nature of manual view selection. Nevertheless, the experiment does not evaluate the consumption time of view selection.