Diversity-Driven View Subset Selection for Indoor Novel View Synthesis
Authors: Zehao Wang
Han Zhou
Matthew B. Blaschko
Tinne Tuytelaars
Minye Wu
Venue: Accepted by TMLR ()

Abstract: Novel view synthesis of indoor scenes can be achieved by capturing a monocular video sequence of the environment. However, redundant information caused by artificial movements in the input video data reduces the efficiency of scene modeling. To address this, we formulate the problem as a combinatorial optimization task for view subset selection. In this work, we propose a novel subset selection framework that integrates a comprehensive diversity-based measurement with well-designed utility functions. We provide a theoretical analysis of these utility functions and validate their effectiveness through extensive experiments. Furthermore, we introduce IndoorTraj, a novel dataset designed for indoor novel view synthesis, featuring complex and extended trajectories that simulate intricate human behaviors. Experiments on IndoorTraj show that our framework consistently outperforms baseline strategies while using only 5–20% of the data, highlighting its remarkable efficiency and effectiveness.

Published in Transactions on Machine Learning Research (05/2025)
Diversity-Driven View Subset Selection for Indoor Novel View Synthesis
Zehao Wang
zehoo. wwwaaaaattttttuuuuueeeeeee
ESAT Processing Speech and Images
KU Leuven
Han Zhou
ESAT Processing Speech and Images
KU Leuven
Matthew B. Blaschko
matthew. hwaghheww.bascchcoooeeeee...eeed
ESAT - Processing Speech and Images
KU Leuven
Tinne Tuytelaars
ESAT Processing Speech and Images
KU Leween
Minye Wu
minge. www.gyeewwweesat..mm www.wettt..uuuue..nn...mm kulewwen.. be
ESAT Processing Speech and Images
KU Leuven
Reviewed OpenReview: /ttps://openrevieee net/ forum? id= id=#42CffcP30
Abstract
Novel view synthesis of indoor scenes can be achieved by capturing monocular video sequence of the environment. However, redundant information caused by artificial move- ments in the input video data reduces the efficiency of scene modeling. To address this, formulate the problem as combinatorial optimization task for view subset selection. In this work, we propose novel subset selection framework that integrates a comprehensive diversity-based measurement with well-designed utility functions. We provide a theoretical analysis of these utility functions and validate their offectiveness through extensive experi- ments. Furthermore, we introduce INDOORTRAJ, novel dataset designed for indoor novel view synthesis, featuring complex and extended trajectories that simulate intricate human behaviors. Experiments on INDOORTRAI show that our framework consistently outperforms baseline strategies while using only 5-20% of the data, highlighting its remarkable efficiency and effectiveness. The code is available at: htttps://githhhe com zehao http://ehhwwww//ndooooo wang/IndoorTra.
1 Introduction
Novel view synthesis using monocular RGB camera in indoor environments holds significant practical value for real-world applications. This technology allows users reconstruct indoor scenes from self-captured video data, making it particularly beneficial for VR/AR applications, such as virtual tours. However, these vidcos often contain lengthy sequences with redundant information caused by complex camera movements resulting from human actions. This redundancy makes scene modeling less efficient, leading to longer processing times. It crucial develop methods that efficiently select subset of frames within memory constraints. These target methods aim maintain novel view synthesis performance that is comparable using the full frame set, while significantly reducing memory and computation demands.

---

Published in Transactions on Machine Learning Research (05/2025)
In this paper, we formulate this problem a combinatorial optimization task that aims to maximize novel view synthesis quality with a fixed subset size K. The function z(S) serves as our utility function to measure the novel view syuthesis quality, where S is a subset of the full set D. This subset selection problem is expressed as follows:
max (=(S) |S| S K) SCD
(1)
The quality of novel view synthesis is typically evaluated using the Peak Signal-to-Noise Ratio (PSNR) and related metrics applied to synthesized 2D images. However, computing these metrics requires expensive reconstruction and rendering for each chosen subset S. Moreover, selecting subset of size K that maximizes performance among all possible combinations poses an NP-hard challenge. All of these computational complexity make it impossible to solve this subset selection problem in practice.
From the literature, several measures have been identified as useful for assessing reconstruction quality. Warping consistency (Ye et al. 2024) and reconstruction uncertainty (Wu al. 2015 Goli et al., 2024 Pan
et al. 2022) emphasize per-camera importance and are effective for Next-Best- View (NBV) selection tasks. The calculation of these measures no longer requires completing the full reconstruction process; they can be computed during several iterations of the reconstruction, significantly reducing computational intensity, Nevertheless, they still require substantial training and inference efforts, making them less idcal for the effi- ciency requirements of subset, selection problems. Another category of measures, focusing on view diversity, is commonly used in SLAM and neural SLAM tasks (Raúl Mur-Artal & Tardós 2015 Sucar et al.) 2021 Yang et al. 2022 Zhu et al. 2022). These diversity- based methods focus on geometric criteria such as distance and frustum overlap, but fail to capture angle variation and scene semantics comprehensively. Building on these insights, we propose an efficient multifactor distance measure that combines pairwise distances across 3D, angular, and semantic spaces. This measure jointly promotes spatial coverage, directional diversity, and content variation, ensuring the selected views encompass wide physical area, varied perspectives, and diverse scene content to better support view selection and scene reconstruction.
Given the distance mcasure and objective of our task, we define two utility functions 2(S) and apply an efficient greedy algorithm to maximize z(8) by iteratively optimizing its marginal contribution. The first utility function is adopted from determinantal point process (DPP) (Kulesza et al. 2012), probabilistic model known for its effectiveness in selecting diverse subsets. The second is inducted from diversity related heuristics used in recommender system (Carbonell & Goldstein 1998) and NBV (Xiao et al. 2024 selection. While these heuristics have shown good empirical performance in other tasks, their effectiveness in the view subset selection problem has not been thoroughly explored. Additionally, we analyze a coverage- utility function derived from Kopanas & Drettakis (2023). We reformulate the first two utility functions using our distance measure and assess the effectiveness of all three both theoretically and empirically.
Existing indoor scene datasets often feature fixed camera movement speeds or relatively short and simple tra- jectories, such as those in Straub et al. (2019): Zhu et al. (2023). To enable more comprchensive experiments that better simulate complex human capture behaviors, we introduce a new dataset called INDOOR TRAJ. Experimental results on INDOORTRAL show that the best strategy derived from our framework consistently outperforms baseline strategies when using 5-20% of the data. Moreover, training with just 10-20% of the data selected by our strategy yields superior results compared to training on the full dataset with the same time constraints. This demonstrates its practical efficiency for indoor novel view synthesis.
We summarize our contribution as follows:
1. We propose a framework for subset selection in novel view synthesis to address memory and time constraints encountered in real-life applications.
2. We propose a comprehensive measurement for the camera subset from the perspective of diversity, provide theoretical analysis of the different utility functions, and validate their effectiveness through extensive empirical experiments.
3. We introduce INDOORTRAJ, novel dataset designed for indoor novel view synthesis, featuring complex and extended trajectories that simulate intricate human behaviors.
2

---

Published in Transactions on Machine Learning Research (05/2025)
2
Related work
2.1
Neural Rendering in Indoor Scenes
The development of neural rendering methods enables efficient dense reconstruction of 3D scenes and realistic novel view synthesis from 2D images (Lombardi et al. 2019 Mildenhall et al., 2020 Müller et al. 2022 Kerbl
et al. 2023 Wang et al. 2021p. Despite this progress, reconstructing indoor scenes poses distinct cha lengessss. Indoor environments often contain significant occlusions between objects, textureless flat surfaces, and fine- grained small structures (Zhu et al. 2023). These characteristics complicate data collection, leading to long and intricate trajectories that usually contain lot of redundant information. While some approaches focus on optimizing the model structures (Murez et al. 2020 Wang et al., 2023), another line of work addresses
geometric difficulties by incorporating prior information, such as depth (Yu et al. 2022 Zhu et al.) 2023
Xiang et al., 2024,, normal prior (Yu et al.) 2022: Zhu et al.) 2023 Wang et al. 2022 Xiang et al. 2024 or semantic prior (Guo et al. 2022). g. In this work, we focus on efficiently utilizing cameras from a trajectory for neural rendering of indoor scenes. We design subset sampling strategics that enable the same basc
models (Kerbl al. 2023 Müller el. al., 2022) to achieve performance comparable to using the full frame
set while utilizing only 10% to 20% of the data within a limited time constraint.
2.2 Sampling in Neural Rendering Methods
The reconstruction quality and convergence speed are strongly influenced by the sampling strategy. Sampling in 3D reconstruction can involve points, rays, or views. Earlier works such as Agarwal et al. (2011); Mendez 2017) emphasize the need for an informative sampling stralegy, particularly in large-scale and time- intensive traditional methods. Despite recent advancements in efficient neural rendering models, such as iNGP Müller et al. (2022) and 3DGS Kerbl et al. (002)), the need for efficient sampling persists, cspecially
eet al.
when the memory or time has some limit in real applications. A line of work focus on importance sampling
on the points level along the rays, such as the representative NerfAcc solution sets (Li
2023). Others
explore sampling strategies at the ray level, as demonstrated in works such as Wu et al. (2023) Sun et al.
2024h: Pais et al. (2025). However, relatively few works address the aspect of view subset sampling.
The most relevant line of research in the context of view subset sampling is the Next-Best-View (NBV)
selection task, where heuristics are designed to optimize local information gain iteratively. Uncertainty-based heuristics (Pan et al. 2022 Goli et al., 2024 Lee et al., 2022) or warping consistency related heuristics (Ye et al. 2024) are particularly notable. Nonetheless, these approaches often require intensive computation, as they depend on a full or partial rendering procedure for each addod camera. Additionally, they encounter challenges in scenarios where cameras are more freely distributed, as discussed in Kopanas & Drettakis
(2023). An alternative branch of research employs diversity-related heuristics, focusing on factors such as spatial coverage (Kopanas & Drettakis 2023 or Euclidean distances between camera pairs (Xiao et al.
(2023]; Lee et al.
2024 an et al. 2022). These strategies show promise for computational efficiency, but they are designed purely empirically, and the theoretical effectiveness of their objectives, combined with a greedy algorithm for identifying the best set of views, remains underexplored. In our work, we revisit this task as a subset selection problem. Specifically, we provide a comprehensive discussion of diversity-related measures in the context of view synthesis, exploring utility functions based on these measures and our task objective, leading to efficient strategy for view subset sampling. Through both theoretical analysis and empirical studies, demonstrate the effectiveness of our methods in novel view synthesis.
3 Preliminary
The following preliminary concepts will be used in the following sections. We will formally define and explain them below.
Definition 1 (Submodularity). set function 2: 2D -+ R defined on a finile set Do is called submodular if it satisfies the diminishing returns property:
=(A U (k)) =(A) 2 (BU (k)) =(B)
3

---

Published in Transactions on Machine Learning Research (05/2025)
Algorithm Greedy Algorithm for Maxinizing Marginal Contribution
Require: Camera set D, utility function 2, subset size K
Ensure: A subset $ - D such that [8] =
1: $ 4-0 0
> Initialize the solution set
2: for = 1 to do
j -- arg janjs ())) 2(S)
S S- [5-}
Add the selected element to $
end for return S
Table 1: Distance measures that contribute to the diversity. Dist3D and Ang3D quantify spatial diversity, while the DistSem captures semantic diversity.
Factor
Formula
Dist3D
fa(t,5) = exp
v, l1" 2x²
where v; stands for the 3D position of camera
Ang3D
fa(i,))
trace n. R1
where R, stands for the rotation matrix of camera i
DistSem
fp(1,))
P.Pr
where p, stands for the image feature vector at camera
PP Py
for all A CCCCD and k E EDD BB.
This diminishing returns property of functions, enabling greedy algorithms to achieve effective and provably near-optimal solutions (Nemhauser et al. 1978 Kulesza et al. 2012) for the NP-hard tasks defined by Problem
Definition 2 (Monotonicity). set function =: 2D - R defined on a finile set is called non-decreasing if for all & C TD, it holds that:
2(A) $ 2 B
This monotonicity (Feige et al. 2007 Nemhauser et al. the combination of these properties enable the derivation of performance bounds for algorithms designed to solve Problem [ Specifically, for monotone submodular functions, the greedy algorithm provides approximation guarantee by achieving approximation ratio of (e 1)/e (approximately 63.2% of the optimal value), as demonstrated by Nemmhaser et al. 19778 Sec. 4). This algorithm is designed on marginal contribution of utility function =(S).
1978) often discussed alongside submodularity, as
3.1 Distance Measures
We propose multifactor distance measure that efficiently integrates spatial, angular, and semantic dis- tances, ensuring further diverse and informative view selection. We formulate these measures in Table 1 Tn particular, Dist3D and Ang3D are two fundamental measures in the spatial domain that target on position diversity and angular diversity respectively. Additionally, semantic distance offers prior information that helps emphasize the selection of visually diverse cameras, which can further enhance the global semantic information entropy. All mcasures are normalized within the range of to 1, where value of indicates minimal distance and 0 indicates maximal distance.
Some works have considered the Euclidean distance between views factor that relevant to 3D reconstrue- tion quality (Agarwal et al. 2011 Pan et al.) 2022 Xiao et al.) 2024), but rarcly considers angular distance or semantic distance. We argue that neglecting this aspect can result in suboptimal outcomes. For example, when selecting two cameras out three that provide similar observations and are positioned closely, as illustrated the Ang3D) scenario of Figure D the Ang3D mcasurement can guide the sclection towards the camera pair with the greatest angular separation. Therefore, our approach integrates this measure with Dist3D and Sem to establish comprehensive distance measure. This promotes diverse sampling results

---

Published in Transactions on Machine Learning Research (05/2025)
Ang Case
Pairwise Distoner:
In
4.5 a
Ms M4.
(,65) (65,6 144455
Greedy Sampling
Neual Rendering
03
DisiSem Case
06
DistID Case
Pairwise Distance:
ETI
0.0
(01.2)
(AL.3)
(22.3)
Pairsise Distance:
M
Camera
AngID
M a
Dist3D
DistSem
(17.8) (45.) (K7.9)
Mg.7
M79
Figure 1: Data flow illustrating our subset selection framework. The depicted examples highlight how different factors in the distance measures influence the selection process when choosing two out of three cameras. The distance matrix is symmetric and integrates multiple factors, as defined in Table [
for the full view set ID of size N. We define the multifactor distance matrix Ma,3 E RNNN follows, with each component calculated in Table !
M.j = fa(i.) fa(i.) + (f-((11), i,j = 1,2,......
(2)
where the corresponding weights satisfy 0 S a, 8, S 1, a + 1. The impact of each factor on overall diversity can vary, depending on the specific weights assigned to them.
4 Utility Function Design for Subset Selection
In this section, we propose and analyze two utility functions as alternatives to 2(S) for the subset selection problem D based on our distance matrix M. Additionally, we analyze a utility function derived from a representative baseline Kopanas Drettakis (2023) with coverage measures.
4.1 Log Determinant Function
The first utility function is derived from Determinantal Point Process (DPP) (Kulesza et al. 2012 which a probabilistic model designed for diverse subset sampling. The likelihood of selecting a subset proportional to the determinant of the principal minor (sub-matrix) determined by the subset. Specifically, DPP defines discrete probability distribution over the power set 2D through positive semi-definite matrix M Raxx.. For subset S C D, the probability of selecting S is given by:
det (Mg)
P(S) = det(M + I)
(3)
where Mg denotes the submatrix of M with rows and columns selected according to the clements in S and I is the identity matrix. By definition, if M is distance matrix, the repulsive property of the determinant encourages the DPP to prefer diverse subsets.
Proposition 1. The log determinant function
log(det P(S))
if + 0,
(S)
(4)
iSS 0.
is submodular.
The proof of submodularity is provided in Equation (74) Kulesza al.
2012

---

Published in Transactions on Machine Learning Research (05/2025)
4.2 Max- Min Distance Function
The second utility function is Max-Min Distance Function, we induct it from widely used heuristics in various domains. These include the farthest point sampling method in image acquisition (Eldar et al. 1997). Maximal Marginal (cM)) in information retrieval (Carbonell & Goldstein 1998 Lin & Bilmos 2011) and NBV methods (Pan et al. 2022 Xiao et al., 2024). And this can be adapted to distance measure between views for subset selection task. First, we need define the Domain for the utility functions used afterward,
Definition 3 (Domain of utility functions Defined by the Marginal Gain). Let X be the domain of a utility function defined by the marginal gain. A nonemply set S E X if and only if there crists k E S such that
1s
argmax max z(S) 2(8 {}*}) and $ (k) X
is (B(()))))))))))
where z(S) is utility function defined on set S.
Then, following the way of definition discussed by Lin & Bilmes 2011 and Ashkan et al. (0015). we define the utility function as:
Definition 4 (Max-Min Distance Function). Define the set function EDF X + R, where the domain X follows the Definition 3 with ZDF as the utility function
2. The function recursively defined as:
( ( 2DF (S) min(1 M,,k),
(6)
and (0) = 0. Mi, is distance measure between elements and k, 0 S Ma S 1.
Using this definition, we establish the following proposition: Proposition 2. The function 2DF is monolone and submodular.
Proof. To establish submodularity, we verify the diminishing returns property. Let A C C and consider an element D \ B. Since & S B, it follows that any element in is also in B. Therefore, the set over which the minimum is taken for B contains all elements in the set over which the minimum is taken for A. Such that the marginal contribution from adding k to set A and B holds the following:
min M,,x) 2 min(1 -M,,),
EEA
KEB
which consistent with the dininishing returns property.
To establish monotonicity, ie. ZDF(A) < SDF(B), note that 0 Mi S 1 by definition, which implies:
min(1 MCx) 2
(8)
EEB
Since the marginal contribution is non-negative for any k D D. B, the function 2pF is monotone
decreasing.
4.3 Uniform Coverage Function
Spatial coverage has been identified mcasure positively correlated with the quality of neural rendering,
highlighted by Kopanas & Drettakis 2023). To further explore its potential efficacy in the context. of submodularity, this measure can be formalized into a utility function as follows:
Definition 5 (Uniform Coverage Function). Consider a finite set of disjoint regions in 3D space denoted by Ω2 where each view E ID covers a subset of regions Ali] E S determined by its frustum. Define the set function ZCF: : + IR, where the domain X. follows the Desonitionn with ZCP as the utility function z. The function is recursively defined as:
2cr(SU (k+)) (S)
Σ (<)))))())))* + +(1 -VV (z)) #EN
(9)
6

---

Published in Transactions on Machine Learning Research (05/2025)
where X is the scaling factor and (CF(0) 0. (a)( denotes the ratio of views from S U {k} that cover region E S2 formally given by:
Cau(s) (2) Eiesucay Io))(i,z)
(10)
IS [U
Function ()) is on indicator function that is if view cover the region H, and 0 otherwise. The term (k) represents the total variation between the distribution of accumulated views over the ongular space ④ at region x,, given by:
(V) = Σ
pl,, v) -u()))
(11)
WEB
where p(x.u) denotes the probability density function (PDF) for direction at region x, and u(z) represents the uniform PDF at x.
The function 2CF is monotone since 2CF (BU - ZCF (S) 2.
However, ZCF is non-submodular. This can be observed by analyzing the two primary components of Equa- tion 9 When the set size increases, the value of Equation I0 can either increase or decrease, depending on whether the region x is covered by the newly added views. Consequently, the value of the first component EeER (x))) of Equation 9 depends on the balance between the portions of the increase and decrcase. This violates the diminishing relurn in Definition [ In addition, the TV function in the second component Exen (1 (V) (x)) of Equation 9 depends on how redistributes the views in the angular space. The marginal gain for larger set can exceed that for smaller set, if & introduces uniformity to the larger sot but increases non-uniformity for the smaller set.
This implies that the greedy algorithm this non-submodular utility function may prioritize elements with high immediate gains but low overall contribution to the global optimum. This is particularly evident in the clustering phenomenon observed in the subset selection results, as demonstrated in Experiment 5.5
5
Experiments
5.1 Dataset
To accurately assess the impact of our sampling strategy in indoor scenes, select the widely-used Replica dataset (Straub et al. 2019 one of our evaluation sets. However, in natural human motion captures, people may stop, move, or rapidly change direction. As a result, camera movements often do not maintain a consistent speed. These characteristics are not adequately captured by the Replica dataset. To complement this, we include two real-world Matasets-Mip-Noas 2017) both captured by professional annolators and sharing some of Replica's 's characteristics, but still limited in their motion complexity. To address these limitations, we create new dataset called Indoor- Traj, where the trajectories are captured within photorealistic indoor environments by real humans who have full control over the camera. The recorded trajectories feature long and complex camera movements. Additionally, a separate and disjoint testing trajectory is provided for each scene. This design, inspired by Warburg et al. (2023). ensures that the evaluation faithfully reflects the model's performance. More details can be found in the Appendix. A
360 (Barron et al. 2022 and Scan (Dai et al.
5.2 Baselines
In the experiments, primarily compare three baseline sampling strategies with our proposed method. The Random strategy samples a random subset from the full dataset. The Uniform strategy samples frames at equal time intervals throughout the sequence. Starting from random initial frame, this strategy samples frames at regular time intervals. If the end of the sequence is reached before selecting enough frames, it wraps around to continue sampling from the beginning.
The third baseline about the greedy strategy with uniform coverage function, as introduced and analyzed in Section 4.3. The heuristic of uniform coverage was initially proposed in the NBV selection problem
7

---

Published in Transactions on Machine Learning Research (05/2025)
Table 2: Results on synthetic scenes: quantitative evaluation on the testing scenes. We provide average rendering PSNRs and the standard deviations on five random seeds while sampling. The evaluation is performed on four scenes from the Replica dataset and an additional four scenes from our INDOORTRAA dataset. The camera selection ratio is set to 5%.
Method
Replica Dataset
IndoorTraj
office-2
office-3
room-1
room-2
kitchen-1
kitchen-2
openplan-1
living-1
Random
38.29# 0.59
39.65 = 0.59
40.17 + 0.66 42.23 + 0.56
24.95 + 0.33
23.5 = 0.42
25.3 = 0.22
24.4 = 0.75
3DGS
Uniform
40.38
0.14
41.09 = 0.06
42.26 + 0.12
43.33=0.07
25.45 = 0.21
23.89 + 0.2
26.12 0.07
25.51 = 0.2
SCF
35.33 0.00
37.92 = 0.00
38.63 = 0.00
39.64 + 0.00
26.96 + 0.00
24.0 = 0.00
26.55
0.00
25.74 0.00
ADDP
39.64 + 0.38
40.84 = 0.21 41.86 + 0.31
42.84 # 0.19
27.03 # 0.18
24.63 + 0.37
26.41 = 0.09
26.69 = 0.29
40.46=0.20
41.25=0.11
42.32+0.16
43.28 + 0.22
27.44+0.09
25.58=0.27
26.78=0.05
27.09=0.32
Random
35.79 + 0.64
36.84 + 0.15
36.77
+ 0.69
38.61 1 0.34
24.83 1 0.23
21.95
1 0.63
24.75 1: 0.1
21.59 1 0.61
-NGP
Uniform
37.38 +: 0.07
37.43
-: 0.04
38.25
1 0.14
19.22-0.01
25.49
1 0.15
21.76
4 0.22
25.12
0.09
22.82 1 0.28
SCF
34.59 0.00
36.12 + 0.0
35.44
0.00
37.67
1 0.00
26.65
1 0.00
23.22
1 0.00
25.82
0.00
22.61 0.00
2DDP
36.73
0.35 37.24
0.12
37.75
0.40
- 0.40
38.91
+ 0.25
26.45 + 0.23
23.14
0.41
25.44
0.11 23.53 + 0.30
2DF
37.45 37.4710.07
38.2910.25 39.13
+ 0.07
26.9610.14 23.30=0.15 25.86.10.09
24.0211.10
Table 3: Results on rcal-world scenes: quantitative evaluation on the Mip-NeRF 360 indoor scenes and ScanNet scenes with 3DGS. The camera selection ratio for Mip-NeRF 360 is to 20% instead of our standard 5% setting, the original views are already sparse.
Method
Mip-NeRF 360 kitchen
ScanNet
bonsai
counter
rUOM
scenc0050 .0
scenc0073 01
scenc0085 .00
scenc0134 02
Random
26.91+0.60
23.66++0.46
25.45 + 0.65
26.07+1.12
20.83 + 0.48
23.10 + 0.42
25.57 + 0.25
21.45 + 0.48
Uniform
28.94:1.115
24.82:0.225
26.36
1 0.48
26.5710.56
22.43 1 0.25
24.59 1 0.17
26.58 1 0.11
22.00 = 0.95
ZCP
28.071 0.00
24.334 0.00
26.451 0.00
27.211 0.00
20.58 1 0.00
23.91
0.00
6.27 :: 0.00
8.22 + 0.00
#DDP
27.71+t0.59
24.28++0.30
25.86+0.48
26.56+0.44
21.25 + 0.32
25.06 + 0.29
26.05
0.52
22.34 0.87
Anp
29.11+0.16
24.79:10.9
26.67:0.50
27.34+0.46
22.16 = 0.32
25.82+0.27
26.52 = 0.26
23.1840.55
(2023). This proposed heuristic aligns well with our subset selection framework. It quantifies 3D diversity using composite coverage measure, which introduces strong dependencies in sequential sampling. We select it as a representative for its line of work, and it is denoted by the utility function 2CB in the experiment section.
by Kopanas & Drettakis
5.3
Implementation Details
The experiments are conducted using a single NVIDIA P100 GPU for 30,000 itcrations. We utilize the default hyperparamelers recommended by the two classical neural rendering methods, instant-NGP (iNGP) (Mülle et al. 2022) and Gaussian Splatting (3DGS) (Kerbl et al. 2023). The hyperparameters a and / of each factor in distance matrix M controlled the general relative importance among different measures, we set up separate evaluation set with the four out of eight scenes in the Replica dataset (Straub et al. 2019 and empirically tune these two parameters. They set to 0.7 and 0.2 respectively.
All the camera locations are scaled to the range [-4,4]. We set = 0.5 in the Gaussian kernel Dist3D to ensure smooth distance measurements. For DistSem measure, the image features are encoded using CLIP (Rafford ltt al. 2021) trained on WebLI following Zhai et al. (2023). Since the axis aligned bounding box (AABB) scale significantly impacts the performance of iNGP (Müller et al. 2022). we set the value to 16 for the Replica dataset and for others to achieve reasonable reconstruction results.
5.4 Main Results
In Table 2 and Table 3 we evaluate the rendering quality of three bascline sampling stratogies as well as ours with the utility function 2DDP and 2pF on both synthetic scenes and real scenes. Except for the greedy sampling with the utility function zcr (Kopanas & Drettakis 2023). the subsets generated by the other four
8

---

Published in Transactions on Machine Learning Research (05/2025)
Pr
P2
Ro
Random
Uniform
ZCF
EDPF
ZDF
Figure Qualitative results on scene openplan-1 with a camera selection ratio of 5%: The first column presents a top-down heatmap at the same scale, showing the densily of selected camera positions. For clearer demonstration, 30% subset of the selected cameras from each method drawn on the heatmap. The camera poses of the testing views marked in red labeled as Pt, P2, and Pa.
methods are influenced by random scod. To ensure robustness, we generate five sample sets for cach method and report the average rendering Peak Signal-to-Noise Ratio (PSNR) along with the standard deviation.
For the standard Replica dataset, the camera movement is smooth with largely uniform speed within small region. We can find from Table 2 that the greedy strategy with 2DF performs comparably to the Uniform stralegy, offering a slight performance improvement on average. In contrast, the greedy method with ECF (Kopanas Drettakis 2023) underperforms significantly on the Replica dataset, showing an average of 2.25 PSNR drop even compared to the Random selection strategy. real-world scenes, such as 5% samples on ScanNet 0085 00 and 0134 02 in Table 3 the 2CF utility function (Kopanas & Drettakis 2023) can sometimes lead to situations where there not enough informative views for effective reconstruction.

---

Published in Transactions on Machine Learning Research (05/2025)
On the more complex INDOORTRAJ dataset, the Uniform strategy is not effective anymore. The experiments show that, with the same sample size, the average PSNR gap between the Uniform strategy and our greedy solution with EDF is approximately 1.46. Due to the more diverse and natural camera movement, the solution with 2CF (Kopanas & Drettakis 2023) begins to outperform both the Uniform and Random strategios. However, it still results an average 0.89 PSNR lower than the method with DDF Specifically, in the scene 1ivin--,, this utility function ECF even has 0.9 PSNR, worse than ZDDP and 1.5 PSNR worse than 2DF.
Notably, our greedy method using 2DDP consistently underperforms relative to 2pp. This discrepancy can be attributed to two main measure.. First, ZDDP lacks the monotonicity property, which limits its effectiveness during greedy sampling (Kulesza et al. 2012). Second, numerical precision issues in floating-point computa- tions may contribute to the difficulty in distinguishing samples that are extremely close in the feature space, potentially leading to suboptimal selections.
Additionally, wC observe a consistently large performance gap between the Random strategy and the Uni- form strategy. Intuitively, the Random stralegy can lead to clusters of cameras with similar poses capturing redundant scene information. A more theoretical explanation is related to Quasi-Monte Carlo (QMC) the- ory [Neederreieer 1978 which demonstrates that the effectiveness of specific sampling patterns in a function approximation problem is theoretically superior to pure Monte Carlo sampling. We highlight this concept here as a reference for those interested in exploring this theory further.
5.5 Qualitative Evaluation
In addition to the quantitative evaluation, we also conduct an analysis of how the selection strategy behaves in the specific scenario. We select the scene openplan-1 as an example. This scene is the largest one within our INDOORTRAJ dataset, which includes a long training trajectory that spans two rooms.
Camera distribution: In the first column of Figure 2 we present a top-down heatmap showing the density of selected camera positions. Our solution with 2DF shows smoother coverage of the entire scene. Even in relatively dense regions like RG, the selected cameras exhibit diverse orientations throughout the arca.
Instead, greedy solution with #CF (Kopanas & Drettakis 2023) tends to densely select the cameras distributed on the corners of the scene, and there are multiple cameras having very similar poses (location and orientation) as seen in region R4 and region R5- This non-submodular metric prioritizes locally optimal solutions by favoring cameras whose frustum covers a larger portion of the scene's bounding box. The non-monotone submodular metric 2DDP exhibits a slight clustering effect in region Rg. The Uniform strategy largely relies on the moving speed of the camera. If the trajectory is captured with flexible speeds, as is common in user-gene data, the method will be biased toward the temporally dense area, such region R2 and region Rg. The Random strategy often samples cameras with similar poses, such as Lhose in the region R1, and even highly overlapping ones, as seen in Ro.
Rendering Quality: From the testing view at P1 the obscrvation aligns with what we identified with zCF- This metric tends to select cameras positioned at corners facing large open spaces, often overlooking scene boundaries. The slight clustering effect causod by the metric 2DDP results in poor quality in missing regions, such as the bottom of the chair in the P1 view. For the Random and Uniform strategy, on the other hand, sampling is based directly temporal factors. Consequently, both methods show a sparse distribution in the faster-moving parts of the scene. This sparse distribution causes lack of cameras to render high-quality objects in these areas, such as the lamp in the testing view Pz. In the region around the testing view Pa. the Random, Uniform, and #CF strategies concentrale more densely than our approach with ZDDP and apF. However, despite the lower camera density, our strategies still achieve equally good rendering quality in this region. This outcome suggests a certain level of redundancy in the other three methods.
5.6 Performance Gaps with Varying Sample Ratios
In this experiment, we assess the rendering performance of various sampling strategies across different sample ratios. Specifically, we evaluate sample ratios of [0.01, 0.03, 0.05, 0.10, 0.15, 0.20). For reference, the performance of the model trained on the full dataset over 30,000 iterations is indicated by dashed line in Figure 3. The base neural rendering model used in these experiments is 3DGS (Kerbl et al. 2023)
10

---

Published in Transactions on Machine Learning Research (05/2025)
27.5
28
25.0
26
DD
PDF
PSNR
DPP
PSNR
22.5
DPP
24
PCP
2CF
Uniform
20.0
Uniform
22
Random
Random
17.5
Full set
Full set
0.00
0.05
0.10
0.15
0.20
0.00
0.05
0.10
0.15
0.20
Sample Ratio
Sample Ratio
(a) Scene kitchen-1
(b) Scene kitchen-2
28
26
25
PpF
ADF
PSNR
24
DPP
PSNR
ADPP
CCF
20
RCF
22
Uniform
Uniform
Random
Random
20
15
Full set
Full set
18
0.00
0.05
0.10
0.15
0.20
0.00
0.05
0.10
0.15
0.20
Sample Ratio
Sample Ratio
(c) Scene openplan-1
(d) Scene living-1
Figure 3: Performance across different sample ratios. The experiments are set on INDOORTRAJ dataset. Results are based on 30k iterations of training using the Gaussian Splatting backbone across different sampling strategies.
As shown in Figure 8 the approach with the utility function 2DF and 2DDP outperforms other strategies in three out of four scenes when sample ratio is larger than 0.03. The greedy solution with 2DF achieves on average 0.8 higher PSNR for kitchen-1 and kitchen-2 scenarios using just 20% data samples. Impressively, it matches or even exceeds the performance of models trained with the full dataset. with the same 30k iterations. Notably, as shown in Figure 3a it surpasses the full dataset model by 0.2 PSNR using only 10% of the data, and this advantage grows 0.6 PSNR at 20%. Similar trends are observed in Figure 3b and
3d
The greedy approach with baseline utility function ZCF outperforms other strategies when the sample ratio is below 0.03. This result suggests that the frustum coverage measure is effective for extremely small subsets. However, its performance deteriorates as the sample ratio increases in most scenes. Additionally, this strategy becomes computationally expensive, with sampling time increasing exponentially with respect the sample ratio. It requires over 100 times more sampling time compared to strategies using 2DF and ZDDP.
We also observe that the solution with 2DDP is significantly constrained by the mumerical precision issues. The metric becomes unsolvable even with 64-bit conting-point precision for an instance sample ratio of 0.2 in the cases of kitchen-1, kitchen-2, and openplan-1.
The underperformance of the model trained on the full dataset with a 30k iteration training budget suggests underfitting, which points the presence of redundancy. We can also observe that the training trajectories across different scenes differ in the levels of redundancy, explaining why the performances converge at different sample ratios. These results highlight the necessity of crafting effective sampling strategies, especially for
11

---

Published in Transactions on Machine Learning Research (05/2025)
efficient novel view synthesis involving extensively long trajectories. Our method can serve solid bascline for such strategies.
5.7 Abalation Study
To better understand the contribution of each component in our multifactor distance matrix, performed ablation study. By comparing the results, system for current evaluation data. The experiment setting follows the main experiment, and we report the average PSNR in Table 4 We evaluate our best-performed solution with metric 2pp. Fach row in the table represents a different combination of measures included in the distance matrix.
quantify the importance of each aspect in the overall
From Table we can observe that the model with all three measures consistently performs best across most scenes, indicating the importance of all three measures working together. For each individual measure, Dist3D proves to be the most informative across all scenes. For specific scenes, such office-3, kitchen-1, and living-1, distance matrix that includes only Dist3D achieves performance comparable to the multifaclor version. We can observe that, the combination of AngDist and Dist3D typically yields the second/thin t/irddbbsttt rendering results, with a performance gap of less than 0.2 PSNR on average compared to the full distance matrix. Therefore, real-world applications, prioritizing time efficiency, most cases can be effectively handled by combining AngDist and Dist3D.
Table 4: Ablation study on distance measures. We experiment various combinations of measures and compare the PSNRs of the rendering results.
Method
Measures
Replica Dataset office-3 room-1
IndoorTraj kitchen-2
DistSem AngDist
Dist3D
office-2
room-2
kitchen-1
openplan-1
living-I
x
X
38.93
40.74
40.91
42.24
22.57
23.4
25.33
23.83
A
/
x
38.26
40.20
41.45
42.89
24.68
22.14
25.77
23.12
3DGS
X
/
40.3
41.21
42.02
43.17
27.41
25.31
26.45
27.16
40.41
41.19
42.26
43.24
27.38
25.26
26.77
26.94
V
V
40.46
41.25
42.32
43.28
27.44
25.58
26.78
27.09
X
X
35.82
37.3
37.28
38.58
24.02
22.2
24.86
22.14
J
X
35.61
36.89
37.66
38.9
21.93
20.75
24.87
21.59
iNCP
37.22
37.43
37.9
39.09
26.84
23.01
25.69
23.81
37.41
37.45
38.22
39.11
26.89
23.32
25.81
23.97
37.45
37.47
38.29
39.13
26.96
23.30
25.86
24.02
6
Conclusion
This study introduces novel view subset selection framework for indoor novel view synthesis from monocular frames. The framework leverages several well-defined metrics designed from diversity-lased measurement, enabling more informed and efficient selection of views. To enable more comprehensive experiments that better simulate complex human capture behaviors, we introduce a new dataset called INDOORTRAY Through theoretical analysis and extensive experimentation, we demonstrate the effectiveness and practicality of the proposed strategies, highlighting their ability enhance efficiency and scalability indoor scene modeling.
7 Acknowledgement
This project is supported by Flanders Al Research Program and Methusalem Lifelines. HZ is supported by the China Scholarship Counci..
12

---

Published in Transactions on Machine Learning Research (05/2025)
References
Sameer Agarwal, Yasutaka Furukawa, Noah Snavely, lan Simon, Brian Curless, Steven M Seitz, and Richard Szeliski. Building rome in day. Communications of the ACM, 54(10):105-112, 2011.
Azin Ashkan, Branislav Kveton, Shlomo Berkovsky, and Zheng Wen. Optimal greody diversity for recom- mendation. In IJCAI, volume 15, pp. 1742 1748, 2015.
Jonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan, and Peter Hedman. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. CVPR, 2022.
Jaime Carbonell and Jade Goldstein. The use of nmrr, diversity-based reranking for reordering documents and producing summaries. In Proceedings of the s1 annual international ACM SIGIR conference Research and development in information retrieval, pp. 335 336, 1998.
Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proc. Computer Vision and Pattern Recognition (CVPR), IEEE, 2017.
Yuval Eldar, Michael Lindenbaum, Moshe Porat, and Yehoshua Y Zeevi. The farthest point stralegy for progressive image sampling. IEEE transactions on image processing, 6(9):1305 1315, 1997.
Uriel Feige, Vahab S Mirrokni, and Jan Vondrak. Maximizing non- mom-mootooee submodular functions. In 48th Annual TEEE Symposium on Foundations of Computer Science (FOCS pp. 461-471. IEEE, 2007.
Lily Goli, Cody Reading, Silvia Sellán, Alec Jacobson, and Andrea Tagliasacchi. Bayes' rays: Uncertainty quantification for neural radiance fields. In Proceedings of the IEEE/CVE Conference Computer Vision and Pattern Recognition, pp. 20061-20070, 2024.
Haoyu Guo, Sida Peng, Haotong Lin, Qianqian Wang, Guofeng Zhang, Hujun Bao, and Xiaowei Zhou. Neural 3d scene reconstruction with the manhattan-wond assumption. In Proceedings of the IEEE/CVP Conference Computer Vision and Pattern Recognition, pp. 5511-51166 11-55,20,44 2022.
Bernhard Kerbl, Georgios Kopanas, Thomas Leimküühler and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4), July 2023. URL https:
7/repo-sam. ienia.iiiaiai..mm /tt.gaa...//tgggaaa/////
Georgios Kopanas and George Drettakis. Improving nerf quality by progressive camera placement for free- viewpoint navigation. 2023.
Alex Kulesza, Ben Taskar, et al. Determinantal point processes for machine learning. Foundations and Trendo in Machine Learning, 5(2-3):123-285 2012.
Keifer Lee, Shubham Gupta, Sunglyoung Kim, Bhargav Makwana, Chao Chen, and Chen Feng. So-nerf: Active view planning for nerf using surrogate objectives. arXiv preprint arXiv:2312.033 03266, 2023.
Soomin Lee, Le Chen, Jiahao Wang, Alexander Liniger, Suryansh Kumar, and Fisher Yu. Uncertainty guided policy for active robotic 3d reconstruction using neural radiance ficlds. IEEE Robotics and Automation Letters, 7(4):12070 1200:0077711077,,, 2022.
Ruilong Li, Hang Gao, Matthew Tancik, and Angjoo Kanazawa. Nerface: Efficient sampling accelerates nerfs. In 2023 IEEE/CVE International Conference on Computer Vision (ICCV), pp. 18491-18500 IEFE, 2023.
Hui Lin and Jeff Bilmes. A class of submodular functions for document summarization. In Proceedings of the 49th annual meeting of the association for computational linguistics: human language technologies, pp- 510 520, 2011.
Stephen Lombardi, Tomas Simon, Jason Saragih, Gabriel Schwartz, Andreas Lehrmann, and Yaser Sheikh. Neural volumes: learning dynamic renderable volumes from images. ACM Transactions on Grophics (TOG), 38(4):1-14, 2019.
13

---

Published in Transactions on Machine Learning Research (05/2025)
Oscar Mendez, Simon Hadfield, Nicolas Pugeault, and Richard Bowden. Taking the scenic route to 3d: Optimising reconstruction from moving cameras. In Proceedings of the IEEE International Conference Computer Vision, pp. 4677 4685, 2017.
Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. In ECCV, 2020.
Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with a multiresolution hash encoding. ACM Trans. Graph., (4):::102 102:15, July 2022. doi: 10.1145/
3528223.3530127, URL https://doi. org/ 1145/3528223..... 3530127
Zak Murez, Tarrence Van As, James Bartolozzi, Ayan Sinha, Vijay Badrinarayanan, and Andrew Rabinovich. Atlas: End-to-end 3d scene reconstruction from posed images. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part VIT 16, pp. 414-431. Springer, 2020.
G. L. Nemhauser, L. A. Wolsey, and M. L. Fisher. An analysis of approximations for maximizing submodular set functions-I. Mathematical Programming, 24()):265-294, December 1978. ISSN 0025-5610, 1436-4646. doi: 10.1007 0000//FF1588977
Harald Niederreiter. Quasi-monte carlo methods and pseudo-random numbers. Bulletin of the American mathemalical sociely, 84(6):957-1041 1978.
Gonçalo Dias Pais, Valter Piedade, Moitreya Chatterjee, Marcus Greiff, and Pedro Miraldo. A probability- guided sampler for neural implicit surface rendering. In European Conference Computer Vision, pp. 164-182. Springer, 2025.
Xuran Pan, Zihang Lai, Shiji Song, and Gao Huang. Activenerf: Learning where to see with uncertainty estimation. In European Conference Computer Vision, pp. 230 246. Springer, 2022.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference machine learning, pp. 8748-8763. PMLR, 2021.
J. M. M. Montiel Raúl Mur-Artal and Juan D. Tardós. ORB-SLAM: versatile and accurate monocular SLAM system. IEEE Transactions on Robotics, 31(5):1147-1163 2015. doi: 10.109/TRR.2015.222
Julian Straub, Thomas Whelan, Lingni Ma, Yufan Chen, Erik Wijmans, Simon Green, Jakob Engel, Raul Mur-Artal, Carl Ren, Shobhit Verma, et al. The replica dataset: A digital replica of indoor spaces. arXiv preprint ar Xrx: 0579777, 2019.
Edgar Sucar, Shikun Liu, Joseph Ortiz, and Andrew Davison. imap: Implicit mapping and positioning in real-time. In Proceedings of the TEEE/CVE international conference on computer vision, pp. 0220-6233 2021.
Shilei Sun, Ming Liu, Zhongyi Fan, Qingliang Jiao, Yuxue Liu, Liquan Dong, and Lingqin Kong. Efficient ray sampling for radiance fields reconstruction. Computers & Graphics, 118:48 59, 2024.
Jiepeng Wang, Peng Wang, Xiaoxiao Long, Christian Theobalt, Taku Komura, Lingjie Liu, and Wenping Wang. Neuris: Neural reconstruction of indoor scenes using normal priors. In European Conference on Computer Vision, pp. 139-155. Springer, 2022.
Peng Wang, Lingjie Liu, Yuan Läu, Christian Theobalt, Taku Komura, and Wenping Wang. Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. Advonces in Neural Information Processing Systems, 34:27171 27183, 2021.
Yiming Wang, Qin Han, Mare Habermann, Kostas Daniilidis, Christian Thcobalt, and Lingjie Liu. Neus2: Fast learning of neural implicit surfaces for multi-view reconstruction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3295 1306,,33,,60 2023.
14

---

Published in Transactions on Machine Learning Research (05/2025)
Frederik Warburg, Ethan Weber, Matthew Tancik, Aleksander Holynski, and Angjoo Kanazawa. Nerfbusters: Removing ghostly artifacts from casually captured nerfs. In Proceedings of the IEER/CVF International Conference on Computer Vision pp. 11200-8130,, 2023.
Jiangkai Wu, Liming Liu, Yumpeng Tan, Quanlu Jia, Haodan Zhang, and Xinggong Zhang. Actray: Online active ray sampling for radiance ficlds. In SIGGRAPH Asia 2023 Conference Papers, pp. 1-10, 2023.
Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: deep representation for volumetric shapes. In Proceedings of the IEEE conference on compuler vision and pallern recognilion, pp. 1912 1920, 2015.
Haodong Xiang, Xinghui Li, Xiansong Lai, Wanting Zhang, Zhichao Liao, Kai Cheng, and Xueping Liu. Gaussianroom: Improving 3d gaussian splatting with sdf guidance and monocular cues for indoor scene roconstruction. ai preprint ar arXiv:2405. 19671, 2024.
Wenhui Xiao, Rodrigo Santa Cruz, David Ahmedt-Aristizalle Aristizabal, Olivier Salvado, Clinton Fookes, and Leo Le- brat. Nerf director: Revisiting view selection in neural volume rendering. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 20742-20751, 2024.
Xingrui Yang, Hai Li, Hongjia Zhai, Yuhang Ming, Yugian Liu, and Guoleng Zhang. Vox-fusion: Dense tracking and mapping with voxel- based neural implicit representation. In 2022 IEEE International Sym- posium on Mixed and Augmented Reality (ISMAR), pp. 499 507. IEEE, 2022.
Sheng Ye, Yuze He, Matthieu Lin, Jenny Sheng, Ruoyu Fan, Yiheng Han, Yubin Hu, Ran Yi, Yu-Hui Wen, Yong-Jin Liu, et al. Pyp-recon: Progressive view planning via warping consistency for sparse-view surface reconstruction. ACM Transactions on Graphics (TOG), 43(6):1-13, 2024.
Zehao Yu, Songyou Peng, Michael Niemeyer, Torsten Sattler, and Andreas Geiger. Monosdf: Exploring monocular geometric cues for neural implicit surface reconstruction. Advances in neural information processing systems, 35:25018 25032, 2022.
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVE International Conference on Computer Vision, pp. 11975- 11986, 2023.
Jingsen Zhu, Yuchi Huo, Qi Ye, Fujun Luan, Jifan Li, Dianbing Xi, Lisha Wang, Rui Tang. Wei Hua, Hujun Bao, et al. T2-sdf: Intrinsic indoor scene reconstruction and editing via raytracing neural sdfs. In Proceedings of the IEFE/CV Conference on Computer Vision and Pattern Recognation, pp. 12489-12498, 2023.
Zihan Zhu, Songyou Peng, Viktor Larsson, Weiwei Xu, Hujun Bao, Zhaopeng Cui, Martin R Oswald, and Marc Pollefeys. Nice-slam: Neural implicit scalable encoding for slam. In Proceedings of the IEEE/CVP conference on computer vision and pattern recognátion, pp. 12786-11711 12786-12796, 2022.
15

---

Published in Transactions on Machine Learning Research (05/2025)
A Indoor IndoorTraj dataset
We collect data from four realistic indoor scenes in Blender, cach featuring objects with varying surface roughness and natural light reflection effects. Each scene includes a long training trajectory that spans the entire space. Annotators are tasked with controlling a 6-DOF (Degrees of Freedom) camera to navigate through virtual environments. Camera translation managed using the WASD keys on the keyboard, while rotation is controlled through mouse movement. The trajectories are recorded at 24 frames per second with 1024x1024 resolution. To ensure diversity in camera movements and capture preferences, the trajectories for the training and testing splits recorded by different annotators. The dataset statistics as follows:
kitchen-1
kitchen-2
openplan-1
living-1
Train
2253
3084
3212
2632
Test
133
99
182
102
Table 5: Data size for INDOORTRAJ dataset. We render 10 frames per second for training set and 1 frame per second for testing set.
These scenes also feature diverse objects, such as somi-transparent glasses and highly reflective surfaces, as shown in the example frames in Figure 4 These characteristics present additional challenges for accurate novel view synthesis. In Figure 5 we provide the top-down trajectories of each scene and example rendering views.
Figure 4: Visual characteristies in the INDOORTRA TRAJ dataset, featuring object arrangement complexity, transparent objects, highly reflective surfaces, and areas with intricate textures.
16

---

Published in Transactions on Machine Learning Research (05/2025)
Sample
Sample
Training Testing traj.
kitchen-1
Sample
Sample
iimii
Sample
Sample
Training traj.
Testing truj.
Sample
Sample
Sample
Sample
Training draj.
openplun-1
Testing traj.
Sumple
Sumple 4
Sample
Sample
traj- Testing traj.
living-1
Sample
Sample
Figure 5: INDOORTRAJ dataset: Training and testing trajectories and example views
17

---

Published in Transactions on Machine Learning Research (05/2025)
B Analysis on Hyperparameters
We selected the hyperparameters through grid search on four tuning scenes in Replica, mentioned in implementation details. We found that this set of parameters consistently achieves good performance across different datasets in the main results. For the discussion on parameter sensitivity, we provide grid search tables two representative scenes, i.e. Replica-Office and Replica- Office0 in Figure B
Although different scenes under the same sample ratio perform different range of PSNRs, we still observe that the optimal parameters generally 0.7, 3 0.2, and 7 0.1. The results showed relatively low sensitivity to variations around the chosen parameters. Therefore, we empirically chose this set of parameters.
Replica-Officed PSNRs
Replica Office4 PSNRS
0.0
38
38.13
37.93
30.50
30.27
38
00.34
38.45
43.7
0.1
01-38.68
99.11
58.57
38.25
39.16
39.20
38.6038.63)
38.8779.00
0.2
43.51
43.80 43.3943
43.45
8357
43.00
43.63
43.6343.000
43.48
0.7
0.7-38.99
38.00
38.00
38.61
39.95
39:20
39.03
39.00
39.09
38.93
39.00
0.3
43.58
43.6243.62445
43.55433.
43.51
43
0.3
03--9.0771
39.08
39.27
99.15
39.05
38.89
38.84
38.80
43.48
43.4
43.53
43.65
43.51
43.61
43.5
38.50
0.6
0.7
43.49
43.72
43.66
0.7
00.83
30.95
09
09:49 39.06
38.00
0.8
43.65
43.68
43.3
0.8
38.81
39.26
43.46
0.9
39.27
38.63
37.50
1.0-88.9
0.0
0.1
0.2
0.3
0.1
0.5
0.6
0.7
0.8
0.9
1.0
0.0
0.1
02
0.3
0.6
0.7
0.8
0.9
Figure 6: PSNRs grid search table for hyperparameters tuning scenes Replica Office0 and Office 4.
C Time efficiency
Regarding the subset sampling time, we provide a reference table below. The sampling time for the method ZCF proposed in Koppnas & Drettakis (2023) is significantly higher. Applying this method in real-world scenarios may require additional effort to optimize its efficiency at the code implementation stage. For 2DPP and 2DF, the sampling times are slightly slower than Uniform and Random sampling, but this introduces only minor delay to the overall training time.
Replica
IndoorTraj
ScanNet
MipNeRF360
Random Uniform
<1s
<1s
<1s
<1s 1s
<1s
<1s
<1s
ZCF
239s
325s
184s
96s
ZDPP
2.32s
6.18s
1.02s
<1s
EDF
1.73s
2.78s
1.48s
<18
18


Reviews:
[{"review_id": "V7bZKGLPbk", "reviewer": "Reviewer_bC5i", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n- The paper presents a well-structured combinatorial optimization approach to subset selection for novel view synthesis, addressing redundancy in monocular video sequences.\n\n- The authors provide rigorous theoretical analysis of their metric functions and validate them with experiments.\n\n-   The method significantly reduces data requirements while maintaining or improving rendering quality, making it practical for real-world applications.\n\n- It introduces a new dataset (IndoorTraj) that features complex, realistic human-driven camera trajectories\n\n- Experiments demonstrate consistent improvements over baseline strategies\n\nWeaknesses:\n\n- The framework is tested primarily on IndoorTraj and Replica, but its effectiveness on more diverse datasets or real-world capture scenarios is not deeply explored.\n\n- Certain metric functions (e.g., log-determinant DPP) may still suffer from numerical precision issues or scalability challenges?\n\n- The framework requires tuning parameters (such as $\\alpha,\\beta,\\gamma$) in the distance matrix, and the robustness of these settings across different datasets is not fully analyzed.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces a framework to improve efficiency in indoor novel view synthesis from monocular video sequences. By formulating the problem as a combinatorial optimization task, the authors develop diversity-based metric functions that account for spatial, angular, and semantic variations. They also introduce IndoorTraj, a novel dataset with complex human-like camera trajectories, demonstrating that their method achieves superior synthesis quality using only 5-20% of the data. Theoretical analysis and experiments validate that their greedy optimization strategy effectively balances computational efficiency and reconstruction performance.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "It would be good if authors can expand evaluation to additional datasets and analyzing sensitivity to hyperparameters in different environments.", "broader_impact_concerns": "na", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "3e607lwvne", "reviewer": "Reviewer_vC6f", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**-----Strengths-----**\n\n1. The paper is well-structured and addresses a significant yet often overlooked problem in the literature: selecting an optimal subset of views for novel view synthesis.\n\n2. The proposed multifactor measure is interesting and effectively combines spatial coverage, directional diversity, and content variation. The quantitative results demonstrate that this approach outperforms all proposed baselines across multiple scenes.\n\n3. I appreciate the paper's focus on providing theoretical analysis and guarantees for their approach. Such considerations are both interesting and important for the research community.\n\n4. The comprehensive ablation study effectively demonstrates the importance and impact of each component in the proposed metric.\n\n5. The introduction of IndoorTraj, a new dataset specifically designed for indoor novel view synthesis with complex camera trajectories, represents a valuable contribution to the research community.\n\n**-----Weaknesses-----**\n1. The quantitative evaluation is limited to only 8 scenes, which is insufficient for validating a view selection method. Given that the proposed method is applicable to any RGB image dataset, evaluation on very common larger-scale datasets (such as ScanNet) would strengthen the paper's claims, particularly for indoor environments where such datasets are readily available.\n\n2. The qualitative evaluation lacks depth, with only a single example provided to demonstrate the performance of the method.\n\n3. Several mathematical claims require additional justification or contain errors. Most notably, the set function introduced in Definition 3 is not properly defined. The function is described through an iterative process, but for it to be well-defined on all subsets of $\\\\mathbb{D}$, the value obtained with this iterative process should be order-independent: For a given subset $S=\\\\{I_1,...,I_k\\\\}$, the value of $z_{DF}(S)$ should not depend on the order of the elements $I_i$ used during computation. The paper neither proves this property nor addresses potential order dependency. It is actually possible to find counter-examples for which the function is not order-independent.\n\nFor instance, let's consider a set $\\\\mathbb{D} = \\\\{I_1, I_2, I_3\\\\}$ for which the distance matrix $M$ is defined as follows:\n\\\\begin{equation}\n    M = \\\\begin{bmatrix}\n        0 & 0.5 & 0.4 \\\\\\\\\n        0.5 & 0 & 0.9 \\\\\\\\\n        0.4 & 0.9 & 0\n    \\\\end{bmatrix}\n\\\\end{equation}\nThe matrix $M$ is a proper distance measure with elements in $[0, 1]$, as required by Definition 3. Such matrix can trivially be obtained by taking $D \\\\subset \\\\mathbb{R}$ equipped with the euclidean distance, and $I_0 = 0$, $I_1 = -0.5$, and $I_2 = 0.4$.\nThe matrix $1-M$ is then equal to:\n\\\\begin{equation}\n    1-M = \\\\begin{bmatrix}\n        1 & 0.5 & 0.6 \\\\\\\\\n        0.5 & 1 & 0.1 \\\\\\\\\n        0.6 & 0.1 & 1\n    \\\\end{bmatrix}\n\\\\end{equation}\n\nLet's start with the subset $S = \\\\{0\\\\}$. The paper does not specify what the initial value of $S$ should be for a single element set, so let's denote $z_0 := z_{DF}(\\\\{0\\\\})$.\n\nWe can prove that depending on the order of the elements used for computing $z_{DF}(D)$, the result of the iterative process is different. \n\n**Let's first consider the order $0, 1, 2$.**\n\nWe apply equation (5) with $S = \\\\{0\\\\}$, and $k = 1$.\n\nWe obtain $z_{DF}(\\\\{0, 1\\\\}) = z_{DF}(\\\\{0\\\\}) + \\\\min_{i \\\\in \\\\{0\\\\}} (1-M_{i,1}) = z_0 + 0.5$.\n\nWe then reapply equation (5) with $S = \\\\{0, 1\\\\}$, and $k = 2$.\n\nWe obtain $z_{DF}(\\\\{0, 1, 2\\\\}) = z_{DF}(\\\\{0, 1\\\\}) + \\\\min_{i \\\\in \\\\{0, 1\\\\}} (1-M_{i,2}) = z_{DF}(\\\\{0, 1\\\\}) + 0.1 = z_0 + 0.6$.\n\n**Let's now consider the order $0, 2, 1$.**\n\nWe apply equation (5) with $S = \\\\{0\\\\}$, and $k = 2$.\n\nWe obtain $z_{DF}(\\\\{0, 2\\\\}) = z_0 + \\\\min_{i \\\\in \\\\{0\\\\}} (1-M_{i,2}) = z_0 + 0.6$.\n\nWe then reapply equation (5) with $S = \\\\{0, 2\\\\}$, and $k = 1$.\n\nWe obtain $z_{DF}(\\\\{0, 1, 2\\\\}) = z_{DF}(\\\\{0, 2\\\\}) + \\\\min_{i \\\\in \\\\{0, 2\\\\}} (1-M_{i,1}) = z_{DF}(\\\\{0, 2\\\\}) + 0.1 = z_0 + 0.7$.\n\nIt appears that the function $z_{DF}$ is not well-defined on the collection of all subsets of $\\\\mathbb{D}$. I acknowledge that I might have misunderstood the definition of $z_{DF}$ or missed an important detail, so I would be grateful if the authors could clarify this point. \n\nIf I'm mistaken and the function is well-defined, I would be glad to modify my review as the paper would then provide very interesting theoretical results on the view selection problem.\nHowever, if my interpretation is correct, then the theoretical results and guarantees of the paper are not valid, which would be particularly concerning as the function $z_{DF}$ reaches the highest performance on the datasets and can be considered as the most important contribution of the paper.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "**-----Context-----**\n\nThis paper addresses several key challenges in view subset selection for novel view synthesis. Computing traditional rendering metrics like PSNR requires first reconstructing the scene from input images, making these metrics computationally expensive for optimal view subset selection where any possible subset of views could be considered. Additionally, selecting an optimal subset of K views that maximizes performance is an NP-hard problem. While some existing methods rely on time-consuming training iterations for view selection, and others focus primarily on view diversity through distance and frustum overlap, they fail to adequately capture angle variation and scene semantics.\n\n**-----Contributions-----**\n\nTo address these limitations, the authors propose a comprehensive framework for efficient, greedy view subset selection that optimizes both memory usage and computational time. \n\nTheir approach introduces a novel multifactor distance measure that simultaneously promotes (1) spatial coverage, (2) directional diversity, and (3) content variation. \n\nAuthors investigate the mathematical properties of the proposed distance measure in order to provide theoretical guarantees on the performance of the greedy view subset selection algorithm. \n\nMoreover, the authors contribute a new dataset of 4 scenes that better reflects complex human capture behaviors by incorporating greater variety in camera motion speed and trajectory length.\n\nAuthors show that their greedy view selection algorithm outperforms several baseline methods on their two datasets (including their own), and that the proposed measure appears to be a good proxy for traditional, expensive-to-compute rendering metrics.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "**-----Critical for acceptance-----**\n\n- Definition 3: The paper must absolutely clarify that the function $z_{DF}$ is well-defined on the collection of all subsets of $\\\\mathbb{D}$.\n\n**-----Non Critical for acceptance-----**\n\n- It would be useful to clearly explain mathematically what $\\\\mathbb{D}$ is at the beginning of the paper. I understand that $\\\\mathbb{D}$ is the set of all images $I_i \\\\in [0, 1]^{H_i \\\\times W_i \\\\times 3}$ but right now, $\\\\mathbb{D}$ is introduced without proper definition.\n\n- Figure 1: *Neual Rendering* > *Neural Rendering*.\n\n- Section 4.1: The sentence \"*the determinant of the principal minor (sub-matrix)*\" is not correct. A minor is, by definition, the determinant of a sub-matrix. The sentence should be \"*the principal minor (determinant of sub-matrix)*\".\n\n- Proposition 1: Writing \"log(det(P(S)))\" is not correct as P(S) is a scalar probability, it should be \"log(P(S))\".\n\n- Definition 3: What are the initial values, for $S = \\\\emptyset$ and $S = \\\\{k\\\\}$?\n\n- Section 5.7: *Abalation Study* > *Ablation Study*", "broader_impact_concerns": "I did not identify any broader impact concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "DD48wcUhiT", "reviewer": "Reviewer_7FME", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The topic is interesting and practical.\n2. The authors compare three difference metric functions for subset selection, and collect a dataset for testing their method.\n\nWeakness:\n1. The authors claim their subset selection method could reduce memory and time constrains in real-world application, but they didn't provide comparisons. For example, memory/optimization time of methods in Table 2 and Figure 3.\n2. They only conduct experiments on 8 scenes, which is limited. Experiments on more scenes and other datasets should be provided. For example, RealEstate10k\n3. Optimization time is related to optimization iterations, the authors should provide figures of PSNR v.s. iterations of training on their selected set and full set.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors propose a framework for subset selection in indoor scene reconstruction task. To address memory and time constraints, they propose three metric functions to select only a subset of images for model training. Theoretical analysis of the metric functions is provided and experiments on several indoor scenes validate the effectiveness of the proposed method.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please refer to weaknesses.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers all recommend acceptance, finding the paper well structured, the approach well motivated, and the experiments convincing. The review and rebuttal period appears to have been productive as well, making the paper even stronger. The AE recommends acceptance without further review necessary, but the authors are encouraged to integrate the work presented in the rebuttal.
Decision: Accept (poster)