Paper: Fiber Monte Carlo (ICLR-2024)

Reviewer comment (Evaluation):
While the broad application areas presented are impactful, the actual evaluation is very limited. Specifically, the rendering application is evaluated using only qualitative (not quantitative) results, on a single scenario.

Author response:
Thank you for taking a close read of our work and dedicating the time to review it. We agree that a comprehensive and deep investigation of just one or two applications is often the right approach. We felt some tension around this ourselves, but in this work, our aim was to establish the generality of our method, as applied to several (seemingly) disparate areas of application. 

The tradeoff in doing so is that each of these application areas are research areas in their own right of course (which we can’t do justice to within a few pages of writing). That is all to say, the core (application-oriented) claim we are making is that our method solves a novel problem with a surprisingly far reach in applications. 

We considered differentiable rendering more broadly to be out of scope of the paper, exactly in the spirit written above (although we are fans of the work you listed). We chose to consider image stylization specifically because it is an appropriate target for fiber Monte Carlo, and no general method had been used to achieve stylization in the literature. Further, it’s worth noting that image stylization is an application which lacks a great/insightful quantitative metric. Technically one frames the application as an optimization problem, but this isn’t the complete picture since solving the problem as posed would not actually result in a stylized image (the minimum of the objective is attained by rendering the original target image perfectly, which hardly “stylizes” it). Here we felt the relevant comparison to our method is attempting to stylize the image without using a procedure to estimate derivatives with respect to geometry (instead only with respect to color), and we provide the results attained by doing so in a side-by-side comparison (see figure 2).

Reviewer comment (Experiments):
In the case of the convex hull example, no comparison to alternative methods is provided.

Author response:
Finally, our application of approximately computing convex hulls is not an application that has been seriously examined in the literature, to our knowledge. We display out-of-sample examples from our data (figure 4) and report out-of-sample accuracy on the intersection over union objective. We could devise yet another method to approach the problem, and then attempt to compare two novel methods, but we don’t feel that would helpfully contextualize the results given this problem has not been approached previously in the literature to our knowledge.

Reviewer comment (Novelty):
There seem to be missing key references with respect to the claimed applications, for example, in the case of differentiable rendering and simulation. Are such references considered to be out of the scope of the paper? As mere examples, in the case of differentiable rendering (by no means an exhaustive list):  
- Merlin Nimier-David and Delio Vicini and Tizian Zeltner and Wenzel Jakob. Mitsuba 2: A Retargetable Forward and Inverse Renderer.  
- Tzu-Mao Li, Miika Aittala, Frédo Durand, Jaakko Lehtinen. Differentiable Monte Carlo Ray Tracing through Edge Sampling.

Author response:
We considered differentiable rendering more broadly to be out of scope of the paper, exactly in the spirit written above (although we are fans of the work you listed). We chose to consider image stylization specifically because it is an appropriate target for fiber Monte Carlo, and no general method had been used to achieve stylization in the literature. Further, it’s worth noting that image stylization is an application which lacks a great/insightful quantitative metric. Technically one frames the application as an optimization problem, but this isn’t the complete picture since solving the problem as posed would not actually result in a stylized image (the minimum of the objective is attained by rendering the original target image perfectly, which hardly “stylizes” it). Here we felt the relevant comparison to our method is attempting to stylize the image without using a procedure to estimate derivatives with respect to geometry (instead only with respect to color), and we provide the results attained by doing so in a side-by-side comparison (see figure 2).

Reviewer comment (Theory):
In some applications (e.g. Monte Carlo Path Tracing Rendering), the integration domain must be sampled according to a non-uniform probability distribution. This is commonly achieved by inverse-transform sampling, whereby random variables are sampled from a uniform distribution, then mapped to the target distribution via its inverse-CDF. Is Fiber Monte Carlo compatible with such sampling requirements?

Author response:
You ask a great question about non-uniform sampling. We have not explicitly attempted to work out the details of the scenario you mention. Speculatively, if one can push the fibers through the inverse function (possibly representing the fibers as parametric curves), it should still be possible to apply our implicit function formulation to estimate the integrals. This is an interesting use-case that should probably be the focus of a follow-up study. Happy to hear your thoughts or speculations on this as well!

Reviewer comment (Presentation):
In the previous NeurIPS discussion, you referenced a plot of variance as dimension increased. Can you include this again in supplementary? I think it's useful for noting this limitation.

Author response:
Dear reviewer, great to hear from you again! Thanks for taking the time to review (again) our work. We’ll be brief in our response to your two questions. 

2. Absolutely, we will include this plot: thanks for this suggestion.

Reviewer comment (Experiments):
The application to topology optimization is more compelling because topology optimization seeks solutions in indicator functions, but the choice to represent the indicator function on a grid is puzzling given that a major advantage of a sampling-based estimator coupled to automatic differentiation is the flexibility of the underlying representation.

Author response:
It is true that the geometry representation is still a grid. In general, level-set topology optimization is known to be very difficult to do stably. Using flexible representations, although in theory possible, is difficult to get right. Existing literature has focused on grid representations, so to align with existing work we chose to stick with the grid representation. Also of note is that topology optimization is a large field with a vast literature, and we do not expect to show that our method is the “best” out of the box on two examples; the marginal improvement in objective function is possibly only incidental. A rigorous evaluation of fiber sampling with topology optimization potentially deserves its own paper; here we simply present that it performs reasonably out of the box compared to a standard approach from the literature, since the focus of this work is about explicating the fundamental method of fiber sampling.

Reviewer comment (Evaluation):
The convex hull application appears the weakest. The generated approximate hulls do not seem to be convex and do not contain all the points, making their usefulness in downstream applications such as collision detection unclear. Furthermore, classical convex hull algorithms in low dimensions are already fast, with correctness and performance guarantees, so any learned “oracle” would need to meet a very high standard to replace them.

Author response:
Visually, we see that in certain portions of the image it is not immediately apparent that the approximate hulls are convex sets (the pale coloring makes it hard to distinguish); this is useful feedback for regenerating finalized figures with a better color set. Rest assured that this is a plotting artifact: the approximate hulls are convex by construction. We take the intersection of a finite set of half-spaces. Since a halfspace is a convex set, and the intersection of a finite set of convex sets is itself convex, the approximate hulls are convex (for any value of the parameter). 

It is true that our approximate hulls are not supersets of the true convex hulls (i.e., they do not contain all the points, as you mention). This why we use the term “approximate” convex hulls (necessarily convex sets which are not necessarily equal to the convex hull). We do not propose the replacement of classical convex hull algorithms, and completely agree that such a proposal would require significantly more comprehensive analysis (in fact, this is an immense area of study and could be the subject of an entire paper).

Reviewer comment (Novelty):
The method appears to be limited to settings where the integrand is described by a superposition of indicator functions parameterized by implicit sublevel sets. The question arises as to why this representation is used instead of a continuous implicit function directly, such as SIREN, and in what application domains this representation would be uniquely advantageous.

Author response:
1. There may be some confusion here as to the notion of a “primitive” in the image stylization experiment. The core property of a primitive is its associated color. The single sentence answer to your question is: if we had used any one implicit function (i.e., primitive): we would have precisely one color in the image. Put another way, you can think of each individual SIREN network for $i = 1, \dots n$ as having associated with it a particular color, which allows us to render non-monochromatic images. When $i=1$ we have just $c_1$ and thus we are optimizing the spatial distribution of the color $c_1$ across the image canvas. It is not the case that the method is limited to settings where the integrand is described by a superposition of indicator functions parameterized by implicit sub-level sets, although this is absolutely a useful formulation to flexibility represent geometry. As a counterexample, consider our first stylization with triangles: which applies the method to an (1) an integrand which does not contain any implicit functions and (2) does not parameterize geometry with sub-level sets. That said, the implicit function formulation is hardly a limitation: in fact it is the key to parameterizing nearly arbitrary geometries.