Fast 3D Acoustic Scattering via Discrete Laplacian Based Implicit Function Encoders
Authors: Hsien-Yu Meng
Zhenyu Tang
Dinesh Manocha
Venue:  (2021)
Keywords: wave equation
wave acoustics
geometric deep learning
sound simulation
shape laplacian

Abstract: Acoustic properties of objects corresponding to scattering characteristics are frequently used for 3D audio content creation,  environmental acoustic effects, localization and acoustic scene analysis, etc. The numeric solvers used to compute these acoustic properties are too slow for interactive applications. We present a novel geometric deep learning algorithm based on discrete-laplacian and implicit encoders to compute these characteristics for rigid or deformable objects at interactive rates. We use a point cloud approximation of each object, and each point is encoded in a high-dimensional latent space. Our multi-layer network can accurately estimate these acoustic properties for arbitrary topologies and takes less than 1ms per object on a NVIDIA GeForce RTX 2080 Ti GPU. We also prove that our learning method is permutation and rotation invariant and demonstrate high accuracy on objects that are quite different from the training data. We highlight its application to generating environmental acoustic effects in dynamic environments.

Under review as a conference paper at ICLR 2021
FAST 3D ACOUSTIC SCATTERING VIA DISCRETE LAPLACIAN BASED IMPLICIT FUNCTION ENCODERS
Anonymous authors Paper under double-blind review
ABSTRACT
Acoustic properties of objects corresponding scattering characteristics fre- quently used for 3D audio content creation, environmental acoustic effects, lo- calization and acoustic scene analysis, etc. The numeric solvers used compute these acoustic properties are too slow for interactive applications. We present a novel geometric deep learning algorithm based on discrete-laplacian and implicit encoders to compute these characteristics for general 3D objects at interactive rates. We use a point cloud approximation each object, and each point en- coded in high-dimensional latent space. Our multi-layer network can accurately estimate these acoustic properties for arbitrary topologies and takes less than Ims per object on NVIDIA GeForce RTX 2080 Ti GPU. We also prove that our Iearn- ing method is permutation and rotation invariant and demonstrate high accuracy objects that quite different from the training data. We highlight its applica- tion to generating environmental acoustic effects in dynamic environments.
INTRODUCTION
Acoustic scattering corresponds to the disturbance of a given incident sound field due to an object's shape and surface properties. It can be regarded as one of the fundamental characteristics of an object. The effect of scattering can be expressed in terms of a scattered sound field, which satis- fies Sommerfield's radiation condition. There is considerable work on modeling and measuring the acoustic scattering properties in physics and acoustics and these characteristics are widely used for sound rendering in games and virtual reality (Mehra et al., 22155] |Rngtta et al., 2008)) noise analysis in indoor scenes (Morales & Manochal 2018). acoustic modeling of concert halls (Shtrepi et al. 2013), non- offe----f--- (NLOS) imaging (Lindell et al. 2019). understanding room shapes (Dok- manic al. 2013), receiver placement (Morales al.) 2219)) robot sound source localization (An et al. 2019). 3D mapping (Kim et al., 2020), vudio--viual analysis (Sterling et al. 2018). etc.
Acoustic scattering of objects can be modeled accurately using the theory of wave acoustics (Kut- truff] 2016). The scattering characteristics of objects are widely used for sound propagation, which reduces solving the wave equation in large environments. Given a sound source location and its vibration patterns, acoustic simulation methods are used to predict the perceived sound at another specified location considering the medium it passes through and objects/boundaries interacts with. While the wave behavior of sound is well understood in physics, it is much more difficult to com- pute acoustic scattering and sound propagation effects, especially for higher frequencies. Even with of-the-t-t acoustic wave solvers, it can take from hours to days to solve moderately modeled room environment on powerful workstation. One the contributing factors to this difficulty is that wave behaviors are frequency dependent, so many frequency bands need to be analyzed separately.
Current methods for computing the acoustic scattering characteristics can use numeric solvers like boundary-clement methods (BEM). However, their complexity increases cubic function of the frequencies and most current implementations are limited to static scenes or environments. No good practical solutions are known to compute the acoustic scattering properties for dynamic environments or when objects move or undergo deformation.
We present novel techniques based geometric deep learning differential co- ordinates to approximate the acoustic scattering properties of arbitrary objects. Our approach is general and makes assumption about object's shape, genus, rigidity. We approximate the
Main Results:

---

Under review as a conference paper at ICLR 2021
objects using point-clouds, and each point in the point cloud representation is encoded in high- dimensional latent space. Moreover, the local surface shapes in the latent space are encoded using implicit surfaces. This enables us to handle arbitrary topology. Our network takes the point cloud as an input and outputs the spherical harmonic coefficients that represent the acoustic scattering field.
We present techniques to generate the ground truth data using accurate wave-solver geometric dataset. We have evaluated the performance on thousands of objects that are very different from the training database (with varying convexity, genus, shape, size, orientation) and observe high accuracy. We also perform an ablation study to highlight the benefits of our approach. The additional runtime overhead of estimating the scattering field from neural networks less than Ims per object on NVIDIA GeForce RTX 2080 Ti GPU. We also prove that our learning method is permutation and rotation invariant, which is an important characteristic for accurate computation of acoustic scattering fields.
large
RELATED WORKS
2.1 WAVE-ACOUSTIC SIMULATION
Wave-acoustic simulation methods aim to solve the wave equation that governs the propagation and scattering of sound waves. Some conventionally used numerie methods include the finite-clement method [Thompson] 2006), the boundary-elemen method (Wrobel & Kassab] 2003). and the finite- difference time domain (Botteldooren 1995). The common requirement of these methods is the proper discretization of the problem domain (e.g., spatial resolution, time resolution), which means the time complexity of them will scale drastically with the simulation frequency. Recent works man- age to speed up wave simulations by parallelized rectangular decomposition (Morales et al. 2015) or by using compmuttttio structures (Mehra al., (015). An alternative way to use wave simulation results in real- time applications is to pre pre-compute a large amount of sound fields in a scene and com- pressing the results using perceptual metrics (Raghuvanshi & Snyder 2014). While pre-computation methods can save significant runtime cost, they still require non-trivial pre-computation efforts for unseen scenarios, restricting their use cases.
2.2 LEARNING-BASED ACOUSTICS
Machine learning techniques have been widely applied in many popular computer science research areas. There is considerable work on developing machine learning methods for applications cor- responding to audio-visual analysis (Zhang et al. 2017) and acoustic scene classification (AbeBer) 2020) (see Bianco et al. (2019) for a thorough list of work). In comparison, there are much fewer works studying the generation of acoustic data from physical perspective. Therefore, methodolo- gies from existing popular domains often do not directly apply to our problem of interest. Some more relevant works focus on estimating room acoustics parameters from recorded signals (Eaton al., 2016 Genovese et al. 2019 T'sokaktsidis et al. 2019 Tang et af.) (200)), which can help physically- based simulators to model real-world acoustics more faithfully. Pulkki & Svensson| (2019) propose to use a neural network to model the acoustic scattering effect from rectangular plates without run- ning simulation. This is closely related to our goal of bypassing expensive wave simulation while generating plausible sound, although we aim to model objects of more general shapes. Recently, Fan et al. (2020) train convolutional neural networks (CNNs) to learn to map planar sound fields in- duced by convex scatterers in 2D, which provides promising results. Motivated by the last method, we aim to develop networks that can deal with object geometry in 3D and extend the generality of this learning based approach.
2.3 GEOMETRIC DEEP LEARNING AND SHAPE REPRESENTATION
There is considerable recent work on generating plausible shape representations for 3D data, includ- ing voxel-based (Zhou & Tuzel, 2018 Sindagi et al.) 2019 Meng et al.) 2019 Wu et al. 2015) point-based (Charles et al. | [017
Qi et al. 2017] W Wang et al. 2019: Li et al. 2018a| Monti et al. 2017 Li al.] 2 Yi et al. 2017) and mesh-based based (Hanocka et al. 2019) shape representations.
This includes work on shape representation by learning implicit surfaces on point clouds (Smirnov
al. 2019), designing mesh Laplacian for convolution (Tan et al. 2018), hierarchical graph
2

---

Under review as a conference paper at ICLR 2021
convolution meshes (Mo et al. 2019), encoding signed distance functions for surface reconstruc- tion (Park et al. 2 etc. However, previous methods on point cloud shape representations learn by designing loss functions to constrain surface smoothness functions only provide spatial information of each point and lack information about local shape of Laplace-Beltrami
global Cartesian coordinates. Such
the surface compared to the explicit discretization of the continuous Laplace- Beltrami operator and curvilinear integral (Do Carmo 2016). Instead, we use point cloud based learning algorithms, which do not require mesh Laplacians for graph neural networks. This makes our approach applicable to all kind of dynamic objects, including changing topologies.
3
OUR APPROACH
3.1 OVERVIEW
Our goal is to directly predict the acoustic scattering field (ASF) resulting from an incident wave and a sound scatterer of known shape (described by a triangle mesh). set of frequency dependent ASFs visualized in Figure I on disk plane. The two key steps defining compact and efficient way to encode the ASF, and designing a neural network architecture that can effectively exploit the 3D geometric information this section, we provide necessary background on wave acoustics and explain the mechanism of our approach. Our notations are summarized in Table !
as establish the mapping between the object shape and the ASF. In
Table I: Notation and symbols used in this section.
x
3D Cartesian coordinates. acoustic particle velocity. Frequency of sound. Spherical coordinates.
m. Y, (0,0)
Order and degree of basis functions. Spherical harmonics basis function. Wavenumber.
(r. 0, ②)
cf"h (w) by (kr)
Spherical harmonics coefficients.
Speed of sound in air.
Hankel function.
|Pacul
125Hz
250HZ
500Hz
1000Hz
Figure I: Acoustic scattering of the same object at four different frequencics, assuming plane wave travelling to the = direction and sound-hard boundary condition for the object/scatterer in the center. The scattering patterns are significantly different among frequency bands. Our learning method can compute them with high accuracy for arbitrary objects at interactive rates, including high-genus and deforming objects.
3.2 ACOUSTIC SCATTERING FIELDS
We are interested in knowing how the incident sound field will be scattered when it acts an object surface. Let the sound pressure location and time be P(x, t), and the wave equation describes the propagation of sound:
1 02 d 0i P(x, t) = 0,
(1)
where c is the speed of sound, which is commonly assumed to be constant in non-large scale indoor environments. Transforming into the frequency domain, we obtain the homogeneous Helmholtz equation:
(v2 + x w) =
(2)
where w is the wave frequency and k is the wavenumber. When a specific boundary condition given, its solution in 3D spherical coordinates can be decomposed
(r,0,o,w) Σ (KT) Σ ('))
(3)
m=-
3

---

Under review as a conference paper at ICLR 2021
where (kr))) is the Hankel function which encodes the radial pressure changes, Y,T (8. Φ) is the spherical harmonics term which represents the angular part of the solution, and ci" (w) is the spheri-
cal harmonics coefficients for frequency w. For maximum order L. there are (L + 1)² coefficients. Since the Hankel and the Spherical Harmonics functions are easy to evaluate, we only need to find the coefficients et" (w) to fully describe the acoustic scattering field.
3.3 NETWORK INFERENCE
Our goal is to use appropriate geometric representation for the underlying objects in the scene so that we can apply geometric deep learning methods to infer the acoustic scattering field from the object shape. It is also important that our approach should be able handle highly dynamic scenes with general objects (i.e., no assumption on their convexity or genus). Moreover, our approach can still produce plausible ASFs results for changing typologies. The input our system will be the point cloud representation (N x 3 matrix, N being the number of 3D points) of an object. We represent each point and its local surface by a higher dimension implicit surface in the latent space formed by an implicit surface encoder. [Thanks to Kolmogorov-Amoll representation theorem, we can represent the pressure field function as a function defined on set of points (N x 3), which is a multivariate continuous function composed of continuous functions of one variable encoded in higher dimension. Moreover, borrowing the power of the De Finetti Theorem, our pressure field function is further formulated as Eq. Please refer to & 44 for more details of our design. TTe desired output is the spherical harmonic coefficients vector up to order L = described in $ 3.2 In practice, acoustic scattering corresponding different frequencies can exhibit different distri- butions. Therefore, we train several networks for different frequency bands and in this paper we present the results for 125 1000//. However, this range is not limit of our approach, as we can handle a wider frequency range, though the complexity of computing the training data can increase as cubic function of the frequency. We present novel network architecture to compute these characteristics for arbitrary or deforming objects.
4 NETWORK DESIGN AND PROPERTIES
In this section we introduce our shape Laplacian based point representation using implicit surface functions, For predicting ASF, [the detailed geometric characteristics of object representation play an essential role with respect to the simulation frequency.JThis motivates our design in terms of differential coordinates (Sorkine) 2006 Do Carmo| 2016), which describe the discretization of the curvilinear integral around a given point. Morcover, we also provide the proof of its permutation invariant property via Newton's ' identities in the appendix, [as that is important in the context of our sound propagation algorithm]. Finally, following de Finetti theorem (Heath & Sudderth 1976), we justify our design of implicit surface functions and [show its benefits in terms of accurately computing the acoustic scattering functions for arbitrary objects].
4.1 LOCAL SURFACE SHAPE AND IMPLICIT SURFACE ENCODER
Previous works point cloud learning algorithms mostly focus designing per-point opera- tions (Charles et al., (017), encoding per-point features to estimate continuous shape functions (Park) et al. 2019 Xu et al. 2209), or minimizing loss between point normal vector and its connected vertices (Liu et al. 2019). We extend these ideas to compute the acoustic scattering functions.
[R3Q1: For each point in the input cloud and its neighborhood in the Euclidean space, we assume that it can form a piecewise smooth surface around the point.]Fach point is encoded by the shared multi-layer perceptron (MLP) (Rumelhart
al. 1985) and can be represented by a vector in the higher dimensional latent space Z E R (see Figure 2h [R3QI: Thus, a piccewise-linear approx- imation of the surface around a given point can be used to estimate the local surface shape, where the differential coordinate] (Sorkine 2006 Do Carmo 2016) (i.e., 5- coordinates) of each vertex i can be expressed as:
オ = E [ - T)). d:
1
(4)
Here & encapsulates the local surface shape, (i))) represents the K nearest neighbors of vertex U; in the Fuclidean space, and d, [i())] is the number of immediate neighbors of es. To estimate
4

---

Under review as a conference paper at ICLR 2021
the mean curvature of the local surfaces formed by each point and its spatial neighbors, radial basis function (RBF) (o)) =epp -I-I" to weigh each vector, rather than using the uniform weight shown in Equation 4 Since there are !! permutations for a point cloud with N vertices, every operation on point clouds should be permutation invariant (i.e., the permutation of input points should not change the output of our network). [Therefore, we design an algorithm based on discrete laplacian and prove its permutation invariance property [.4 jOur weight function is designed to be positive definite and symmetric for any choice of data coordinates.
the
4.2 [R1Q4,R3Q3 4:00003--4IMPPIIIT SURFACES AND DISCRETE LAPLACIAN]
To encapsulate the local surface shape, each point e; is projected onto a higher dimensional space using MIPss, and the implicit surface is defined on the latent space z as shown on the top of F
Specifically, for one layer MLP, the representation is 2 relu पt 8, + b), where wt and b are learnable parameters our network. To calculate the d- coordinates of point v; and the closed simple surface curve around it, its immediate neighbors in the Euclidean space (i.e., N(u))) ) is used to evaluate Equation
cimplici (m))
Eje
S EjeNio
In the latent space, the local surface shape is encoded as an implicit surface where (2z;) is evalu- ated. The direction of the differential coordinate vector, also defined in Equation 5 approximates the local normal direction. Following Taubbn [995]. the discrete Laplacian of a point U; is given by
the weighted average over the neighborhood is represented as:
To compare the two å- coordinate representations, we highlight the error in decibel (dB) between the pressure fields reconstructed from groundtruth spherical harmonics term and the predicted ones using different neural networks in Table 2 We observe that gimplicit__ coordinates result in smaller errors. This indicates that our formulation provides a better approximation of ASFs.
4.3 [R1Q6-7, R3Q3-4: NEURAL NETWORK DESIGN]
Our neural network takes the point cloud as an N x 3 input where N represents the number of points in the point cloud. The output is the spherical harmonic coefficients ci". -1 S m I 1,0 < 1 < 3, resulting a vector of length 16. The network is illustrated in Figure
[ For each point is in the N x 3 point cloud, the discrete laplacian is evaluated on its immediate neighbors ((x)) according to Equation [ Then one layer of CNN and three layers of MLP are used to encode the piecewise-smane
local shape around 22 into a latent space Z C 128 128 For each 2, E 2, the discrete laplacian Equation (5) is further evaluated where N (us) (i.e. neighbors of i;
in the Euclidean space R9) and (2,) (i.c. neighbors of 2; in the latent space Z 2228) scales the 8- 128, coordinates R228 respectively, yielding 2K x 128 matrix. The final representation of 0 is shown below:
T पp ( T)²²
কल ( =R2
feature(us)
21 MjeNt 1t)) .....
Σjee(o) 03)
(6)
- =
T - -
ΣjEN(a))(2 z)
EjeN z))
where K = 5 is the number of the nearest neighbors and 128 is the dimension of the latent space.
Now we have the piecewise approximation of the local shape around i; as described in Equation 6 which composed of the center point z; E R228 and the 2K x 128 weighted discrete laplacian matrix. The vectors are concatenated and convolved with kernels of size [1, -2K), followed by two layers MLP and four fully connected layers with tanh activation functions since the value of of ranges from [-1, 1]. The output of the network is the spherical harmonics coefficients of and
the loss function is the L2- norm of the difference hetween the predicted et" and the ground truth calculated using FaSIBEM Acoustics solver
https::/ /www///wwwwww.o fastbem.com
5

---

Under review as a conference paper at ICLR 2021
RBF
Implicit Surface Encoder
Com2D charnel Kor si20. eridec)
18
864.
Latent Space Z E 12u
UL
11 D.10
11.1
8.30
RBF weighted coordinates
R) x
(1 + k) 128
X 128
((x:) in Euclidean Space [(v] -e -
N(z.) in Latent Space [(4 - 1f
2A - TIXL
Gonda (256,
ComvD 1532.
Consto (1026,
FC
256
128
16
11.210411 11.38
p.l. ELADK
DAL ELAD
Per-point feature (1 + 2k) x 12H
The input of our neural network is a N x point cloud (N 1024) and the output of our network the spherical harmonic coefficients harmonics). [R1Q6-7, R3Q3-4: We encode the piecewise smooth local shape around every point in the point cloud, forming a high dimensional latent space in R 125 The 8 coordinates where neighbors are found in the Euclidean space and the latent space are concatenated to represent the per-point feature. We leverage the geometric details by several Conv2D and MLP layers and output 16 spherical harmonics coefficients ct"
Figure 2: Network architecture:
vector of length (up 3rd order spherical
4.4 [R1Q2-5, R2: DISCRETE LAPLACIAN AND PERMUTATION INVARIANCE]
In this section we provide the proof of the permutation invariant property of the RBF-weighted discrete Laplacian Equation ( We only discuss the function composed of vectors in the high dimensional latent space projected by the implicit surface encoder shown on the top of Figure 2
Let fi(e)..... un) = SjeNee)) - m2), where vb is a continuous function composed of MLP layers. According to Kolmogorov-Arnoke representation theorem (or superposition theorem) and its relationship multi-layer neural networks [Kürková 1992), which states that every multivariate continuous function can be represented as a superposition of continuous functions of one variable, Ji(ti,.... ,N) can rewritten as: & (EN (v;)), where V : R3 + R22+1 and $ : R22++ - R3. In this paper, we do not project v onto R2N+1, +1; instead, we encode the v as = R First of all, we mapping defined by the coordinate functions. Sec-
show that ft can be regarded ond, we show that such mapping is injective and has continuous inverse mapping. The domain of Si is compact set, and thereby the image of fi is a compact set as well. Thus Si is homeomorphism between its domain and its image. Further, we demonstrate that fa is a permutation invariant continuous function since it can be considered as compositions of sum-of-power mappings.
sum-of-powe
Consider V (2) := [1, v² M1 V RR3 + MM1, where M is the dimension of our implicit surface encoder space. Let E(v)
= D.M ==1
V (U,)
M
M
M
(lu)
[M,
Uma
2
Um
M
mmm1
mm
(7)
[M (UL, 22,
UM (.)), (2 U2,-.- UM), ... .PM (M1, 02, ,UM)] [Eo(U1, U2, (MM) UM)) (V1, u2,-.- eeM), B2 "2 UM), -.. Exs (11, 1a (M)) (uM))
where p is the power-sums (defined in Equation 12 Appendix and we can observe that p is symmetric (i.e., does not change if we permute VI, , UM) and homogeneous of degree M. E, is (Appendix Ab. According Lemma
defined in Lemma 4.1
each element in E(v) injective. E(v) has inverse continuous mapping. Therefore, E is homeomorphism be-
Then by Lemma A.2
tween Ra and R T
Since & is the composition of several continuous functions vb. the continuity
of & holds as well.
Lemma 4.1. Let X (( RM) E [0, 1j : 21 S x2 S ... S S..). The sum-of-power mapping E X + M+1 defined by the coordinate functions :X, == Em (am)) ),
0, 0,.., M is injective.
6

---

Under review as a conference paper at ICLR 2021
The proof of the lemma is in Appendix A. We conclude that there are O and V such that fi is permutation invariant as long as V/ and & are continuous functions. To compute the acoustic seatter- ing functions, we further introduce the De Finetti Theorem and exhibit the connection between our design of Discrete Laplacian based implicit surface functions and pressure field approximation.
4.5 DE FINETTI THEOREM AND IMPLICIT SURFACE FUNCTIONS
Our idea is to approximate the pressure field defined by set of points. And we borrow power from the following theorem (Heath & Sudderth 1976]
Theorem 4.2 (De Finetti Theorem), sequence (2,, In) is infinitely exchangeable
p(Z1....,,..) In) P():))))((d))
(8)
for some measure P on 0.
Thanks to the continuity nature of neural networks, we replace P(d0) in Theorem ( with more general form P(f)dd since we assume that the underlying latent layer is absolute contimuous ac- cording Radon Nakon-Nikodym theorem (i.e., the distribution singular measures such as Dirac delta does not have density (Radon-Nikodym derivative). For ex- ample, each Radon measure (including Lebesgue measure. Haar measure, Dirac measure) can be decomposed into one absolutely continuous measure and one mutually singular measure.)). Then we derive that the acoustic scattering function of given set of points X' {21, 22, and the underlying latent space Z can be formulated in the form:
has density. [R1Q5: Note that some
Sseatter (2) - f d= II i=1
p(.
[Since our latent space vector z can be regarded as compositions of sum-of-power mappings (Equa- Appendix the output pressure field of our network is permutation invariant.]
tion
5 EVALUATION
5.1 DATA GENERATION AND TRAINING
We need a sufficiently large dataset for training and evaluating our method. We sample 100,000 3D objects from the ABC Datasel (Koch et al. 2019). All mesh files are pre-processed to be randomly rotated, scaled between 1 ~ 2m and centered. The FastBEM Acousties iis used to generate accurate acoustic scattering fields. Specifically, we simulate plane wave travelling to the -x axis direction, and all objects are assumed to have a zero Neumann boundary condition solve Equation ( While other boundary conditions are possible, we do not extensively study this variable in this work. After the acoustic seattering field has been solved at frequencies bands (125, 250, 1000) Hz, use the pyshtools package (Wieczorek & Meschede, 2018) to project the directional field to spherical harmonic coefficients ci". By using a maximum order of L = 3, we are able to maintain relative projection error below 2%. The original object meshes are also converted point clouds using furthest sampling for 1024 points. The data generation pipeline takes about 12 days using a desktop with 32 Intel(R) Xeon(R) Gold 5218 CPUs.
We split our dataset into training, validation and test set following a 8 network is trained on NVIDIA GeForce RTX 2080 Ti GPU and takes less than Ims per object for inference. The Adam algorithm is used in optimizer and it decays exponentially with decay rate and decay step equal to 0.9 and 10x #training examples, respectively. Our code and data will be released upon publication.
ratio. Our neural
5.2 RESULTS AND COMPARISONS
We evaluate our network performance on unseen objects from the ABC dataset, perform ablation study on RBF-weighted 5- coordinates and implicit surface encoders, and show that our network
https://www. http://wwww.aaabbb...oo.o
https://shton oca. www.we.w........................ shtool wwwothsspuullllllo// httpa//applllliiiiiioaeoc html

---

Under review as a conference paper at ICLR 2021
robust to Gaussian noise (~ (0,, 0.05)). We compare results with PointNet (Charles al. 2017) and DGCNN (Wang et al. 2019) in Table and Table 3 We justify the design of our network with this ablation study, including the use of 8- coordinates, RBF-weighted function well the implicit surface encoder, as highlighted in Figure We use PointNet, where only per-point MLP layers are applied on each point in the point cloud, and DGCNN, which dynamically build graphs according to K nearest neighbors, as the baseline. RBF-weighted 8- coordinates (as described in Equation (5), uniformly weighted Å- coordinates (described in Equation (4)) and implicit surface encoder shown at the top of Figure considered as subjects of ablation studies. We observe that our fine-gu ann-ggained geometric feature representation in Equation (6) results in larger reduction dB
enror. Moreover, our discrete Laplacian based representation is more robust to the Gaussian noise in the test set compared DGCNN.
Laia 1250
2521/4
M00
3000
Largel 3251
350
500 U
10001
PEEN
-00
-6
200 priue
-5
-00
orevic
Siinee 201
brigt 125kc
biget 335
300
piectae SARLE PHIGHERAIEE
SERGUE 2011
50
25
25
53
Figure 3: ASF prediction and groundtruth comparison: We expand the predicted spherical harmonics co- efficients onto a latitude-longitude (lat-long) map, representing the directional distribution of the scattering field. We randomly choose 6 point clouds from the test dataset, which have not been seen during training. Each point cloud is visualized from viewing angles, followed their groundtruthkargee. predicted, and error/difference lat-long maps in four frequency hands. We see a good match between our predicted fields the groundtruth fields.
5.3 APPLICATION IN INTERACTIVE SOUND RENDERING
[R1Q1:
Our geometric deep learning scheme facilitates a wide range of applications related to
wave acoustics simulation. Pre-computed acoustic transfer (Janes et al. 2006 Wang James) 2019 wave acoustics perceptual encoding (Raghuvanshi et al. (016) and wave-ray sound propaga- tion Yeh al. 2013), which heavily rely on extensive simulation using accurate wave solvers, can all benefit from efficient learning-basse based approach. Our method utilizes the connection between an object's 's shape and its scattering property to remove the pre-computation overhead in existing wave acoustics rendering methods. Our additional spherical harmonics encoding makes it suitable to be integrated into most wave-ray hybrid sound rendering engines.
To illustrate such application, integrate our predicted ASFs into state-of-the the---rttth-tttt ray- tracing based acoustic simulator (Schissler Manochal 2017). The ray-tracing based simulator assumes all sound waves travel rays. This works well for high-frequency sounds but do not hold for low-
8

---

Under review as a conference paper at ICLR 2021
Table 2: Ablation study: In this evaluation, we compare the performance on uniform 8- coordinate in Eq. 4. weighted 6- coordinates in Eq. (5) and implicit surface estimation in Eq. (6) on our test dataset (including 10k objects) frequency bands {125Hz, 250Hz, 500Hz, 1000Hz). The best result for cach froquency highlighted bold (lower error better). We alter between choosing Eq. ( and Eq. ( that results four different combinations: Row and 2, 3 and Next, we experiment the use implicit surface encoder (Row and 4). Our proposed network design (Row 4) shows superior performance in terms of ASF approximation for all four frequencies. A lower value indicates better result.
Methods
Error dB
Inference Time
weig-teighted 5-coord
implicit surface)
T25Hz
250Hz
5001IZ
1000Hz
#Params
sec. 10000frames
3.49
3.56
3.71
4.23
30k
3.38
3.41
3.57
4.47
30k
3.28
3.38
3.52
3.85
30k
(ours)
2.98
3.06
3,22
3.76
34k
47.10
PointNct Charles al. 2017
3.66
3.44
3.43
4.16
80k
8.95
DGCNN Wang al. 2019)
3.71
3.56
3.59
4.21
103k
26.80
Table 3: Robustness test: In this robustness test, we add i.i.d. noise to every point in the point cloud, which follows (0, 0.05). to the test Note that DGCNN finds K- neighbors each iteration as we did. However, algorithm more robust the noisy data input thanks the design of the shape Laplacian based point representation.
Error in dB
Methods
125Hz.
250Hz
500Hz
1000Hz
PointNet
5.88
5.50
6.02
8.37
DGCNN
7.91
6.65
7.31
11.32
Ours
5.20
5.09
4.70
5.51
frequency sounds. Instead, our goal is to use our wave-based simulator to better capture the sound propagation effects corresponding low frequency sounds in dynamic scenes. Therefore, we have simulated wave acoustics up to 1000//= to compensate the low-frequency effects like sound diffrac- tion using our learning method and compute the acoustic scattering functions of each object in the
scene. This choice of the simulation frequency of 1000H 2 is similar to Mehra et al. 2013). Raghu- vanshi & Snyder (2018), Rungta et al. (2018) and Morales et al. (2019). as they can provide plausible effects for games, virtual reality, and computer-aided design applications.
The specific wave-ray coupling scheme we use resembles Rungta et al. 20188) but can be switched to other compatible rendering engines. We use trained network at runtime predict ASFs of
static and dynamic objects, and evaluate the spherical harmonic expansion in Equation _ Then the sound is rendered at given listener locations. A summary of details also provided in Appendix B We demonstrate the final audio-visual rendering in our supplemental video.
CONCLUSION AND LIMITATIONS
We present a novel geometric deep learning approach that infers the 3D acoustic scattering field induced by an object from its shape. Our algorithm uses discrete-laplacian based implicit function encoders to better capture the geometric properties of an object and achieves lower error compared with existing general point-based frameworks. Our shape and sound field representation are well- suited for applications that require real-time wave acoustics characteristics.
Currently we study the case of sound-hard surfaces and a limited range of simulation frequencies, but these are not inherent limitation of our method. The simulation time needed create large training set can grow as a cubic function of the sound frequency. Incorporating more types of sound surfaces can further increase the training overhead of all geometric deep learning methods, including ours. In the future, we propose to extend these methods to real-world objects and use them for localization and oon-line-of-sight computations.

---

Under review as a conference paper at ICLR 2021
REFERENCES
Jakob AbeBer. A review of deep learning based methods for acoustic scene classification. Applied Sciences, 10(6), 2020.
Inkyu An, Doheon Lee, Jung-woo Choi, Dinesh Manocha, and Sung-eui Yoon. Diffraction-awanl sound localization for non-line-of-sight source. In 2019 International Conference Roboties and Automation (ICRA), pp. 4061-4067. IEEE, 2019.
Michael J Bianco, Peter Gerstoft, James Traer, Emma Ozanich, Marie A Roch, Sharon Gannot, and Charles-Alban Deledalle. Machine learning in acoustics: Theory and applications. The Journal of the Acoustical Society of America, 146(5)3550-36285 2019.
Dick Botteldooren. Fiite-diifeerence time-domain simulation of low frequecyyn room acoustic prob- lems. The Journal of the Acoustical Society of America, 98(6):3302-33)) 3308, 1995.
R. Qi Charles, Hao Su, Mo Kaichun, and Leonidas J. Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jul 2017. doi: 10.1109/cvp:2011..66 URL http: doi. org/10. 00099/VVRR.007...0
Manfredo P Do Carmo. Differential geometry of curves and surfaces: revised and updated second edition. Courier Dover Publications, 2016.
Ivan Dokmanić, Reza Parhizkar, Andreas Walther, Yue M Lu, and Martin Vetterli. Acoustic echoes reveal room shape. Proceedings of the National Academy of Sciences, 11060):12186-12119 2013.
James Eaton, Nikolay D Gaubitch, Alastair H Moore, Patrick Naylor, James Eaton, Nikolay D Gaubitch, Alastair H Moore, Patrick A Naylor, Nikolay D Gaubitch, James Eaton, et al. Estima- tion of room acoustic parameters: The ace challenge. IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP), 1681:)))::11116313 2016.
Ziqi Fan, Vibhav Vineet, Hannes Gamper, and Nikunj Raghuvanshi. Fast acoustic scattering using convolutional neural networks. In ICASSP 2020-2020 IEEE International Conference on Acous- fics, Speech and Signal Processing (ICASSP), pp. 171-175. IEEE, 2020.
Andrea F Genovese, Hannes Gamper, Ville Pulkki, Nikunj Raghuvanshi, and Ivan Tashev. Blind room volume estimation from single-channel noisy speech. In ICASSP 2019-2019 IEEE Inter- national Conference on Acoustics. Speech and Signal Processing (ICASSP), pp. 231-235. IEEE, 2019.
Rana Hanocka, Amir Hertz, Noa Fish, Raja Giryes, Shachar Fleishman, and Daniel Cohen-Or. Meshenn: a network with an edge. ACM Transactions on Graphics (TOG), 38(4):1-12, 2019.
David Heath and William Sudderth. De finetti's theorem on exchangeable variables. The American Statistician, 3084),188-18, 1976. doi: 10. 10.108800000311 www. www. www.daannooonnnnnnmmmmmm wtw doi abs 110. 111.0..110000001111000000 19w..9966.... 197771777.7755
URL https://
Doug L James, Jernej Barbič, and Dinesh K Pai. Precomputed acoustic transfer: output-sensitive, accurate sound generation for geometrically complex vibration sources. ACM Transactions Graphics (TOG), 25(3):987-995, 2006.
Taeyoung Kim, Youngsun Kwon, and Sung-Fui Yoon. Real-time 3-d mapping with estimating acoustic materials. In 2020 IFFF/SICE International Symposium on System Integration (SII), pp- 646-651. IEEE, 2020.
Sebastian Koch, Albert Matveev, Zhongshi Jiang, Francis Williams, Alexey Artemov, Evgeny Bur- naev, Mare Alexa, Denis Zorin, and Daniele Panozzo. Abc: A big cad model dataset for geometric deep learning. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
Heinrich Kuttruff. Room Acoustics. Taylor & Francis Group, London, U. K., 6th edition, 2016.
10

---

Under review as a conference paper at ICLR 2021
K Heinrich Kuttruff. Auralization of impulse responses modeled on the basis of ray-tracing results. Journal of the Audio Engineering Society, 41(11):876 880, 1993.
Véra Kúrková. Kolmogorov's 's theorem and multilayer neural melt networkss 506, 1992. ISSN 0893-6080. doi: http:///c..oog////.0ooo009990000 URL http:
Neural Networks, 5(3):50)
//www. www.sccienceeireeeeeoo rect com wtt.c///weeeeeeearrrree/////////
Jiaxin Li, Ben M. Chen, and Gim Hee Lee. So-net: Self-organizing network for point cloud analysis.
2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun 2018a. doi: 10. 1109/cvpr: 20.220018.0000
88. 5,00979. BRLLhtpp:///
10......//..1.11//VV..... 2w1.:///............. 00979
Yangyan Li. Rui Bu, Mingchao Sun, Wei Wu, Xinhan Di, and Baoquan Chen. pointenn: Convolu- tion on x-transformed points. In Advances in neural information processing systems, pp. 820-830, 20.
David B Lindell, Gordon Wetzstein, and Vladlen Koltun. Acoustic non-line-of-sight imaging. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6780- 6789, 2019.
Shichen Liu, Shunsuke Saito, Weikai Chen, and Hao Li. Learning to infer implicit surfaces without 3d supervision. In Advances in Neural Information Processing Systems, pp. 8293 8304, 2019.
Ravish Mehra, Nikunj Raghuvanshi, Lakulish Antani, Anish Chandak, Sean Curtis, and Dinesh Manocha. Wave-based sound propagation in large open scenes using an equivalent source formu- lation. ACM Transactions on Graphics (TOG), 32(2):19, 2013.
Ravish Mehra, Atul Rungta, Abhinay Golas, Ming Lin, and Dinesh Manocha. Wave: Interactive wave-based sound propagation for virtual environments. IEEE transactions on visualization and computer graphics, 21(4))434-442, 2015.
Hsien- Yu Meng, Lin Gao, Yu-Kun Lai, and Dinesh Manocha. Vv-net: Voxel vae net with group convolutions for point cloud segmentation. In Proceedings of the IEEE International Conference on Computer Vision, pp. 8500-8500 8508, 2019.
Kaichun Mo, Paul Guerrero, Li Yi, Hao Su, Peter Wonka, Niloy J. Mitra, and Leonidas J. Guibas. Structurenet. ACM Transactions on Graphics (TOG), 18 Nov 2019. ISSN 1557- 7368. doi: (0.1145/33550099333677, 3356527
http:/ttp//ee doi www.oogg11100 1ttp:/10..111/33330000
Federico Monti, Davide Boscaini, Jonathan Masci, Emanuele Rodola, Jan Svoboda, and Michael M Bronstein. Geometric deep learning on graphs and manifolds using mixture model enns. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5115- 5124, 2017.
Nicolas Morales and Dinesh Manocha. Optimizing source placement for noise minimization using hybrid acoustic simulation. Computer-Aided Design, 96: 1-12, 2018.
Nicolas Morales, Ravish Mehra, and Dinesh Manocha. A parallel time-domain wave simulator based on rectangular decomposition for distributed memory architectures. Applied Acousties, 97: 104, 2015.
Nicolas Morales, Zhenyu Tang, and Dinesh Manocha. Receiver placement for speech enhancement using sound propagation optimization. Applied Acoustics, 155:53-60 62, 2019.
Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. In Proceed- ings of the IEEE Conference on Computer Vision and Puttern Recognition, pp. 165-174, 2019.
Ville Pulkki and Peter Svensson. Maahine-lcarningg- based estimation and rendering of scattering in virtual reality. The Journal of the Acoustical Society of America, 145(4):2664-25) 26766 2019.
Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In Advances in neural information processing systems, pp. 5099-5108, 2017.
11

---

Under review as a conference paper at ICLR 2021
Nikunj Raghuvanshi and John Snyder. Parametric wave field coding for precomputed sound prop- agation. ACM Trans. Graph.. 33(4):38:1 -88:11, July 2014. ISSN 0730-0301. doi: 10. 145/ 2601097.2601 1884 URL /ttt::://dl. acm. acm. 10. 1145/ 201997 26011188
Nikunj Raghuvanshi and John Snyder. Parametric directional coding for precomputed sound prop- agation. ACM Transactions on Graphics (TOG), 37(4):108, 2018
Nikunj Raghuvanshi, Andrew Allen, and John Snyder. Numerical wave simulation for interactive audio-visual applications. The Journal of the Acoustical Society of America, 139(4):2008-2)) 2009, 2016.
David E Rumelhart, Geoffrey E Hinton, and Ronald J Williamss Learning internal representations by error propagation. Technical report, California Univ San Diego La Jolla Inst for Cognitive Science, 1985.
Atul Rungta, Carl Schissler, Nicholas Rewkowski, Ravish Mehra, and Dinesh Manocha. Diffrac- tion kernels for interactive sound propagation in dynamic environments. IEEE Transactions on Visualization and Computer Gruphics, 24(4):1613-1622, 2018.
Carl Schissler and Dinesh Manocha. Interactive sound propagation and rendering for large multi- source scenes. ACM Transactions on Graphics (TOG), 36(1):2, 2017
Louena Shtrepi, Arianna Astolfi, Sönke Pelzer, Renzo Vitale, and Monika Rychtáriková. Objective and perceptual assessment of the scattered sound field in a simulated concert hall. The Journal of the Acoustical Society of America, 138(3):1455-1497, 2015.
Vishwanath A. Sindagi, Yin Zhou, and Oncel Tuzel. Mvx-net:
Multimodal voxelnet for 3d object de-
tection. 2019 International Conference on Robotics and Automation (ICRA), May 2019. doi: 10. (019/icra2019.879497 URL / dx doi www.or..gooooooo..oo 109/ICRA, 201 9.8794111
Dmitriy Smimov, Matthew Fisher, Vladimir G. Kim, Richard Zhang, and Justin Solomon. Deep parametric shape predictions using distance fields, 2019.
Olga Sorkine. Differential representations for mesh processing. In Computer Graphics Forum, volume 25, pp. 789-807. Wiley Online Library, 2006.
Auston Sterling, Justin Wilson, Sam Lowe, and Ming C Lin. Isnn: Impact sound neural network for audio-visual object classification. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 555-572, 2018.
Qingyang Tan, Lin Gao, Yu-Kun Lai, Jie Yang, and Shihong Xia. Mesh-based autoencoders for localized deformation component analysis. In Thirly-Second AAAI Conference on Artificial Intel- ligence, 2018
Zhenyu Tang, Nicholas Bryan, Dingzeyu Li, Timothy Langlois, and Dinesh Manocha. Scene- aware audio rendering via deep acoustic analysis. IEEE Transactions Visualization and Com- puter Graphics, 2020.
Gabriel Taubin. A signal processing approach fair surface design. In Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, pp. 351-358. ACM, 1995.
Lonny L Thompson. A review of finite-element methods for time-harmonic acoustics. The Journal of the Acoustical Society of America, 11963:1315--330, 2006.
Dimitrios Tsokaktsidis, Timo Von Wysocki, Frank Gauterin, and Steffen Marburg. Artificial neural network predicts noise transfer as a function of excitation and geometry. In Proc. International Congress on Acoustics (ICA), pp. 4392-4396, 2019.
Jui-Hsien Wang and Doug I. James. Kleinpat: optimal mode conflation for time-domain precompu- tation of acoustic transfer. ACM Trans. Graph., 38(4):122-1, 2019.
Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E. Sarma, Michael M. Bronstein, and Justin M. Solomon. Dynamic graph enn for learning point clouds. ACM Transactions on Graphics, -8, Oct 2019. ISSN 0730-0301. doi: 10.1145/332666. URL http:/ ///:://xxxxc doi. 3rc/100.145/332655
12

---

Under review as a conference paper at ICLR 2021
Mark A Wieczorek and Matthias Meschede. Shtools: Tools for working with spherical harmonies. Geochemistry, Geophysics, Geosystems, 1908)22574-2592,2222 2018.
Luiz C Wrobel and AJ Kassab. Boundary element method, volume 1: Applications in thermo-fluids
and acoustics. Appl. Mech. Rev. 56(2):B17-B11 2003.
Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1912-1920, 2015.
Qiangeng Xu, Weiyue Wang. Duygu Ceylan, Radomir Mech, and Ulrich Neumann. Disn: Deep implicit surface network for high-quality single-view 3d reconstruction. In Advances in Neural Information Processing Systems, pp. 490-500, 2019.
Hengchin Yeh, Ravish Mehra, Zhimin Ren, Lakulish Antani, Dinesh Manocha, and Ming Lin. Wave-ray coupling for interactive sound propagation large complex scenes. ACM Transac- tions on Graphics (TOG), 32(6):1-11, 2013.
Li Yi, Hao Su, Xingwen Guo, and Leonidas Guibas. Synespeccnn: Synchronized spectral cnn for 3d shape segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2282-2290, 2017.
Zhoutong Zhang, Jiajun Wu, Qiujia Li, Zhengjia Huang, James Traer, Josh H McDermott, Joshua B Tenenbaum, and William T Freeman. Generative modeling of audible shapes for object percep- tion. In Proceedings of the IEEE International Conference Computer Vision, pp. 1251-1260, 2017.
Yin Zhou and Oncel Tuzel. Voxelnet: End-to-end learning for point cloud based 3d object detection. 2018 IEEF/CVF Conference on Computer Vision and Puttern Recognition, Jun 2018. doi: 10. 1109/cvpr.2018..mmm 20.020722. http://t://d do.. org/100.010... 1109 2 00472]
Branko Curgus and Vania Mascioni. Roots and polynomials as homeomorphic spaces. Expo- sitiones Mathematicae, 24(1):81 95, 2006. ISSN 0723-0869. doi: https://doi.org/10.00166
j.cxmath. 2005.07..00 URL http:://///// //tt::://ww../// www.wssieeenerrr.....mmmmm http:a/wweirrrec.ccccoeaeeeeeeee htt.://c....ccccccccee//rrrrraa
p11/507230855050000
13

---

Under review as a conference paper at ICLR 2021
Appendices
A PERMUTATION INVARIANCE
In this section we provide the proof of permutation invariant property of the RBF-weighted discrete
Laplacian Equation O thanks to Newton's identity. First of all, Sec. A. serves as recap of Newton's 's identity, which is applied in the proof of Lemma oower mapping defined Lemma provide the background for the permutation invariance of our Discrete Laplacian in Equation d Furthermore, we demonstrate with the De Finetti Theorem that the acoustic scattering functions can be approximated using the underlying latent space.
Lemma A.2 shows that the sum- has continuous inverse mapping. Both of the lemmas
A.1 NEWTON'S IDENTITIES
Newton's identities formulas, also known theory of equations, build bridge between elementary symmetric polynomials and power sums. Consider polynomial of degree n with roots T1, " Fn. Assume that so 1, monic.
f(x) = S0x" + ... 8n-12 8n
n
(x - xi}
(10)
8, (-1)" Σ
Zjj Bj
8, 8, (-1)"
Note that and s are symmetric (i.e. does not change if we permute ".,"" and homoge- neous degree r.
These & polynomials, often called elementary symmetric polynomials, form a bases for all sym- metric polynomials.
For example,
$1 (-1) (I1 ... 2n)
= 31 2143 + 31 +...
83 (-1) (IiZgTS + x1 + 212225 +...)
(11)
8n = (-1)" * (112203
Another family symmetric polynomials are power sums, which form basis for the space of all symmetric polynomials:
pr(21, ,.,) = x +... x""
(12)
The transition formulas between power sums and elementary symmetric polynomials are called Newton's identities.
Lemma 4.1 Let X = (x11 (.M) E 1] æ S x2 ... ... S TM.. The sum-of-power
mapping E: X - RM++ defined by the coordinate functions
(X)
(xm)) 9.9 0,... M
(13)
injective.
14

---

Under review as a conference paper at ICLR 2021
Proof: For some u, X, Eu)) E()), show
Let u, u be the roots polynomial degree M, we have:
Af
fa(x)
II (ar Um m=1
M-
+ aM-11 aM
(14)
[o(x) II (x Um)
2 by 2 + bM--1 bM
a, are elementary symmetric polynomials as described in Equation 10 als(i.e. a, b) can be expressed power sums.
Elementary polynomi-
(u)
1
0
0
E2(u)
Ei(u)
2
0M M!
(M-
(u-2(u)
EM-s(u)
EM-4 EM-3(u)
M
EM (u)
Ea--(()) EM-1(u)
(1(
(15)
En
(w)
(v
Ei(v)
2
0
bm
M!
EM-1 (v)
(
EM-a(D)
EM-4(U)
M M-1 E1(u)
(D)
EM-1(W)
EM-2(D)
EM-s(U)
E(u) E(v) implies that a b. and therefore u U.
Theorem A.1. The function f CM + CM,, f(c)(()))))))))))))) = x + Cir M-1 +... + (-1) cas-13 --M----------"
(-1)Mca car, c E C is homeomorphism (Curgus & Mascioni 2006).
Lemma A.2. The sum-of-powed mapping defined Lemma has continuous inverse mapping.
The domain of E is compact set, and E is continuous function. Therefore, the image of E is compact set. From Lemma 4.1 is continuous function of the power sums E.
Goal: Show continuity of inverse mapping of E.
Proof: From Theorem A.I the continuity of roots u depends on the coefficients a. W.L.G, show continuity from to u. Due to the nature of homeomorphism, the mapping from u to a as well as its inverse mapping from a to u are both continuous.
B HYBRID WAVE-RAY SOUND RENDERING
(w,w)
p(9,, 04.
Specular
reflection
Incoming ray
1l0,, or w)
Diffuse reflection
Incoming ray
plaa (5.w)
Geometric Ray-tracing
Hybrid wave-ray interface
Figure 4: Pure geometric ray-tracing (left) hybrid wave-ray coupling (right).
Conventional ray- ray-tracing algorithm finds many ray paths from a sound source to receiver/lsttnerr and time delay and energy damping is calculated according to the path length and the order of
15

---

Under review as a conference paper at ICLR 2021
reflections of each path. These information be used to compose acoustic impulse response for sound rendering in real-time (Kuttruff 1993).
The of the ray-tracing process is sample large number of reflection directions and their corresponding energy when ray hits a surface. pure geometric sound propagation system as- sumes specular and diffuse reflections (Figure ( In this setup, the reflected rays can only be in directions which have positive dot product with the normal direction the hitting point. There- fore, it is impossible for rays "bypass" obstacle, while for wave acoustics, such phenomenon (diffraction) always happens. As an alternative, the acoustic scattering field (ASF) we computed can be used model diffraction effects. Specifically, when an incoming ray hits sound scatterer, we sample outgoing directions among all directions, and calculate the energy decay by evaluating the ASF each direction (0;, (e) for frequeney w (Figure right). This hybrid wave-ray propagation formulation is general enough to be used with most sound propagation engines.
(a) Floor: One static sound (b) Havana: Two moving (c) Trinity: Six flying ob- (d) Sibenik: Two dis- scatterer in open space. walls in half-open space. jects in large indoor joint revolving objects in 10.6 00.65ms/framc..
6.78ms/frame.
room. 12. 12.95ms/frmme.
church. 6.87ms/frame.
Figure 5: Benchmark scenes used for audio-visual rendering in our supplemental video. These dynamic scences where objects come in close proximity and change topologies. Our learning methods compute accurate scattering fields such real-time applications.
We also test sound rendering on four benchmark scenes of different complexity in Figure 5 to per- ceptually show the smooth and realistie sound rendering results of method.
16


Reviews:
[{"review_id": "tWPKgjQukA", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary: This paper outlines a geometric deep learning approach for predicting acoustic scattering fields at different frequencies. The fields are represented with spherical harmonic coefficients (only the directional parts), and they use a nearest neighbor approach to construct an implicit surface in a latent space, and leverage a Laplacian in this domain. In their experimental section, they show superior results to other geometric deep-learning architectures that use point clouds, and also perform an ablation study.\n\nStrengths: The results for the method seem to beat out other similar point-cloud-based methods and I appreciate the application to a specific sound-modelling problem.\n\nWeaknesses: I found their description of the geometric encoding part to be sparse on the expository front. Their use of differential coordinates, use of a surface in latent space, and the construction of their feature vector are presented without much discussion of motivation. It would help to compare and contrast with existing point-cloud based methods.\n\nRecommendation: I gave a rating of 6, as I found it difficult to see what insights I should glean from their method. On the other hand, their method seems to work well from a practical standpoint, and has a concrete application that seems useful. For a higher score, I think the paper could benefit from a clearer exposition and a more extensive appendix describing the details of their network architecture. This could include a clearer delineation of the differences between their method and similar geometric deep-learning models.\n\nTypos:\n* typologies -> topologies in 3.3\n* exponential powers squared in (5)?", "extra_scores": {}, "extra_text": {}}, {"review_id": "fJrQZfoYaO", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This work introduces a new architecture for predicting the acoustic scattering fields from an object and an incident plane wave. This is done by describing the object using a point cloud mesh whose local geometric properties is encoded in a set of latent space vectors and processed by a deep neural network. The resulting architecture can predict the acoustic scattering fields quickly with high accuracy for a given test set.\n\nThe problem formulated is an interesting one and the numerical results are very impressive. The description of the method, leaves quite a bit to be desired. The details of the network are described in vague terms and when formulas are introduced, much of the notation is not defined or not well motivated. As a result, it becomes very hard to understand exactly how and why the proposed method works. For this reason, I do not recommend that this work be published as part of the proceedings.\n\nAs stated above, the numerical results are impressive. The authors state that their network is able to calculate the acoustic scattering field in less than one millisecond for a given object. It is not clear how this compares to the baseline method or the state of the art (PointNet and DGCNN). Section 5.1 mentions it taking twelve days to generate the ground truth data set of 100000 objects, which comes out to about 10 seconds, so this would be a considerable speedup. Still, it is not clear how much faster the proposed method is compared to the state of the art.\n\nThe main problem is the description of the method, starting in eq. (4), where “differential coordinates” are defined. These seem to be vectors in R³, but what is their purpose? What do they encode? Then a variant of this is introduced in (5), where a learned latent space is used in the form of the z coordinates. How exactly are these latent space coordinates learned? What is the objective? Then the vector in eq. (6) is introduced. It is sometimes referred to as a discrete Laplacian, but it is not clear how (in the equation, it is simply labeled “feature”). Sections 4.4 and 4.5 are similarly impenetrable. Furthermore Section 4.5 refers to Appendix B, which defines a theorem, but which has no proof.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "CZWSu6EENqX", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes to train a neural network to predict the acoustic scattering effects of a 3D object. The input of the system is a 3D point cloud representing the object, and the outputs are 16 coefficients from the spherical harmonic decomposition of the sound field resulting from an incoming planar wave to the object at a given frequency (4 frequencies are considered, with one specialized network trained for each). The main methodological contribution is a carefully crafted learned representation of the 3D point cloud. Reported results show that the proposed architecture outperform two other architectures while using less parameters and is able to predict sound fields with relatively small errors, using orders of magnitude less computational time than the acoustic simulators used to train the network.\n\nPros:\n- The proposed 3D point cloud representaiton is interesting and potentially applicable to other tasks\n- The general idea of using deep learning to accelerate acoustic simulation is interesting\n- Reported results and comparisons look reasonnable\n\nCons:\n- The scope of the paper is very narrow, making it of limited interest. The approach is only tested on simple simulations (no real data involved) with rather unrealistic assumptions (zero Neumann boundary, perfect plane wave, SH decomposition to order 3 only) and few frequency bands (125,250,500,1000 Hz), but several days of computations to generate the training data are still reported by the authors. It is not clear if and how the proposed approach would scale to more realistic/useful settings.\n- I found sections 4.4, 4.5 and the Appendix cryptic. A context of what exactly is discussed here and how it relates to the proposed method is seriously missing.  Two obscure lemmas and a thorem are packed in a dozen lines of text without any references. I was not able to follow the mathematical soundness of this, or indeed even to understand how it precisely relates to the rest of the paper.\n- In general the paper looks like it has been typed in a rush, with some typos but more importantly several technical inconsistencies, making ts soundness hard to verify. One important source of confusion is the radial basis function \\phi. Below equation (4) it is defined as exp(-||.||^2). In equation (5) an exponential of norm is used but this time with no square. This is again used in the denominator of the first line of eq. (6), although this is presumably a typo? Then in section 4.4, \\phi is defined as a function of completely different nature, from a scalar to a vector of powers, alllegedly \"without loss of generality\". I do not see how to make sense of all these different definitions of phi. Another issue are the cryptic 4 lines preceding Lemma 4.2, ending with \"the distribution on \\theta has density\" : has density what? Also, below equation (6), the authors write \"Equation (6), is further fed into the MLP, forming the differential coordinates in Equation (5)\" it is not clear in what sense eq. (5) relates to the MLP or equation (6). Last, details are missing in the description of the neural network architecture: in what precise sense are the MLP weights shared? Over which dimension is the Max pool applied? What non-linearities are used?\n\nOverall, I feel like the flaws of the paper outweights its strength. It would benefit from an expansion to a longer format in order to better explain the theoretical and methodological aspects and extend experiments to more diverse and/or realistic settings.\n\n===== Edit after authors' revisions ======\nThe authors made some efforts to improve the exposition of the most obscure parts of the paper, but in my opinion this is not sufficient and I am still not able to fully grasp the connection between section 4.4 and 4.5 and the rest of the paper. The added parts (in blue) contain additional typos and, like the rest of the paper, look like they have been type in a rush. For instance:\n- \"First of all, we show that f_i can be regarded as a sum-of-power mapping defined by the coordinate function\" -> I don't see where this is showed in the paper\n- \"f_i(v_1, · · · , v_N ) can rewritten as\" -> can be rewritten as\n\nI'd encourage the authors to resubmit their work using a longer paper format that would allow them to better expose the details of their investigation.\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors were responsive to the comments of the reviewers, both in
the rebuttal and in the revision to the manuscript.  However, the
reviewers were still concerned about the lack of clarity of the
manuscript, the motivation for the design decisions, and errors,
present also in the rebuttal revisions.

Decision: Reject