Blaschke Product Neural Networks (BPNN): A Physics-Infused Neural Network for Phase Retrieval of Meromorphic Functions
Authors: Juncheng Dong
Simiao Ren
Yang Deng
Omar Khatib
Jordan Malof
Mohammadreza Soltani
Willie Padilla
Vahid Tarokh
Venue: ICLR 2022 Poster (2022)
Keywords: Blaschke Product
Neural Network
Phase Retrieval
Metamaterial
Meromorphic Functions

Abstract: Numerous physical systems are described by ordinary or partial differential equations whose solutions are given by holomorphic or meromorphic functions in the complex domain. In many cases, only the magnitude of these functions are observed on various points on the purely imaginary $j\omega$-axis since coherent measurement of their phases is often expensive.  However, it is desirable to retrieve the lost phases from the magnitudes when possible. To this end, we propose a physics-infused deep neural network based on the Blaschke products for phase retrieval. Inspired by the Helson and Sarason Theorem,  we recover coefficients of a rational function of Blaschke products using a Blaschke Product Neural Network (BPNN), based upon the magnitude observations as input. The resulting rational function is then used for phase retrieval. We compare the BPNN to conventional deep neural networks (NNs) on several phase retrieval problems, comprising both synthetic and contemporary real-world problems (e.g., metamaterials for which data collection requires substantial expertise and is time consuming). On each phase retrieval problem, we compare against a population of conventional NNs of varying size and hyperparameter settings. Even without any hyper-parameter search, we find that BPNNs consistently outperform the population of optimized NNs in scarce data scenarios, and do so despite being much smaller models. The results can in turn be applied to calculate the refractive index of metamaterials, which is an important problem in emerging areas of material science.

Published as a conference paper at ICLR 2022
BLASCHKE PRODUCT NEURAL NETWORK (BPNN): A PHYSICS-INFUSED NEURAL NETWORK FOR PHASE RETRIEVAL OF MEROMORPHIC FUNCTIONS
Juncheng Dong, Simiao Ren, Yang Deng,
Omar Khatib, Jordan Malof, Mohammadreza Soltani,
Willie Padilla & Vahid Tarokh
Department of Electrical and Computer Engineering
Duke University Durham, NC 27708, USA
ABSTRACT
Numerous physical systems are described by ordinary or partial differential equa- tions whose solutions are given by holomorphic or meromorphic functions in the complex domain. many cases, only the magnitude of these functions are ob- served on various points on the purely imaginary jw-axis since coherent mea- surement of their phases often expensive. However, is desirable to retrieve the lost phases from the magnitudes when possible. To this end, we propose a physics-infused deep neural network based on the Blaschke products for phase re- trieval. Inspired by the Helson and Sarason Theorem, recover coefficients of rational function of Blaschke products using a Blaschke Product Neural Network (BPNN), based upon the magnitude observations input. The resulting rational function is then used for phase retrieval. We compare the BPNN to conventional deep neural networks (NNs) several phase retrieval problems, comprising both synthetic and contemporary real-world problems (e.g., metamaterials for which data collection requires substantial expertise and time consuming). On each phase retrieval problem, we compare against a population of conventional NNs of varying size and hyperparameter settings. Even without any hyper-paramette search, we find that BPNNs consistently outperform the population of optimized NNs in scarce data scenarios, and do so despite being much smaller models. The results can in turn be applied to calculate the refractive index of metamaterials, which is an important problem emerging areas material science.
INTRODUCTION
Numerous physical systems are described by ordinary or partial differential equations whose solu- tions are given by holomorphic or meromorphic functions f(2) = If(2) exp(j0(2)) in the complex domain, where [f()) and ((2) are respectively the magnitude and phase of f()). In many cases, only (f((ww)) the magnitude of (=)), is observed on a finite set of points (52) of the purely imaginary jw-axis since the coherent measurement of their phases may be expensive (see Ch. 13 of Stern (016)) or require significant expertise (King) 2009). However,
desirable to retrieve the lost phase 0(jw) from measurements of the magnitudes [fw] for E Ω, since the phase often en- codes important information. For example in both spectroscopy and optical imaging, estimation of ((jw) is often not possible. However, phase information determines the optical constants in the spec- troscopy of materials (Lucarini 2005). and is more important than [jw]] in optical imaging (Ster) 2016). The phase retrieval problem is also important in a number of other disciplines, including holography (Fienup) 1982), x-ray crystallography (Nugent) 2007), astronomy (Stark) 1987). blind deconvolution (Shechtman et al. 2015). and coherent diffractive imaging (Latychevskaia. 2018).
For readers unfamiliar with phase retrieval problems and have questions about why function values only observed on the purely imaginary line, please refer to the explanation we have included in the Appendix.

---

Published as a conference paper at ICLR 2022
Many techniques for phase retrieval have been developed in the literature. The Kramers-Kronig Kronig formula, see Lucarimi (2005) (also known as Sokhotski Plemmelj formula), is noteworthy and is de- rived by the application of the Hilbert Transform to log [jw) log [f(w]] j0(jw). Unfortu- nately, for phase retrieval, this elegant formula requires knowledge of the magnitude [f(jw) along the entire ju-axis. However, mentioned above, acquiring these values is often impossible since Q is typically a finite set with a small number of elements. This severely limits the application of Kaaerr-KKroni
formula in many applications. Another approach based on compressed sensing, but it requires sparsity assumptions, and stringent assumptions on the design projection matrix (i.e.. Restricted Isometry Property (RIP) (Candes) 2008), mutual incoherence (Foucart & Rauhut 2013). etc) that often do not hold for meromorphic functions encountered in practice.
To overcome the above limitations, phase retrieval can also cast as a data-driven regression prob- lem, where we wish to learn mapping from the magnitude of a signal, X E R to its phase, O E C.N. If the number of frequencies (192) measured N then the phase can be represented as vector of complex numbers θ = [exp(O(jwi)),
exp(0(jwx)) E CN, where exp(O(jw;) is the phase at i-th frequency. Similarly, the signal's s corresponding magnitude vector can be given by ( = 1, ... .N, and the goal is to infer @ based upon X. We can then learn this mapping using a data-driven model (see Figure below for an example).
Predicted Real Part
Magnitude
FCNN
R.V
C
Predicted Imaginary Part
Figure 1: Example the deep neural network approach to phase retrieval problem. FCNN refers fully-connected neural network here but can be replaced by any machine learning model. The mag- nitude encoded into an input of size while the output is an N-dimensional complex vector of corresponding phases.
With sufficient amount of training data, it is likely that universal function approximator (i.e. neural network), can learn the underlying function well within a tolerance range. For instance, deep learning has shown great success phase retrieval problems in the imaging area where abundant training data is available (Sinha et al. 2017 Metzler et al. 2018). But sufficient training data can be expensive acquire in many practical applications, or sometimes even impossible.
In this paper, we depart from the existing literature by exploring the applications of physics-infused neural networks to this setting. The infusion of physics reduces the number of required training examples to a small enough amount that is practical for many measurements and design applications of interest. The main idea of the approach described next. Given the fact that any phase value has magnitude of (i.e.) ft.e/eep(jjee exp(je(jw)) Helson and Sarason are then satisfied indicating that exp(j0(jw)) can be approximated by rational function whose numerator and denominator are both Blaschke products (Garcia et al.) (2066). This then used design neural network for calculating coefficients of these Blaschke products which may be be used for phase retrieval.
for all w), we observe that conditions of a Theorem of
In Section 2, we briefly review conventional phase retrieval methods, and existing applications of the neural network to phase retrieval problems. In Section 3, we briefly review Blaschke products and the Kelson-Sarason Theorem. In Section 4, we propose our approach for baking these results into neural networks resulting in Blaschke Product Neural Networks. In Section 5, provide experimental results on various synthetic and metamaterials datasets along with a inclusive set of baselines to demonstrate the performance improvements of construction.
2

---

Published as a conference paper at ICLR 2022
2 RELATED WORK
We first briefly review existing methods of phase retrieval applicable to phase retrieval of analytic and meromorphic functions of interest.
Perhaps the most important result in this area is the elegant calculation of Kramers and Kronig (also known as Sokhotski-Plent Plemelj formula) obtained by the application of the Hilbert Transform to log f(j) = log [f(jw]] + j0(jw) and is given by
20 P.V.
log |fG3) d3, 3 w
0(jw)
where P.V. denotes Cauchy's 's principal value. We note that this can also be derived by invoking the Cauchy integration formula along path containing the imaginary jw-axis.
However, there is a major obstacle in the application of this elegant formula. In many important practical scenarios, the set Ω of frequencies (points jw-axis) that have measurements of integral
[f(j]] has a small cardinal number. In such cases, the approximation of Kramers-Krow by a Riemann Sum produces unsatisfactory results. Other approaches used analytically extend or extrapolate [jjw] have also demonstrated limited success (Chalashkanov et al. 2012).
Another approach phase retrieval is by sparse representation. This approach assumes that is priori known that f()) ) has a sparse representation in a known dictionary of functions D = (+), = 2....... .M i.e. can be written as a linear sum of elements of ID with only a small number of non-zero coefficients. Under this assumption, many techniques are developed to recover these coefficients and the phase of f(z). Again, this is an interesting proposal but the underlying assumptions may not hold in many problems of interest. In particular, neither a dictionary D may be known nor classes of functions of interest may be sparse in any fixed dictionary. Recently, some new techniques based on deep learning have been developed to learn the structure of the underlying signal which we want to recover its phase. That is, deep neural networks are used to encode the prior knowledge on the structure of the signal instead of hard-coded assumptions such as sparsity (Jagatap) & Hegde, 2019). However, these methods require a significant number of training samples from the magnitude of the underlying signal, which limits the applicability of these approaches on material science applications.
Recently, DL has demonstrated its capability to solve the phase-retrieval problem, especially in the imaging area: [inha et al. (017)) achieved lensless imaging for phase objects utilizing CNN. Xue et al. (019) constructed a bayesian convolution neural network achieve phase imaging while quantifying the uncertainty of the network output. |Mtzler et al. (2018) combined regularization-by denoising framework with a convolution neural network for noisy phase retrieval for images. Goy et al. (2018) achieved low photon count phase retrieval with Convolutional Neural NetN)rk(CNN) as well. To avoid the difficulty of getting a large set of paired phase and intensity measurements, Zhang et al. Zhang et al. (2021) used an unpaired dataset for phase-retrieval. Other works of Tayal jet al. (2020): Houhou et al. (020) have also used deep learning for end-to-end phase recovery. None of these works use the underlying physics in their deep learning approach or they require large number of training samples; hence, are not directly applicable to the physics applications we interested in this paper. We took different path of applying physical prior knowledge to our phase retrieval process.
jutzue,
One closely related line of work works (Dissanayake & Phan- Thien 1994; Berg & Nyström 2018 Zhang & Bilige, 2019 Chen al. 2018). However, these methods typically need lot of data and require careful selection of p additionally, they do not consider directly solving the phase retrieval problem. Our approach, on the other hand, needs no specific hyper-parameter search algorithm, and achieves high-accuracy result with very limited training samples.
is solving partial differential equations using neural net-
MATHEMATICAL BACKGROUND
We briefly review the mathematical background required in the rest of this paper. In particular, review Blaschke products and the Selson-Sarasso Theorem. We note that the standard presentation of the Blaschke products is motivated by inner functions in Hardy spaces presented over the unit disk
3

---

Published as a conference paper at ICLR 2022
whose boundary is the unit circle (Rudin 1987 Garcia et al. 2016). In case, are interested in the right hand half plane whose boundary is the jw-axis. These two are equivalent using the complex Mobius transformation (z + 1) 2). In this light, our exposition is closer to Akutowiez [956).
Let 01 02, az, ... be an infinite sequence of numbers for which R(a)) 2 0 for all for which
R(aj) +1
< 00,
then for any integer n, any complex function given by
B(²) = ( II a 1
la + 1
0j +1 dj
is referred to as a infinite Blaschke Product. If the set { 42, aa } is finite, B(2) :) is referred to R(e,) as finite Blaschke product. For a finite Blaschke Product, it is clear that Σラ- la P4, so since there are only finite number of coefficients aj, and therefore this condition can be dropped. Note that roots with R(ay) 0 do not contribute to the product.
For both finite and infinite Blaschke products, we can see that [B(=) = for 2 = jز on the purely imaginary axis In this paper, we are only interested in finite Blaschke products because of the following approximation Theorem of Helson and Sarason.
Theorem Let u(jw) be any continuous function on the jw- for which [u(jw)] 1 for all w. Let e > O. Then there exists finite Blaschke products B, (jw) and B2(jw) such that
(iw) B2(ww)
<E
where the || o/ denotes the standard 1 norm computed over the ju axis.
Let b(jw) = exp(j8(jw)). We know that [b(jw)] = because [b(jw)] = exx(()((j))] = coss) j (in() (co)) 2 + sin (iw))
1. Applying the Pelson-Sarason Theorem, for analytic or meromorphic functions f(jw) (with no poles on the imaginary axis), the phase b(jw) can be approximated by rational function of finite Blaschke products. The main theme of this paper is that these Blaschke products can be in turn computed using neural networks with a small amount of data.
From the expression of finite Blaschke products, we can see that B((jw) = A.P.( for i 1.2 where A; complex number with |A,] = and P3 (j(j)) is complex polynomial of and (2) = (-)), where (j(jw) is constructed from P.(jw) by replacing all its coefficients with their respective conjugates. Note that (jw))"
P(jw). Let A A1/ A2 and represent the Blaschke products B₁ and B, in the above theorem with the complex polynomials, we have the approximating rational function written as AP, (jw)) ) P5 (jw)/ PT (2jw) with A Thus we have the estimation for phase function b(jw) as:
(i b(jw) R Pr(jw) Pa(ww = P*(jw)'
P(jw)
(I)
where P(z) (1 and (2) and P (z) correspond to respectively factors of P(z) with roots with positive and negative real parts.
3.1 BLASCHKE PRODUCT NEURAL NETWORK
Inspired by the above, we model the phase function by equation I and propose the Blaschke Prod- uet Neural Network (BPNN). A BPNN is neural network with the magnitude of observations as the input and the coefficients of the rational function of Blaschke products as the output. In this paper, we only present BPNNs which are fully connected neural networks (FCNNs), although such a restriction is not necessary. We next describe the operation of BPNNs by discussing the forward and backward pass mechanisms.
Details proving [Bjw)) be found in the appendix
4

---

Published as a conference paper at ICLR 2022
Forward Pass The output of BPNN are Blaschke coefficients of the complex polynomials that in turn used to construct predicted phase function b(jw) according to equation | Without loss of generality, let Ω (Wi, w2, wN) be the set of frequencies for which experimental results measuring the magnitude of signals [f(jw) is performed, and their corresponding phases must be retrieved. be assumed that wi < wi
wN. The BPNN then produces estimates ((jw,)) of the phase b(ju)). Combining with the known magnitude at wi, this gives an estimate
f(iw,) = lf(j
Backward Pass puted as
After predictions are generated with forward pass, the prediction error is com-
L=
(s(w))) /(wss))
The training seeks minimize over all training samples in standard manner using the stochas- tic gradient descent algorithm and its variants. Cross-validatio and testing can also be done in standard format.
An important papee-parameter of interest is number of degrees for complex polynomials Pi (ijw) and Pa(jw). This hyper-parameter can either be cross-validated or chosen based on practitioner's knowledge about the complexity of the specific phase retrieval problem. In our experiments, we found that complex phase retrieval problems may require larger degree polynomials, and calculating the value of these high degree polynomials over large frequency ranges (100THz 1500THz and above) may lead to the overflow problem. To overcome this problem, we propose a piece-wise implementation of BPNN.
Piece-wise Implementation of BPNN frequency band is partitioned into non-overlapping contiguous segments. The Blaschke method is then applied to each segment (see 2b). The BPNN is then trained to output the respec- tive Blaschke coefficients for each segment. During training, the predictions of phases all seg- ments are concatenated. Then the following training is the same non non-piecewise BPNN. The exact method of partitioning and degrees of corresponding Blaschke polynomials for each sequence are hhper-paranen parameters of choice. We have two example applications of piece-wise BPNNs to metamate- rial datasets in our numerical experiments.
In this approach, given a large frequency range, the whole
Prodicted Part
Truch Real Part
Magnitude
MSE
FCNN
BP
MSE
Blaschke Coefficients
Predicted Imaginery Part
Truth Imaginary Part
Gradient
(a) Illustration BPNNs
Predicted Real Part
0% 0'.a
Predicted Real Part P'
Magnitude
MSE
FCNN
0. a of
BP
Concet
MSE
0, P. a3
Precliched Imaginary Part
Truth Imaginary
Predicted Imaginary Part
Gradiant
(b) Illustration of Piece-wise Implementation of BPNN
Figure 2: Structure of BPNN (and piecewise BPNN): BPNN is an end-to-end model. Predictions are constructed by the combination of the Blaschke coefficients output of FCNN and values of input magnitude. In the piece-wise BPNN case, FCNN generates the sets of Blaschke coefficients for all segments (partitions) of the frequency range, and predictions are made for each segment separately and the results are concatenated.
5

---

Published as a conference paper at ICLR 2022
4 EXPERIMENTS
We conduct simulations on five phase retrieval datasets, two of which are synthetic, while the re- maining three are metamaterial datasets. A detailed listing of the datasets is shown in Table [ We endeavor test the BPNN on diverse, include both holomorphic and meromorphic complex functions. Additionally, the selected metama- terial datasets comprehensive, and well represent the discipline.
complete, datasets, and thus we choose datasets that
4.1 DATASETS
List of phase retrieval problems in Experiments section. (# Frequency) shows total number of frequency points considered for the problem. Please see relevant sections for detailed descrip- tions.
Table 1:
PHASE RETRIEVAL PROBLEM
TYPE
# FREQUENCY
Polynomial ODE (4.1.1
Synthetic
200
Lorentzian response function
A.7
Synthetic
1000
Metamaterial absorber (4. 1.3
Metamaterial
1000
Huygens' metasurface 4.
Metamaterial
2001
Membrane metasurface
Metamaterial
2001
4.1.1 SYNTHETIC I: POLYNOMIAL ODE
We first select points wo < W1 < W2 < ... < WN on the ju axis and fix these values. The first synthetic dataset is the solution ordinary differential equation (ODE) given by:
df de
p(2),
where p(z) is a complex polynomial of degree m = with its roots lying on the unit circle. We uniformly sample roots of p(=) on the unit circle, then solve the above ODE the ju-axis using the Euler method with the real and imaginary part of the imitial condition f(wn) uniformly selected in [-1, 1] for 1050 randomly generated initial values. The values (f(jw1), ... .fjwn)) are then calculated and their magnitudes and phases are used in this experiment, of which 50 are used for training and 1000 for testing.
4.1.2 SYNTHETIC LORENTZIAN RESPONSE FUNCTION
The second synthetic dataset is generated by considering a causal model of non non-magnetic ma- terial, with seequency-depenennn complex dielectric permittivity defined sum of Lorentzian oscillators. A simple Lorentzian model of the dielectric function is applicable to a wide range of res- onant material systems with response functions that must obey Kramers-Kroni relations. Using the wratifr-mmtrrx method, Markoš & Soukoulis (2008), the complex transmission coefficient t(w) can be computed directly from the dielectric function via closed form equations. (Please see appendix for more details). We generate dataset of complex transmission coefficients by uniformly sam- pling Lorentzian parameters for the frequency range 100-500 THz. We consider a fixed set of four Lorentzian oscillators, and generate 50 sample transmission spectra for training, and 2000 spectra for the test dataset.
4.1.3 METAMATERIAL I: METAMATERIAL ABSORBER
Metamaterials are structured materials consisting of periodic arrays of resonant elements that derive their spectral properties from their unit cell geometry /mmthh et al. (000)). With effective material properties that can consist of resonances in both the electric permittivity and magnetic permeability, and hence more unknown material parameters, metamaterials often represent a more complicated phase-retrieval problem than many conventional systems.

---

Published as a conference paper at ICLR 2022
The first metamaterial dataset considers metal-based absorber geometry with metallic ground plane backing a dielectric spacer layer, designed to operate in the millimeter wave (50-200 GHz) frequency range. This metamaterial absorber (MMA) dataset has 55,000 pairs of MMA geometry and complex frequency dependent reflection coefficient r(w) (I(w) is zero due to the metal backing). The data are obtained with commercial computational electromagnetic simulation software (CST Microwave Studios).
4.1.4 METAMATERIAL. II: HUYGENS' METASURFACE
Huygens' metasurfaces made of dielectric materials represent another class of metamaterials Decker ct al. (2013), with underlying properties that are different than their metallic counterparts, including for example the lack of Ohmic losses Liu al. (017). When individual designs combined into supercells, all-dielectric metasurfaces can exhibit very rich spectral phenomena. The Huygens' metasurface dataset consists of the complex reflection and transmission response derived from supercell of elliptical resonators adopted from the work Deng et al. (2021). The phase information can be inferred directly from the complex reflection or transmission. Out of a total of 1000 samples, we randomly sample 80 for training and reserve the rest for testing.
4.1.5 METAMATERIAL III: MEMBRANE METASURFACE
The membrane metasurfaces similar to the Huygens' metasurfaces. However, instead of having four stand-alone elliptical resonators in the supercell, it consists of a thin slab of SiC with regions of empty holes. The SiC slab filling the supercell has a thickness defined by h, and periodicity In the SiC slab, we placed four elliptical holes where the elliptical SiC resonators were placed. The four holes thus have the same geometrical parameters Tma- T'ms and 8. The fourteen geomet- rical parameters share the same range as the elliptical resonators dataset. Similarly, the input of these datasets contains fourteen geometry parameters, and the outputs are complex reflection and transmission with 2001 frequency points range from 100 THz to 500 THz. Out of a total of 7331 samples, we randomly sample 80 for training and reserve the rest for testing.
4.2 BENCHMARKING NEURAL NETWORKS
Comparison is done on each dataset with training datasets of increasing size. For each phase retrieval problem and each training dataset, we compare BPNN to a family of NNs of various sizes and structures. For each structure, we search its hyper-parameters of training to guarantee we have NNs close to optimal. And we record test error at each training epoch and report the best test error. We note that recording test error is uncommon but this guarantees that the reported performance is close to its limit.
We use NNs of various structures and all hidden layers of each of the NNs have the same width. The range of hidden layer width in search [32,64, 128,256) and the number of hidden layers in search is [1,2.3.4]. For hyper-parameters of training, we search dropout rate in the list of [O., 0.05, 0.1, 0.15, 0.2] and learning rate in the list of [le-4, 5e-4, le-3, 5e-3, le-2]. Each NN is trained with 6000 epochs after which training losses of all neural networks have converged. The training is performed with 3 random initialization and we report the best test eтror. Huygens' metasurface dataset is more complex than the other datasets, (64,128, 144,1222,66512 while keeping the ranges of the other searched parameters the same. Unlike the extensive searching for NNs, do not search pyram-eassmeters of BPNN and only for each phase retrieval problem. We also train BPNN with 3 random initialization on each dataset and report its best test error. We use ReLU as activation function for both NNs and BPNNs while Adam<Kingmm & Ba 2017) is used for training of all NNs and BPNNs.
the range of hidden layer size in search is increased to
BPNN
4.3 RESULTS
For each dataset, we compare the family of regular fully-connected neural networks, each with its best searched aoper-parameters to BPNN without Syyper-prrameter search. For three meta-material datasets, further include four baselines: (1)The Kramers Kramers-Kronig method (KK-method): a ventional phase retrieval method (Lucarini, 2005), (2)AAA-network algorithm: a established ratio- nal approximation technique, similar BPNN we used its functional form to approximate the phase

---

Published as a conference paper at ICLR 2022
Synthetic ODE Dataset
Synthetic Lorentzian Dataset
37,
64.3
-10
32.
64.4
256.2
54, 5
125.
25h.
32,
126,
250,
Minimum MSE on Test Data
-12
Minimum MSE Test Data
64.
125.
*h BPHN
-14
64,
125,4
-16
-30
18
128,
-20
-20
64.
128.
-6
8PNN
b4.2
125.
40
Training Data Size
Training Data Size
Figure 3: Performance comparison a number neural networks of different structures, each with its best syper-parameters, and BPNN on two synthetic phase retrieval problems with training set of various size.
function Naakatsukasa et al., 2018b), (3)MUSIC algorithm: data-driven function approximation method that estimates functions as sum of exponentials (Schmidt 1986a), and (4) Linear+BP: using linear regressor instead of neural networks in BPNN. see Appendix for more details about KK-method, AAA algorithm and MUSIC algorithm.) ) All y-axes are plotted with a dB scale in base 10 (10 (e) where is the MSE error) for visual clarity. For the legends in Fig. 3 we use entries of the form (A, B), where A refers to the width (i.e., number neurons) in each hidden layer of the neural network being evaluated, and B refers to the depth of the neural network (i.e., number of hidden layers).
Performance of Baselines We note that both the KK and MUSIC methods exhibit poor perfor- mance, because the KK method requires a large number of magnitude measurements to achieve ac- curate estimates, while the MUSIC method expects that the target function is sparse in some basis, which cannot be assumed for our problems. The Linear+BP model also performs substantially worse than the the BPNN. suggesting that the relationship between the phase magnitude an the Blaschke Product coefficients is non-linear. Although representing the phase function with Blaschke Product decreases the complexity of the problem, mapping from magnitude to the Blaschke coefficients still appears to be beyond the capability of a linear regressor. In the following paragraphs we describe results of each dataset in detail.
Synthetic Datasets We use BPNN of 4 roots with fully-connected neural network of 2 hidden layers, each of size 64 and no dropout. It obvious from figure that BPNN outperforms family of optimized neural networks by a large margin. This supports theoretical justification of using BPNN when the phase functions are precisely meromorphic functions. The outstanding performance of BPNN on the Lorentzian dataset implies its value on phase retrieval problems in metamaterials. To confirm this, we conduct more experiments on real metamaterial datasets.
MMA, ADM, Membrane For the MMA dataset, similar to synthetic datasets, also BPNN with 4 roots while using a fully-connected neural network with 2 hidden layers of size 64. No dropout nor weight-decay
added. Although without any regularization techniques for ral networks, BPNN still demonstrate superior performance to all the optimized neural networks. Given that ADM dataset and Membrane dataset are much more complex than the synthetic datasets and MMA dataset, we use a Piecewise BPNN with frequencies partitioned into 20 equal-length se- quences and we assign each sequence with roots. We note that this the most naive split for Piecewise BPNN and a more reasonable split can lead to improved performance. We find that the best errors regular neural networks are close to BPNN's best error. However, we also find that the median error of BPNN is much smaller than the median error of the best neural network (the neural network with smallest test MSE eтror across all architectures). This shows that the BPNN is making continued good predictions through the sacrifice of small amounts of poor predictions. This property can be valuable when the prediction problem at hand is the on-or-off type due to BPNN's higher probability of making good predictions.
8

---

Published as a conference paper at ICLR 2022
Performance
rndinnl
Minimum MISE Teat Dana
-30
20 Training Data Size
10
30
50
Figure 4: Performance comparison of family of neural networks, each with its best hyper- syper--arameters and BPNN (without hyper-paramen parameter search) on MMA dataset along with four other baselines: the KK method, linear Baggressor+Blaschke Product, MUSIC algorithm and the AAA method.
Performance Huygens' Metasurface Dalesel
Comparisan of Mediaan of Best NNs to BPNN
Red
Minimum MSE on Test Data
MUSC
-10
-17
-32
Median
-14
10
80
10
20
50
Training Data Size
Training Data Size
Performance on Membrane Metasurface Dataset
Comparison of Median of Best NNs to BPNN
Dost to
Minimum MSE Tezz Data
-20
Tumily ofN methoo
BPAN
Useere
-38
-17
-20
-34
-26
Median
-22
-70
-24
-22
20
Training
Training
Figure 5: On the left the performance comparison of family of neural networks, each with its best hyper-parameters and BPNN along with four other baseline methods. On the right is the comparison of median MSE loss of the best neural network in the large family and median MSE loss of BPNN. (On top are performance comparisons on Huygens' metasurface dataset; At bottom are performance comparisons on membrane metasurface dataset.)
5 CONCLUSION
We presented a physics-infused neural network referred to as Blaschke Product Neural Network (BPNN) for phase retrieval problem of holomorphic and meromorphic functions. Through extensive experiments on synthetic and metamaterial datasets, we have found that the data required for training neural networks is significantly less than expected when the underlying functions are holomorphic or meromorphic. By baking the physics into neural networks, even less training data is required and the performance is further improved.

---

Published as a conference paper at ICLR 2022
6 REPRODUCIBILITY STATEMENT
We upload four datasets. Due to restriction of maximum supplementary size of 100MB, we include three datasets in supplementary: two synthetic datasets and Huygens' metasurface dataset. Mem- brane metasurface can be accessed with this link (htts://figshare.comk) wtt.aaae..oom/////11// 96d6 847269a447c). Details about hyper-parameter search space and the process of searching can be found in the exper- iment section. Included in the supplementary we also have our implementation of the BPNN and neural networks by Pytorch with easy option of the
ACKNOWLEDGMENTS
This work was supported in part by the Office of Naval Research (ONR) under grant number N00014-21-1-2590 YD, OK, SR, JM, and WJP acknowledge funding from the Department of Energy under U.S. Department of Energy (DOR) (DESC0014372).
REFERENCES
Edwin J. Akutowicz. On the determination of the phase of a fourier integral, i. Transactions of the American Mathematical Society, 83(1):179-192, 1956. ISSN 00029947. URL htttphtt:///w.
"stor.org// wtt.oo////rrataeeeeeere9. 9992910
Jens Berg and Kaj Nyström. A unified deep artificial neural network approach to partial differential equations in complex geometries. Neurocomputing, 117:28-41, 2018.
Emmanuel J Candes. The restricted isometry property and its implications for compressed sensing. Comptes rendus mathematique, (46(9-00):589
2008.
N. M. Chalashkanov, S. Dodd, and J. C. Fothergill. Limitations of kramers kramers-kronig transform for calculation of the DC conductance magnitude from dielectric measurements. In 2012 Anmual Report Conference Electrical Insulation and Dielectric Phenomena. IEEE, October 2012. 2012.
doi: 10.1 10.1109/ccidp2220266777766
URL httt::://eol..org/11111/////e/
6378768
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. Neural ordinary dif- ferential equations. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, pp. 6572-6583, 2018.
Manuel Decker, Isabelle Staude, Matthias Falkner, Jason Domingue7, Dragomir N Neshev, Igal Brener, Thomas Pertsch, and Yuri S Kivshar. High-efficiency dielectric huygens' surfaces. Ad- vanced Optical Materials, 366)8113-820, 2015.
Yang Deng, Simiao Ren, Kebin Fan, Jordan M Malof, and Willie Padilla. Neural-adjoint method for the inverse design of all-dielectric metasurfaces. Optics Express, 2955)775667554, 2021.
MWMG Dissanayake and Nhan Phan-Thien. beural-network-basedd approximations for solving partial differential equations. communications in Numerical Methods Engineering, 10(3):195- 201, 1994.
A Driscoll, N. Hale, and L. N. Trefethen. Chebfun Guide. Pafnuty Publications, 2014. URL http: //ww: chebfun. org/docs/quide.
J. R. Fienup. Phase retrieval algorithms: comparison. Applied Optics, (1(1))22758 August 1982. doi: 10. 10.136/4a..22 0007588.
0007558)
Simon Foucart and Holger Rauhut. An invitation to compressive sensing. In A mathematical intro- duction compressive sensing, pp. 1-39. Springer, 2013.
Stephan Ramon Garcia, Javad Mashreghi, and William T. Ross. Finite blaschke products: survey, 2016.
Alexandre Goy, Kwabena Arthur, Shuai Li, and George Barbastathis. Low photon count phase retrieval using deep learning. Physical review letters, 121(24):243002, 2018.
10

---

Published as a conference paper at ICLR 2022
P Grosse and V Offermann. Analysis of reflectance data using the kramers- kronig relations. Applied Physics A, 52(2):138-144 1991.
Rola Houhou, Parijat Barman, Micheal Schmitt, Tobias Meyer, Juergen Popp. and Thomas Bocklitz. Deep learning phase retrieval tool for cars spectra. Optics Express, 28( (2(14221002 1)0:2000221,,4 2020.
Gauri Jagatap and Chinmay Hegde. Phase retrieval using untrained neural network priors. NeurlPS Workshop on Deep Inverse Blind Submission, 2019.
Frederick King. Hilbert transforms. Cambridge University Press, Cambridge, UK New York, 2009. ISBN 978-0-521-51720-1.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2017.
Tatiana Latychevskaia. Iterative phase retrieval in coherent diffractive imaging: practical issues. Applied Optics, 57(25)77187, August 2018. doi: 10.1364/2a.55.0071887 URL httpp:///doi
org/10.1364 ao 57 007007
Xinyu Liu, Kebin Fan, Πly V Shadrivov, and Willie Padilla. Experimental realization of terahertz all-dielectric metasurface absorber. Optics express, 25(1):191-211, 2017.
Valerio Lucarini. Ktamers-Krongs relations in optical materials research. Springer, Berlin New York, 2005. ISBN 3-540-23673-2.
Peter Markoš and Costas M. Soukoulis. Wave Propagation: From Electrons to Photonic Crystals and Left-Handed Materials. Princeton University Press, stu student edition edition, 2008, ISBN 9780691 130033. URL http:/ http://ssooo.orrgaaaae////.../.
Christopher Metzler, Phillip Schniter, Ashok Veeraraghavan, et al. prdeep: robust phase retrieval with a flexible deep network. In International Conference Machine Learning, pp. 3501-3510. PMLR, 2018.
Yuji Nakatsukasa, Olivier Sète, and Lloyd N Trefethen. The aaa algorithm for rational approxima- tion. SIAM Journal on Scientific Computing, 40(3):A 1494))4114144122 2018a.
Yuji Nakatsukasa, Olivier Sète, and Lloyd N. Trefethen. The aaa algorithm for rational approxima- Jan 2018b. ISSN 1095-7197. doi: 10.1137/16m1 106122. | htt.::ttt//d......t http://xx.doii.ooooo hww.://..ro..1.. 11.37 6M1 106122
tion. SIAM Journal on Scientific Computing, 4003):41144441153330
Keith A. Nugent. X-ray noninterferometric phase imaging: a unified picture. Journal of the Optical Society of America A, 24(2):536, February 2007. doi: 10. 111.33///saa.24.0000000
URL https:
//doi.org/10, 10. 3os.//............. 24. 000536
Walter Rudin. Real and Complex Analysis, 3rd Ed. McGraw-Hill, Inc., USA, 1987. ISBN 0070542341.
R. Schmidt. Multiple emitter location and signal parameter estimation. IEEE Transactions on Antennas and Propagation, 3463)277--200, 1986a. doi: 10.1109/TAP.198..111 191111998..11300...
Ralph Schmidt. Multiple emitter location and signal parameter estimation. IEEE transactions on antennas and propagation, 34(3):276-280, 1986b.
Yoav Shechtman, Yonina C. Eldar, Oren Cohen, Henry Nicholas Chapman, Jianwei Miao, and Mordechai Segev. Phase retrieval with application to optical imaging: contemporary overview.
IEEE Signal Processing Magazine, 32(3):87-109, May 2015. doi: 10.1 20.1109/mpp.201.233555 110111119/1
URL https: 1tt.:///.0.....11////.....0000 2014. 2352673
Ayan Sinha, Justin Lee, Shuai Li, and George Barbastathis. Lensless computational imaging through deep learning. Optica, 4(9): 1725, 2017.
David R Smith, Willic J Padilla, DC Vier, Syrus C Nemat-Nasser, and Seldon Schultz. Composite medium with simultaneously negative permeability and permittivity. Physical review letters, 84 (18):4184, 2000.
Henry Stark. Image recovery. Academic Press, Orlando, 1987. ISBN 9780323145978.
11

---

Published as a conference paper at ICLR 2022
Adrian Stern (ed.). Optical Compressive Imaging. CRC Press, November 2016. doi: 10.1201/ 9781315371474 URL /ttt:s///ebb..or//...222///////9
Kshitij Tayal, Chich-Hsin Lai, Vipin Kumar, and Ju Sun. Inverse problems, deep learning, and symmetry breaking. arXiv preprint 20Xx:2200300907,, 2020.
Yujia Xue, Shiyi Cheng, Yunzhe Li, and Lei Tian. Reliable deep-learning- phase imaging with uncertainty quantification. Optica, 6(5):61 8 8-829, 2019.
Run-Fa Zhang and Sudao Bilige. Bilinear neural network method to obtain the exact analytical solu- tions of nonlinear partial differential equations and its application to p-gbkp equation. Nonlinear Dynamics, (5(4))3041-3445 2019.
Yuhe Zhang, Mike Andreas Noack, Patrik Vagovic, Kamel Fezzaa, Francisco Garcia-Moreno, To- bias Ritschel, and Pablo Villanueva-Perez. Phasegan: a deep-learning phase-retrieval approach for unpaired datasets. Optics Express, 29(13):1959331190) 2021.
12

---

Published as a conference paper at ICLR 2022
A APPENDIX
A. OBSERVATIONS OF PHASE FUNCTION VALUES
Here we provide short explanation for the reason why the function values are only available on the imaginary line. In physics, electrical engineering and other fields, solutions to many linear PDEs with constant coefficients, and also many other measurements are done in the frequency domain. This in principle is achieved by multiplying real signal x(t) by sinusoidals cos(wt) and sin(wt), and integrating over time (Calculating Cosine and Sine Transforms) for various values w. If x(t) is a real physical signal, these cosine and sine transforms are real and imaginary parts of X (jw), the complex Fourier transform of x(t.. Thus (jw) gives the value of xx.) over the imaginary axis.
A.2 [B(jw) =1
We provide the details proving the norm of Blashke product is 1 if jw. To show that it has norm of one, we should show that all component (that are multiplied) would have a norm of one individually:
B(2) 1+
1- Il
ay -
aj +1 aj
0j 0j +1 a
B(z) (2) B2(z)* B=(=)
[B(z) = [B)( * [B2(z)) * BB(()]
B²(²)=
1+
BB(=)) II
a, +
Ba(=)
II 2 aj
1 1-
0j
0j +1
aj
1 + jw
[Bw =
1 jw
1 + ju
jw
1 + jul
1 jwl
(1 w² 20.5
(1 + w2)0.5
agj 1
la, +
[B2(ww)]
II_
0j 1 0j +1
|0, |a,
II
lag +
1
=1
[Bs(jw)]
II: j 2 + āj
aj
ljw 0j
Jiw + 0z1
liw 0j b,:
(assume a, - ag + bz
fiw + 0j b;j
(af + (b, 30)
II
(a] + (w - b, 23005
13

---

Published as a conference paper at ICLR 2022
A.3
OF PIECEWISE BPNN
In order provide analysis and some insights hyper-parameters, of Piecewise BPNN, conduct experiments on the two most complex datasets, Hyugens' metasurface dataset and membrane meta- surface dataset, have empirical results regarding the effect of hyper-parametee (e.g number of segments and number of roots per segment) on performance. We use training dataset of size 50 for both datasets and fix BPNN's neural networks have hidden layers and hidden size of 64 for each layer. From figure 6 we can see that Piecewise BPNN's performance is robust to hyper-par parameters:: wide range of different settings hyper-parametert can generate satisfactory results long the total number of roots is above the threshold necessary. Similarly, the Piecewise BPNN is also robust to overfitting caused by high number of roots: while the optimal total number roots appears the range of 60-80, overshooting total number of roots to 160 or even 200 doesn't 't cause sharp increase in test error. We believe this is a strong indication that the Piecewise BPNN learns the necessary complexity and thus zeros out the unnecessary degrees
Hyugens' Metasurtace Dataset
Membrane Metasurface Dataset
0.0300
6.0325
0,0288
0,0250
0.0291
0.0295
0,0316
0.0127
0.0113
0.0118
0.0106
0.030
0.0250
numbar sopmonts
Sistufor 20
0,0279
0.0273
0,0062
0.0260
0,0082
0.0198
0.0126
0.0122
0.0100
0.0112
0.029
0.0225
0.028
0.0200
0.00NS
0,0232
0.3745
0.0741
0,8079
0.027
0.0101
0.0009
0.0175
0.028
0,0253
0.025
0,0118
0.0125
0.0100
8
10
Fimmber
number routs
Figure 6: On the left is the heatmap of minimum test MSE (mean squared error) of Piecewise BPNN with different Syper-parameter configurations Hyugen's metasurface dataset. On the right the heatmap of minimum test MSE of Piecewise BPNN with different hyper-parameter configurations on membrane metasurface dataset. Each minimum MSE is calculated with runs of training from different initialization.
A.4 BASELINE MODEL 1: KRAMERS-KEE KROOIG
The Kramers-Kronig relations, implied by causality stable physics system, are often used to convert between imaginary parts and real parts of physical functions such as dielectric functions, susceptibility, and reflection coefficient. We adopt the notation from Grosse & Offermann [199I), since the metamaterials problems included in the main text focus the phase retrieval tasks spectral data, including reflectance. In general, the Kramers-Kronig relations for complex reflection coefficient given by:
1 P n
r" ("(w")
r(()
dw'
r'(w") de'
r"(w)
W'
which (w)) and (w) are real and complex part of reflection coefficient r(w) respectively.
In practice, (w) and (w(w) are often unknown. The reflectance R(w) |r(w)] the physical quantity that is directly measurable by intensity measurement, but the phase information is lost.
14

---

If Kramers- Kramers-Kronig relations are valid for in (w)), we can first measure R(w) from zero frequency to the highest measurable frequency. Then we can apply Kramers- Kronig relations on 0.5 In R(w) as real part of In r(w) infer the imaginary part ((w). In our implementation, we follow the above procedure to derive In r(w), and therefore can get the Kramers Haamers-Kronig reconstructed complex Published as a conference paper at ICLR 2022
Therefore, it is beneficial to derive phase ((w) from reflectance R(w). A general form of complex reflection coefficient can be written as r(w) exp We infer that:
In Inr(w) = 0.5 R(w) ip(w)
reflection coefficient F(w). Since the loss metric in the main text is the averaged mean square error of the real and imaginary parts of the reflection coefficient, (w) and f" (w) are further taken from ((w) to be compared with r' (ww) and r" (w).
The above procedure is built on the premise that there is causality for In r(w). but In r(w) is not a real physical function that is proven be governed by causality. Thus, cannot prove that Kramers- Kronig relations is valid on In r(w). Furthermore, the reflection spectrum in the metamaterials dataset are typically in finite frequency range that degrades the performance of the Kramers- Kronig relations. Nonetheless, we provided the Kramers Kronig relations as a baseline to perform phase retrieval.
A.5 BASELINE MODEL 2: AAA-NETWORE ALGORITHM DETAILS
Here we supply the details of the Adaptive Antoulas-Anderson (AAA) -network baseline we in- cluded in our main text. Following the notation of Nakatsukasa et al. (20T8a), it uses rational barycentric representation of rational apprixmation, which approximate an arbitrary function ra- tional function of type (m, m):
n22) r(:) d(z)
m
W Zj
Σ
m
W
where 2j, wy, fy support points, weights and value parameters of this approximation. It approx- imates complex function over a set complex support points using an iterative greedy selection and uses linear least-squares to solve for weights.
Although AAA also utilizes rational approximation, there are a couple of fundamental differences between AAA and our Blashke product approximation. (1) The functional form difference: AAA uses the barycentric representation, which although limited to type (m, m) rational, covers much larger functional space compared function on the unit circle according to the Helson-Sarason Theorem. (2) The fitting process: AAA uses greedy solution to pick the support point first and then solve the least square for weights and values. Our Blashke product fits aj simultancously and does not require iterative steps internally (although the actual fitting done using gradient-based methods, which requires iterative steps). This means the AAA is difficult to build into neural network and get gradient feedback from the fit, unlike our BPNN approach.
Blashke product, that is specialized to approximate the phase
With AAA, we can successfully approximate the phase function using three sets of barycentric parameters (also known as AAA parameters). However, this process requires knowledge of the phase function at least the ability to sample points the phase function. Therefore naive AAA alone is not able to be applied to our application of phase retrieval, which not only requires the approximation of phase function but also the extrapolation of the mapping between magnitude and the approximation parameters.
For programming implementation, the number of support points comparable to our BP (so that they have similar degrees freedom), first the AAA algorithm for all of our training sets (number of points ranging from 1 to 50) and get the training AAA parameters. Then we used a neural network to model the relation- ship between the measured magnitude to the AAA parameters for the training data. The network structure we used was 64 neurons with 2 hidden layers with Relu activation. After the network is
using the existing package Driscoll et al. 22014 Setting
We used this public Python version: https//gittuubcoomeefhhhaaa
15

---

Published as a conference paper at ICLR 2022
properly trained (reach convergence for 300 epochs), imput the testing spectra magnitude and get the inferred AAA parameters for the test spectra. Then we simply reconstructed the barycentric representation and compared the reconstructed phase function with the ground truth to get the MSE measure.
As graphs in the main text suggested, all of the AAA networks performed poorly compared to BP and to naive NN results. This is expected due to following reasons:
1. The functional form of barycentric rational functions lacks actual physics meaning: Encompassing all (m, m) rational functions, they are too large and not restrictive enough for the phase function. The lack of actual physical meaning in the rational approximation unlike our Blashke product (which takes into account the unit circle property of phase functions) is the most important reason we suspect this working
2. The two-stage process error: There are two stages in our AAA-network baseline, a AAA parameter fitting stage and network mapping learning stage. The error of each stage would compound and damage the overall performance
3. One-to-many issue: Unlike the Blashke product where the parameters form an ordered set, the parameters of the barycentric representation of AAA form an unordered set. The order is important for neural networks to learn meaningful mapping as it would not have one-to-many problems, which damages the training performance
A.6 BASELINE MODEL 3: MUSIC ALGORITHM DETAILS
Multiple Signal Classification (MUSIC) algorithm Schmidt ( (986b) is signal processing algorithm popular in frequency finding and direction finding where the signal is linear combination of multiple signal source of different frequencies and/or directions. MUSIC assumes that the signal vector consist of complex exponentials whose frequencies w are unknown.
Since MUSIC is originally applied phase retrieval problems and there is no widely single way to do phase retrieval, we would describe fully the process of our MUSIC phase retrieval algorithm here. First, we would assume that our signal x is consist of the sum of p complex exponentials with unknown frequency w and noise term n:
As
edwa
e S20p
A
2u2
5 [81, ..., 8p)"
jiM 1)w2
(M (1a 13m1
where A is the Vandermonde matrix of steering vectors and is the amplitude vector. In the assump- tion the actual number of sources is much lower than the observation, therefore i M. With this assumption, we would make another assumption that signal comes from the same distribution, namely their autocorrelation matrix (M by M) should be constant. Therefore, we would estimate this autocorrelation using the average of the sampled correlation matrix:
R XX"
Now with the estimate of autocorrelation, we estimate the frequency of this matrix using the eigenspace method, where the eigenvectors associated with a small eigenvalue would treated as the noise subspace and would be used to recover the signal frequency. Specifically, we decomposite the Rł into eigenvalues and eigenvectors, took the smallest (M-p) eigenvalues and their correspond- ing eigenvectors to form noise subspace UN. Since the eigenvectors from Hermitian matrix should be orthogonal to each other, meaning any pure signal from signal space would be orthogonal to all the vectors that span the noise subspace. Using a squared norm distance, for a signal e, the distance:
d = i "n UNUNEE=
Σ k=t+
"
16

---

Published as a conference paper at ICLR 2022
should be zero for all pure signals. Or, the inverse of d' should be the peaks in the space when we sweep through a range of possible w values. Now, we sweep the w space with a fine grid and
the peaks in the 1 de
would be our w values. Therefore, we can construct the A matrix by the top-p peaks in the curve.
The above is standard MUSIC algorithm to find the signal frequency (assuming the signal is sum of exponential), now with the frequency of the signal found, finding the actual signal x would be equivalent to finding the corresponding amplitude vector Therefore constructed our phase retrieval problem as an optimization problem:
x arg min (||æ) [x arg min (I]As] |<geel
Where the gggel is the ground truth magnitude or experimental measurements. For each test phase retrieval problem (each test magnitude spectra), we would need to run the optimization again using estimated matrix. We tried the Newton raphson method for this operation converge even if we set larger tolerance. Therefore we used the PyTorch framework (without any neural network) and utilized the parallel, GPU-based gradient optimization ability. We constructed the computational graph using the above equation and randomly initialized 1024 initial guesses and used Adam optimizer for 300 epoch till the loss converges. Afterward we would take the best performing single fit among the 1024 candidates
did not
final answer for this single magnitude. Due to time constraints (300 epoch with 1024 initialization for each our test cases), we did 500 test cases instead of the full dataset like other methods.
A.7 DETAILS ON METAMATERIALS DATASET
Metamaterial absorber: The input parameters include the four geometric dimensions of the top metal layer a, Wi, W2: 9, material properties of a top dielectric layer e, tan the conductivity of the metal metal, and thicknesses for all layers deap: dauh, dml, dmn, The output is the complex reflection coefficient r(w), with 1000 frequency points for each of the real and imaginary part of r(w) evaluated from 0 200 GHz.
The metasurface supercell geometry has a periodicity p. including four elliptical SiC resonators with consistent height h. Each elliptical resonator has major axis radius Tma. minor axis radius Tmi and a rotational angle A. The fourteen geometry parameters define inputs of computational simulations and yield complex reflection and transmission responses. This ADM dataset is particularly interested in the frequency-dependee reflection and transmission range from 100 500 THz.
Membrane Metasurface:
Synthetic II: Lorentzian Response Function The second synthetic dataset is generated by sidering a causal model of a non-magmen non-magnetic material, with a permittivity defined as sum of Lorentzian oscillators:
complex dielectric
(x(w) Eoo + Σ
C.p.p
* We.n,iw"
Go.
A simple Lorentzian model of the dielectric function is applicable wide range of resonant ma- terial systems with response functions that must obey Humeers-Kronj relations. Using the transfer- matrix method, Markoš & Soukoulis (2008), the complex transmission coefficient t(w) can be com- puted directly from the dielectric function via the closed form equation:
nww
nwl
((w) = cos
2
sin
c
2
2
where V / I/er, and e, are pre-detined physical constants. We generate dataset of complex transmission coefficients by uniformly sampling Lorentzian parameters for 6, optimized for the frequency range 100-500 THz. We consider fixed set of four Lorentzian oscillators, and generate 50 sample transmission spectra for training, and 2000 spectra for the test dataset.
17


Reviews:
[{"review_id": "pXg5Y3C1Fh6", "reviewer": "Reviewer_cH7K", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I liked a number of aspects of this paper. I thought the problem was well-motivated and the authors did a good job of explaining the underlying theory. Although I am not an expert on phase retrieval, the proposed method seems pretty elegant and clever. The experiments also seemed relatively thorough and the authors definitely tried their method on a number of different problems.\n\nHaving said this, there are a number of ways in which I think the exposition could be improved (it is also possible that I am missing some details).\n1. Reading the paper, my main question was what the fully-connected neural network brings to the table? If I understand correctly, it seems like the authors could have just regressed the coefficients of the rational approximation directly. I’d appreciate some clarity on why this couldn’t be done or why it wasn’t included as a baseline.\n2. I think I’m missing something about the training procedure. In particular, the equation for the backward pass features f(j\\omega_i), which presumably includes phase information. However, I thought the premise was that only |f(j\\omega_i)| was available. Can the authors clarify that this equation is really correct as written?\n3. I think the section on the piecewise BPNN could use some elaboration since it wasn’t totally clear what was happening here.\n4. I, personally, could have used a refresher about why the function values are only available on the imaginary line. \n5. While it is nice that the authors included code (which did help me understand exactly what was happening) I think the authors should clean it up a bit and add some comments before the conference.\n6. There were a number of places in the paper where the authors used terminology such as “it is easy to see that…” Typically I don’t find this style of exposition especially helpful. In particular, I think a little bit more detail on the rational approximation to B_k(j\\omega) could have been nice.\n\nDespite these issues, I do think this paper seems interesting and potentially useful. I also think the trick of using Blaschke products is something I hadn’t seen before and other researchers might find it interesting. So I do support publication. ", "summary_of_the_paper": "This paper introduces a new neural network architecture, called the BPNN, to recover phase information in physical problems where one only has access to the magnitude of a function on the complex line. This is a common problem in experimental physics and scattering where phase information can be lost, but is nonetheless important. To recover the phase in this setting the authors propose approximating the phase using Blaschke Products, which can approximate any continuous function on the imaginary line whose magnitude is one. Specifically, the authors train a fully-connected neural network to compute the coefficients of a rational approximation to the Blaschke product. They demonstrate the success of their method on a number of example problems and include code to reproduce their results.\n", "summary_of_the_review": "A clever paper using a relatively uncommon mathematical trick to make progress on a niche, but important problem. While the paper could be improved, I do think the ideas here could be interesting and useful to the community. The experiments seem relatively compelling and code is included with the submission.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "jw32-0_E7KP", "reviewer": "Reviewer_64Gx", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I am not aware of a publication that considers computing Blaschke products. Therefore, if this is a good idea compared to previous approaches, then the manuscript is valuable and novel. However, while the paper compares against different neural network structures and setups, it does not compare to standard approaches for rational approximation or phase retrieval that do not involve neural networks.  In particular, I find that there is a significant lack of motivation of why one would use a neural network to compute a Blaschke product, as opposed to using existing data-driven methods for approximating functions by rational functions or exponential sums. For example, for data-driven methods for approximating functions by rationals we have the AAA method (“The AAA algorithm for rational approximation” by Nakatsukasa, Sete, & Trefethen) as well as data-driven methods for approximating by exponential sums, see Prony’s method and MUSIC.  \n\nWhat is the main motivation for using piecewise BPNN? Is that so the Blaschke products can be kept low-degree?  A potential weakness is that the Blaschke products have a fixed degree as the structure of the neural network is fixed.  In data-driven methods for approximating with rational functions, it is found to be essential to adaptively select the degree of the rational function; otherwise, there are numerical stability issues and spurious poles. Can the authors comment on this? \n\nI am also confused by the numerical results. What is the y-axis in the MSE plots?  Is it the logarithm of the MSE in base 10 or the natural log? ", "summary_of_the_paper": "In complex analysis, the Blaschke product is an important family of bounded analytic functions in the open unit disk that is constructed to have roots at prescribed complex numbers. Finite Blaschke products are classically used for constructing rational functions with prescribed poles or zeros, which are then used in the approximation theory, differential equations, dynamical systems, and harmonic analysis.  Here, the authors use finite Blaschke products for phase retrieval of holomorphic and meromorphic functions. By looking at the argument of the approximant, the authors are able to estimate the phase at a set of frequencies. The authors use an L2 loss function to learn the parameters in the Blaschke product.", "summary_of_the_review": "I believe this manuscript has a promising idea of using neural networks to learn the parameters inside of a Blaschke product. Since this is fundamentally a technique for approximating functions of one complex dimension, the application to phase retrieval seems very natural. I think the authors have a half-baked idea, though. In my opinion, they fail to fully compare or motivate their choices. In particular, I wonder how the author's approach compares to other rational function approximation schemes that do not use neural networks such as the AAA algorithm. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "Vxq8hNF9M6k", "reviewer": "Reviewer_kQWk", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Strong Points**\n\n- Imposing priors on neural networks based on the physics of the problem is very important. Using Blaschke products to do this is an interesting idea.\n\n\n**Weak Points**\n\n- No comparison to existing methods in phase retrieval.\n- No ablation study.\n\n\n**Questions**\n\n- How does the size of the Blaschke product in Theorem 1 depend on epsilon? How do you choose the degree of the rational function?\n- Why is a neural network used to predict the coefficients for the rational function? How does this compare to simply fitting the ration function directly?\n- In Section 4.1.1, p(z) is “randomly selected.” What does this mean? What is the distribution over p(z)?\n\n\n**Additional Feedback**\n\n- Section 4.2: “Each NNs are trained with 6000…” This is still no guarantee of convergence. Other metrics, like gradient norm should be monitored for this.\n- There are many small grammatical errors. \n", "summary_of_the_paper": "**Summary**\n\nThe paper proposes a new method, BPNN, for phase retrieval, where the phase of some holomorphic or meromorphic function is predicted from measurements of its magnitude. The authors show that the unknown function can be expressed in terms of Blaschke products. Given some tolerance epsilon to the unknown function, the Blaschke product amounts to a rational function. A neural network is then used to predict the coefficients of this rational function and is trained using L2 loss. Finally, BPNN is compared against using a neural network to directly approximate the unknown function.", "summary_of_the_review": "**Recommendation**\n\nMy sense is that in its current form the paper should be rejected. Especially given the lack of comparison to existing methods. I am not an expert on phase retrieval, so I am happy for the other reviewers and rebuttal phase to change my mind.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "K8qGlVoENPI", "reviewer": "Reviewer_9QvZ", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I would suggest to conduct a few additional experiments to better analyze the model behavior. First, a comparison against other baselines than the neural network model should be performed if feasible, e.g. compressed sensing based methods or even the Kramers and Kronig formula. While the latter might not be a good choice due to the problems with the Riemann sum approximation for few datapoints, it could e.g. still provide a more complete picture.  It would further be interesting to see, how the methods behave for larger training/test set splits (e.g 50:50). Similarly, the influence of different possible configurations of BPNNs is underexplored. How do they behave with respect to the number of roots used? What is the influence of different window choices on the piece-wise BPNN implementation? Regarding the results section in general, the paragraph on the MMA experiments (page 8) is hard to follow and should be reworked. In general, including only the learning curves of a few representative NNs in the figures should be sufficient, the rest could e.g. be moved to an appendix.\n\nThe section regarding model architecture would profit from additional information as well. It is, for example, unclear what activation functions were used in the BPNNs and fully connected NNs or what kind of optimizer was used for training the models.\n\nFinally, there is a slight confusion regarding the number of frequencies used in the metamaterial absorber dataset. Table 1 reports 256, while the text states that 1000 frequency points are used. ", "summary_of_the_paper": "The authors introduce a neural network architecture based on Blaschke products for phase retrieval applications.", "summary_of_the_review": "In general, the idea to use neural networks to parametrize Blaschke polynomials for predicting phase functions is well motivated and shows promise based on the experimental results. The architecture and theory are presented in a concise fashion. However, the evaluation of the model is quite minimal and could profit from a more in-depth analysis in the form of additional experiments. Nevertheless, I tend towards accepting the work.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The Authors propose a neural-network based approach for the phase retrieval problem. Solving the phase retrieval problem is key for important application areas such as crystallography or radioastronomy.

After adding more baselines and other changes, 3 out of 4 reviewers recommended acceptance. Reviewer kQWk recommended rejection mostly based on the fact that the paper is quite narrow in scope.

Reviewer kQWk is right that the topic might not appeal to most of the ICLR community. It is worth noting that the main contribution of the paper is not about neural networks but rather about connecting phase retrieval with Blaschke products. As it stands, it seems that after making this connection, any non-linear approximator could do well.

Having said that, this is an important application area and the progress is welcomed. Hopefully, it will draw inspire more research in this area.

Currently, the key issue of the paper is that it is very challenging to understand for people without background in the phase retrieval area or complex analysis. To make this paper more valuable for the ICLR community, I would strongly encourage the Authors to devote at least a page to explanation of what is the phase retrieval problem, and the intuition behind the solution. Perhaps [1] could serve as an inspiration.

[1] Phase retrieval in crystallography and optics, R. P. Millane
Decision: Accept (Poster)