SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Remark 3.3. The required compactness property of the set of input signals  $K \subset C_0([0,T];\mathbb{R}^p)$  in Theorem 3.1 is implied by a suitable smoothness constraint; examples include uniform bounds on the Lipschitz norm, Hölder norms, uniform bounds on higher-order derivatives, or under the assumption that the input signals are uniformly bounded and band-limited. The proposed compactness assumption subsumes all of these smoothness constraints, resulting in a widely applicable universal approximation theorem.
Remark 3.4. In practice, neural ODEs such as (2.2) need to be discretized via suitable numerical schemes. As examples, CoRNN and UnICORNN were implemented in [27] and [28], respectively, with implicit-explicit time discretizations. Nevertheless, universality also applies for such discretizations as long as the time-step is small enough, as the underlying discretization is going to be a sufficiently accurate approximation of (2.2) and Theorem 3.1 can be used for showing universality of the discretized version of the multi-layer neural oscillator (2.2).
3.3 Outline of the Proof
In the following, we outline the proof of the universality Theorem 3.1, while postponing the technical details to the SM. For a given tolerance  $\epsilon$ , we will explicitly construct the weights and biases of the multi-layer neural oscillator (2.2) such that the underlying operator can be approximated within the given tolerance. This construction takes place in the following steps:
(Forced) Harmonic Oscillators compute a time-windowed sine transform. Recall that the forced harmonic oscillator (2.6) is the simplest example of a neural oscillator (2.1). The following lemma, proved by direct calculation in SM B.1, shows that this forced harmonic oscillator actually computes a time-windowed variant of the sine transform at the corresponding frequency:
Lemma 3.5. Assume that  $\omega \neq 0$ . Then the solution of (2.6) is given by
$$
y (t) = \frac {1}{\omega} \int_ {0} ^ {t} u (t - \tau) \sin (\omega \tau) d \tau . \tag {3.1}
$$
Given the last result, for a function  $u$ , we define its time-windowed sine transform as follows,
$$
\mathcal {L} _ {t} u (\omega) := \int_ {0} ^ {t} u (t - \tau) \sin (\omega \tau) d \tau . \tag {3.2}
$$
Figure 1: Illustration of the universal 3-layer neural oscillator architecture constructed in the proof of Theorem 3.1.
Lemma 3.5 shows that a forced harmonic oscillator computes (3.2) up to a constant.
Approximation of causal operators from finite realizations of time-windowed sine transforms. The following novel result, termed the fundamental Lemma, shows that the time-windowed sine transform (3.2) composed with a suitable nonlinear function can approximate causal operators  $\Phi$  to desired accuracy; as a consequence, one can conclude that forced harmonic oscillators combined with a nonlinear read-out defines a universal architecture in the sense of Theorem 3.1.
Lemma 3.6 (Fundamental Lemma). Let  $\Phi : K \subset C_0([0, T]; \mathbb{R}^p) \to C_0([0, T]; \mathbb{R}^q)$  be a causal and continuous operator, with  $K \subset C_0([0, T]; \mathbb{R}^p)$  compact. Then for any  $\epsilon > 0$ , there exists  $N \in \mathbb{N}$ , frequencies  $\omega_1, \ldots, \omega_N$  and a continuous mapping  $\Psi : \mathbb{R}^{p \times N} \times [0, T^2/4] \to \mathbb{R}^q$ , such that
$$
\left| \Phi (u) (t) - \Psi \left(\mathcal {L} _ {t} u \left(\omega_ {1}\right), \dots , \mathcal {L} _ {t} u \left(\omega_ {N}\right); t ^ {2} / 4\right) \right| \leq \epsilon ,
$$
for all  $u \in K$  and  $t \in [0, T]$ .
The function  $\Psi$  defined in Lemma 3.6 can be interpreted as a finite-dimensional encoding of the operator  $\Phi$ . The main insight of the fundamental Lemma 3.6 is that the (infinite-dimensional) operator  $\Phi$  can effectively be replaced by a (finite-dimensional) function  $\Psi$ , making the approximation by neural oscillators more tractable.
The proof of the fundamental Lemma 3.6, detailed in SM B.2, is based on first showing that any continuous function can be reconstructed to desired accuracy, in terms of realizations of its time-windowed sine transform (3.2) at finitely many frequencies  $\omega_{1},\ldots ,\omega_{N}$  (see SM Lemma B.1). Then, we leverage the continuity of the underlying operator  $\Phi$  to approximate it with a finite-dimensional function  $\Psi$ , which takes the time-windowed sine transforms as its arguments.
Given these two results, we can discern a clear strategy to prove the universality Theorem 3.1. First, we will show that a general nonlinear form of the neural oscillator (2.2) can also compute the time-windowed sine transform at arbitrary frequencies. Then, these outputs need to be processed in order to apply the fundamental Lemma 3.6 and approximate the underlying operator  $\Phi$ . To this end, we will also approximate the function  $\Psi$  (mapping finite-dimensional inputs to finite-dimensional outputs) by oscillatory layers. The concrete steps in this strategy are outlined below.
Nonlinear Oscillators approximate the time-windowed sine transform. The building block of multi-layer neural oscillators (2.2) is the nonlinear oscillator of the form,
$$
\ddot {y} = \sigma (w \odot y + V u + b). \tag {3.3}
$$
In the following Lemma (proved in SM B.3), we show that even for a nonlinear activation function  $\sigma$  such as tanh or sin, the nonlinear oscillator (3.3) can approximate the time-windowed sine transform.
Lemma 3.7. Fix  $\omega \neq 0$ . Assume that  $\sigma(0) = 0$ ,  $\sigma'(0) = 1$ . For any  $\epsilon > 0$ , there exist  $w, V, b, A \in \mathbb{R}$  such that the nonlinear oscillator (3.3), initialized at  $y(0) = \dot{y}(0) = 0$ , has output
$$
\left| A y (t) - \mathcal {L} _ {t} u (\omega) \right| \leq \epsilon , \quad \forall u \in K, t \in [ 0, T ],
$$
with  $\mathcal{L}_t u(\omega)$  being the time-windowed sine transform (3.2).
Coupled Nonlinear Oscillators approximate time-delays. The next step in the proof is to show that coupled oscillators can approximate time-delays in the continuous input signal. This fact will be of crucial importance in subsequent arguments. We have the following Lemma (proved in SM B.4),
Lemma 3.8. Let  $K \subset C_0([0, T]; \mathbb{R}^p)$  be a compact subset. For every  $\epsilon > 0$ , and  $\Delta t \geq 0$ , there exist  $m \in \mathbb{N}$ ,  $w \in \mathbb{R}^m$ ,  $V \in \mathbb{R}^{m \times p}$ ,  $b \in \mathbb{R}^m$  and  $A \in \mathbb{R}^{p \times m}$ , such that the oscillator (3.3), initialized at  $y(0) = \dot{y}(0) = 0$ , has output
$$
\sup  _ {t \in [ 0, T ]} | u (t - \Delta t) - A y (t) | \leq \epsilon , \quad \forall u \in K,
$$
where  $u(t)$  is extended to negative values  $t < 0$  by zero.
Two-layer neural oscillators approximate neural networks pointwise. As in the strategy outlined above, the final ingredient in our proof of the universality theorem 3.1 is to show that neural oscillators can approximate continuous functions, such as the  $\Psi$  in the fundamental lemma 3.6, to desired accuracy. To this end, we will first show that neural oscillators can approximate general neural networks (perceptrons) and then use the universality of neural networks in the class of continuous functions to prove the desired result. The main difficulty here is that, due to the underlying oscillatory dynamics, it is not clear whether the output of even a shallow neural network can be reproduced by neural oscillators. The following lemma, visually represented by the bottom half of Figure 1, guarantees this:
Lemma 3.9. Let  $K \subset C_0([0, T]; \mathbb{R}^p)$  be compact. For matrices  $\Sigma, \Lambda$  and bias  $\gamma$ , and any  $\epsilon > 0$ , there exists a two-layer  $(L = 2)$  oscillator (2.2), initialized at  $y^\ell(0) = \dot{y}^\ell(0) = 0$ ,  $\ell = 1, 2$ , such that
$$
\sup  _ {t \in [ 0, T ]} \left| \left[ A y ^ {2} (t) + c \right] - \Sigma \sigma (\Lambda u (t) + \gamma) \right| \leq \epsilon , \quad \forall u \in K.
$$
The proof, detailed in SM B.5, is constructive and the neural oscillator that we construct has two layers. The first layer just processes a nonlinear input function through a nonlinear oscillator and the second layer, approximates the second-derivative (in time) from time-delayed versions of the input signal that were constructed in Lemma 3.8.
Combining the ingredients to prove the universality theorem 3.1. The afore-constructed ingredients are combined in SM B.6 to prove the universality theorem. In this proof, we explicitly construct a three-layer neural oscillator (2.2) which approximates the underlying operator  $\Phi$ . The first layer follows the construction of Lemma 3.7, to approximate the time-windowed sine transform (3.2), for as many frequencies as are required in the fundamental Lemma 3.6. The second- and third-layers imitate the construction of Lemma 3.9 to approximate a neural network (perception), which in turn by the universal approximation of neural networks, approximates the function  $\Psi$  in Lemma 3.6 to desired accuracy. Putting the network together leads to a three-layer oscillator that approximates the continuous and casual operator  $\Phi$ . This construction is depicted in Figure 1.
Machine learning architectures, based on networks of coupled oscillators, for instance sequence models such as CoRNN [27] and UnICORNN [28], graph neural networks such as GraphCON [26] and increasingly, the so-called physical neural networks (PNNs) such as linear and nonlinear mechanical oscillators [35] and spintronic oscillators [24, 32], are being increasingly used. A priori, it is unclear why ML systems based on oscillators can provide competitive performance on a variety of learning benchmarks, e.g. [27, 28, 26, 35], rather than biasing their outputs towards oscillatory functions. In order to address these concerns about their expressivity, we have investigated the theoretical properties of machine learning systems based on oscillators. Our main aim was to answer a fundamental question: "are coupled oscillator based machine learning architectures universal?" In other words, can these architectures, in principle, approximate a large class of input-output maps to desired accuracy.
To answer this fundamental question, we introduced an abstract framework of neural oscillators (2.1) and its particular instantiation, the multi-layer neural oscillators (2.2). This abstract class of second-order neural ODEs encompasses both sequence models such as CoRNN and UnICORNN, as well as a very general and representative PNN, based on the so-called Frenkel-Kontorova model. The main contribution of this paper was to prove the universality theorem 3.1 on the ability of multi-layer neural oscillators (2.2) to approximate a large class of operators, namely causal and continuous maps between spaces of continuous functions, to desired accuracy. Despite the fact that the considered neural oscillators possess a very specific and constrained structure, not even encompassing general Hamiltonian systems, the approximated class of operators is nevertheless very general, including solution operators of general ordinary and even time-delay differential equations.
The crucial theoretical ingredient in our proof was the fundamental Lemma 3.6, which implies that linear oscillator dynamics combined with a pointwise nonlinear read-out suffices for universal operator approximation; our construction can correspondingly be thought of as a large number of linear processors, coupled with nonlinear readouts. This construction could have implications for other models such as structured state space models [9, 8, 10] which follow a similar paradigm, and the extension of our universality results to such models could be of great interest.
Our universality result has many interesting implications. To start with, we rigorously prove that an ML architecture based on coupled oscillators can approximate a very large class of operators. This provides theoretical support to many widely used sequence models and PNNs based on oscillators. Moreover, given the generality of our result, we hope that such a universality result can spur the design of innovative architectures based on oscillators, particularly in the realm of analog devices as ML inference systems or ML accelerators [35].
It is also instructive to lay out some of the limitations of the current article and point to avenues for future work. In this context, our setup currently only considers time-varying functions as inputs and outputs. Roughly speaking, these inputs and outputs have the structure of (infinite) sequences. However, a large class of learning tasks can be reconfigured to take sequential inputs and outputs. These include text (as evident from the tremendous success of large language models [22]), DNA sequences, images [16], timeseries and (offline) reinforcement learning [15]. Nevertheless, a next step would be to extend such universality results to inputs (and outputs) which have some spatial or relational structure, for instance by considering functions which have a spatial dependence or which are defined on graphs. On the other hand, the class of operators that we consider, i.e., casual and continuous, is not only natural in this setting but very general [7, 6].
Another limitation lies in the feed forward structure of the multi-layer neural oscillator (2.2). As mentioned before, most physical (and neurobiological) systems exhibit feedback loops between their constituents. However, this is not common in ML systems. In fact, we had to use a mass ordering in the Frenkel-Kontorova system of coupled pendula (2.8) in order to recast it in the form of the multi-layer neural oscillator (2.2). Such asymptotic ordering may not be possible for arbitrary physical neural networks. Exploring how such ordering mechanisms might arise in physical and biological systems in order to effectively give rise to a feed forward system could be very interesting. One possible mechanism for coupled oscillators that can lead to a hierarchical structure is that of synchronization [34, 31] and references therein. How such synchronization interacts with universality is a very interesting question and will serve as an avenue for future work.
Finally, universality is arguably necessary but far from sufficient to analyze the performance of any ML architecture. Other aspects such as trainability and generalization are equally important, and we do not address these issues here. We do mention that trainability of oscillatory systems would profit from the fact that oscillatory dynamics is (gradient) stable and this formed the basis of the proofs of mitigation of the exploding and vanishing gradient problem for CoRNN in [27] and UnICORNN in [28] as well as GraphCON in [26]. Extending these results to the general second-order neural ODE (2.2), for instance through an analysis of the associated adjoint system, is left for future work.
[21] Allan Pinkus. Approximation theory of the MLP model in neural networks. Acta numerica, 8(1):143-195, 1999.
[22] A. Radford, K. Narasimhan, T. Salimans, and I. Suktskever. Improving language understanding by generative pre-training. arXiv., 2018.
[23] Danilo Rezende and Shakir Mohamed. Variational inference with normalizing flows. In International conference on machine learning, pages 1530-1538. PMLR, 2015.
[24] M. Romero and et. al. Vowel recognition with four coupled spin-torque nano-oscillators. Nature, 588:230–234, 2018.
[25] T. Konstantin Rusch, Michael M Bronstein, and Siddhartha Mishra. A survey on oversmoothing in graph neural networks. arXiv:2303.10993, 2023.
[26] T. Konstantin Rusch, Ben Chamberlain, James Rowbottom, Siddhartha Mishra, and Michael Bronstein. Graph-coupled oscillator networks. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 18888-18909. PMLR, 2022.
[27] T. Konstantin Rusch and Siddhartha Mishra. Coupled oscillatory recurrent neural network (cornn): An accurate and (gradient) stable architecture for learning long time dependencies. In International Conference on Learning Representations, 2021.
[28] T. Konstantin Rusch and Siddhartha Mishra. Unicornn: A recurrent model for learning very long time dependencies. In Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 9168-9178. PMLR, 2021.
[29] K. M. Stiefel and G. B. Ermentrout. Neurons as oscillators. Journal of Neurophysiology, 116:2950-2960, 2016.
[30] S. Strogatz. *Nonlinear Dynamics and Chaos*. Westview, Boulder CO, 2015.
[31] S. H. Strogatz. Exploring complex networks. Nature, 410:268-276, 2001.
[32] J. Torrejon and et. al. Neuromorphic computing with nanoscale spintronic oscillators. Nature, 547:428-431, 2017.
[33] A. Velichko, M. Belyaev, and P. Boriskov. A model of an oscillatory neural network with multilevel neurons for pattern recognition and computing. *Electronics*, 8, 2019.
[34] A. T. Winfree. Biological rhythms and the behavior of populations of coupled oscillators. Journal of Theoretical Biology, 16:15-42, 1967.
[35] Logan G Wright, Tatsuhiro Onodera, Martin M Stein, Tianyu Wang, Darren T Schachter, Zoey Hu, and Peter L McMahon. Deep physical neural networks trained with backpropagation. Nature, 601(7894):549-555, 2022.
[36] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747, 2017.
A Another universality result for neural oscillators
The universal approximation Theorem 3.1 immediately implies another universal approximation results for neural oscillators, as explained next. We consider a continuous map  $F: \mathbb{R}^p \to \mathbb{R}^q$ ; our goal is to show that  $F$  can be approximated to given accuracy  $\epsilon$  by suitably defined neural oscillators. Fix a time interval  $[0, T]$  for (an arbitrary choice)  $T = 2$ . Let  $K_0 \subset \mathbb{R}^p$  be a compact set. Given  $\xi \in \mathbb{R}^p$ , we associate with it a function  $u_{\xi}(t) \in C_0([0, T]; \mathbb{R}^p)$ , by setting
$$
u _ {\xi} (t) := t \xi . \tag {A.1}
$$
Clearly, the set  $K \coloneqq \{u_{\xi} \mid \xi \in K_0\}$  is compact in  $C_0([0,T];\mathbb{R}^p)$ . Furthermore, we can define an operator  $\Phi : C_0([0,T];\mathbb{R}^p) \to C_0([0,T];\mathbb{R}^q)$ , by
$$
\Phi (u) (t) := \left\{ \begin{array}{l l} 0, & t \in [ 0, 1), \\ (t - 1) F (u (1)), & t \in [ 1, T ]. \end{array} \right. \tag {A.2}
$$
where  $F:\mathbb{R}^p\to \mathbb{R}^q$  is the given continuous function that we wish to approximate. One readily checks that  $\Phi$  defines a causal and continuous operator. Note, in particular, that
$$
\Phi (u _ {\xi}) (T) = (T - 1) F (u _ {\xi} (1)) = F (\xi),
$$
is just the evaluation of  $F$  at  $\xi$ , for any  $\xi \in K_0$ .
Since neural oscillators can uniformly approximate the operator  $\Phi$  for inputs  $u_{\xi} \in K$ , then as a consequence of Theorem 3.1 and (2.3), it follows that, for any  $\epsilon > 0$  there exists  $m \in \mathbb{N}$ , matrices  $W \in \mathbb{R}^{m \times m}$ ,  $V \in \mathbb{R}^{m \times p}$  and  $A \in \mathbb{R}^{q \times m}$ , and bias vectors  $b \in \mathbb{R}^m$ ,  $c \in \mathbb{R}^q$ , such that for any  $\xi \in K_0$ , the neural oscillator system,
$$
\left\{ \begin{array}{l} \ddot {y} _ {\xi} (t) = \sigma \left(W y _ {\xi} (t) + t V \xi + b\right), \\ y _ {\xi} (0) = \dot {y} _ {\xi} (0) = 0, \\ z _ {\xi} (t) = A y _ {\xi} (t) + c, \end{array} \right. \tag {A.3}
$$
$$
| z _ {\xi} (T) - F (\xi) | = | z _ {\xi} (T) - \Phi (u _ {\xi}) (T) | \leq \sup  _ {t \in [ 0, T ]} | z _ {\xi} (t) - \Phi (u _ {\xi}) (t) | \leq \epsilon ,
$$
uniformly for all  $\xi \in K_0$ . Hence, neural oscillators can be used to approximate an arbitrary continuous function  $F: \mathbb{R}^p \to \mathbb{R}^q$ , uniformly over compact sets. Thus, neural oscillators also provide universal function approximation.
Proof. We can rewrite  $y(t) = \frac{1}{\omega} \int_{0}^{t} u(\tau) \sin (\omega (t - \tau)) d\tau$ . By direct differentiation, one readily verifies that  $y(t)$  so defined, satisfies
$$
\dot {y} (t) = \int_ {0} ^ {t} u (\tau) \cos (\omega (t - \tau)) d \tau + [ u (\tau) \sin (\omega (t - \tau)) ] _ {\tau = t} = \int_ {0} ^ {t} u (\tau) \cos (\omega (t - \tau)) d \tau ,
$$
in account of the fact that  $\sin (0) = 0$ . Differentiating once more, we find that
$$
\begin{array}{l} \ddot {y} (t) = - \omega \int_ {0} ^ {t} u (\tau) \sin (\omega (t - \tau)) d \tau + [ u (\tau) \cos (\omega (t - \tau)) ] _ {\tau = t} \\ = - \omega^ {2} y (t) + u (t). \\ \end{array}
$$
Thus  $y(t)$  solves the ODE (2.6), with initial condition  $y(0) = \dot{y}(0) = 0$ .
B.2 Proof of Fundamental Lemma 3.6
Reconstruction of a continuous signal from its sine transform. Let  $[0,T] \subset \mathbb{R}$  be an interval. We recall that we define the windowed sine transform  $\mathcal{L}_t u(\omega)$  of a function  $u:[0,T] \to \mathbb{R}^p$ , by
$$
\mathcal {L} _ {t} u (\omega) = \int_ {0} ^ {t} u (t - \tau) \sin (\omega \tau) d \tau , \quad \omega \in \mathbb {R}.
$$
In the following, we fix a compact set  $K \subset C_0([0,T];\mathbb{R}^p)$ . Note that for any  $u \in K$ , we have  $u(0) = 0$ , and hence  $K$  can be identified with a subset of  $C((- \infty, T ]; \mathbb{R}^p)$ , consisting of functions with  $\operatorname{supp}(u) \subset [0, T]$ . We consider the reconstruction of continuous functions  $u \in K$ . We will show that  $u$  can be approximately reconstructed from knowledge of  $\mathcal{L}_t(\omega)$ . More precisely, we provide a detailed proof of the following result:
Lemma B.1. Let  $K \subset C((-\infty, T]; \mathbb{R}^p)$  be compact, such that  $\operatorname{supp}(u) \subset [0, T]$  for all  $u \in K$ . For any  $\epsilon, \Delta t > 0$ , there exists  $N \in \mathbb{N}$ , frequencies  $\omega_1, \ldots, \omega_N \in \mathbb{R} \setminus \{0\}$ , phase-shifts  $\vartheta_1, \ldots, \vartheta_N \in \mathbb{R}$  and weights  $\alpha_1, \ldots, \alpha_N \in \mathbb{R}$ , such that
$$
\sup  _ {\tau \in [ 0, \Delta t ]} \left| u (t - \tau) - \sum_ {j = 1} ^ {N} \alpha_ {j} \mathcal {L} _ {t} u (\omega_ {j}) \sin (\omega_ {j} \tau - \vartheta_ {j}) \right| \leq \epsilon ,
$$
for all  $u\in K$  and for all  $t\in [0,T]$
Proof. Step 0: (Equicontinuity) We recall the following fact from topology. If  $K \subset C((-\infty, T]; \mathbb{R}^p)$  is compact, then it is equicontinuous; i.e., there exists a continuous modulus of continuity  $\phi : [0, \infty) \to [0, \infty)$  with  $\phi(r) \to 0$  as  $r \to 0$ , such that
$$
| u (t - \tau) - u (t) | \leq \phi (\tau), \quad \forall \tau \geq 0, t \in [ 0, T ], \forall u \in K. \tag {B.1}
$$
Step 1: (Connection to Fourier transform) Fix  $t_0 \in [0, T]$  and  $u \in K$  for the moment. Define  $f(\tau) = u(t_0 - \tau)$ . Note that  $f \in C([0, \infty); \mathbb{R}^p)$ , and  $f$  has compact support  $\operatorname{supp}(f) \subset [0, T]$ . We also note that, by (B.1), we have
$$
| f (t + \tau) - f (t) | \leq \phi (\tau), \quad \forall \tau \geq 0, t \in [ 0, T ].
$$
We now consider the following odd extension of  $f$  to all of  $\mathbb{R}$ :
$$
F (\tau) := \left\{ \begin{array}{l l} f (\tau), & \text {f o r} \tau \geq 0, \\ - f (- \tau), & \text {f o r} \tau \leq 0. \end{array} \right.
$$
Since  $F$  is odd, the Fourier transform of  $F$  is given by
$$
\widehat {F} (\omega) := \int_ {- \infty} ^ {\infty} F (\tau) e ^ {- i \omega \tau} d \tau = i \int_ {- \infty} ^ {\infty} F (\tau) \sin (\omega \tau) d \tau = 2 i \int_ {0} ^ {T} f (\tau) \sin (\omega \tau) d \tau = 2 i \mathcal {L} _ {t _ {0}} u (\omega).
$$
Let  $\epsilon > 0$  be arbitrary. Our goal is to uniformly approximate  $F(\tau)$  on the interval  $[0, \Delta t]$ . The main complication here is that  $F$  lacks regularity (is discontinuous), and hence the inverse Fourier transform of  $\widehat{F}$  does not converge to  $F$  uniformly over this interval; instead, a more careful reconstruction based on mollification of  $F$  is needed. We provide the details below.