Statistical Learning and Inverse ProblemsAn Stochastic Gradient Approach
Anonymous Author(s)
AffiliationAddress
Abstract
Inverse problems are paramount in Science and Engineering.
In this paper, we consider the setup of Statistical Inverse Problem (SIP) and demonstrate how Stochastic
Gradient Descent (SGD) algorithms can be used in the linear SIP setting.
Weprovide consistency and finite sample bounds for the excess risk.
We also proposea modification for the SGD algorithm where we leverage machine learning methods to smooth the stochastic gradients and improve empirical performance.
Weexemplify the algorithm in a setting of great interest nowadays: the Functional
Linear Regression model.
In this case we consider a synthetic data example andexamples with a real data classification problem.
Introduction
Inverse Problems (IP) might be described as the search of an unknown parameter (that could be a function) that satisfies a given, known equation.
Considering the notation:
[Equation]
where f and y are elements of given Hilbert spaces, we would like to compute (or estimate)
f giventhe data y for some level of noise.
Typically, IPs are ill-posed in the sense that the solution doesnot depend continuously on the data.
There are several very important and impressive examples of
IPs in our daily lives.
Medical imaging has been using IPs for decades and it has shaped the area,as for instance, Computerized Tomography (CT) and Magnetic Resonance Imaging (MRI).
For anintroductory text, see Vogel [2002].
A vast literature of IPs is devoted to deterministic problems where the noise term is also a element ofa Hilbert space and usually assumed small in norm, which is not usually verified in practice.
In thiswork, we will take a different avenue, known as Statistical Inverse Problems (SIP).
This approach isa formalization of IPs within a probabilistic setting, where the uncertainty of all measurements areproperly considered.
Our focus in this work is to propose a direct and practical method for solving
SIP problems and, at the same time, provide theoretical guarantees for the excess risk performance ofthe algorithm we develop.
Our algorithm is based on a gradient descent framework, where stochasticgradients (or base learner that approximates the stochastic gradients) are used to estimate generalfunctional parameters.
The paper is organized as follows.
We finish this section contextualizing our paper in the broadliterature and stating our main contributions.
In Section 2, we formally introduce the learningproblem that we analyze.
In Section 3, we provide examples of practical problems that fits withinour formulation.
In Section 4 we provide our main results and algorithms.
Finally, in Section 5 weprovide numerical examples and a real data application for a Functional Linear Regression problem(FLR).
Due to space constraints, some of the figures, proofs and experiments were moved to thesupplementary material.
1.1 Contribution
We provide a novel numerical method to estimate functional parameters in SIP problems usingstochastic gradients.
More precisely, we extend the properties and flexibility of SGD and boostingalgorithms to a broader class of problems by bridging the gap between the IP and optimizationcommunities.
Whereas most of the IPs methods focus on regularization strategies to “invert" the operator A , wepropose a gradient descent type of algorithm to estimate the functional parameter directly.
Ouralgorithm works in the same spirit as Stochastic Gradient Descent algorithms with sample averaging.
While results of SGD are well understood in the context of regression problems in finite and infinitedimensions and SGD is well understood in deterministic IPs, SGD have not yet been consideredunder the SIP formulation.
We show that our procedure also ensures risk consistency in expectation and high probability underthe statistical setting.
Furthermore, we propose a modification in our algorithm to substitute thestochastic gradients by base learners similarly to boosting algorithms, Mason et al.
[1999], Friedman[2001].
This modification improve a common challenge faced by SIP problems: the discretizationprocedures of the operator A that arises in SIP.
1.2 Literature Review
Historically, SIP was first introduced in Sudakovand and Khalfin [1964] where IPs from Mathematical
Physics were recast into a statistical framework.
For a more structured introduction, we forward thereader to Kaipio and Somersalo [2004].
Several advances were made in the parametric approach to
SIP, where the unknown function is assumed to be completely described by an unknown parameterliving in a finite dimensional space, see for instance Evans and Stark [2002].
In our paper, however,we will consider the nonparametric framework as described in Cavalier [2008].
In this setting, we seethe IP as a search of an element of an infinite dimensional space.
When considering IPs (and SIP, in particular), there are several ways to regularize the problem in orderto deal with its ill-posedness.
For instance, one could consider roughness penalty or a functional basisas in Tenorio [2001].
Additionally, one could examine Tikhonov and spectral cut-off regularizationsas in Bissantz et al.
For many of those standard approaches, consistency under the SIP settingand rates of convergences were established.
See for instance Bissantz et al.
[2004], Bissantz and
Holzmann [2008].
A thoroughly discussion of stochastic gradient algorithms is outside the scope ofthis work
and we refer the reader to Zinkevich [2003], Nesterov et al.
[2018] and references therein.
Applications of Machine Learning (or Deep Learning, in particular) to solve IPs have been numerousin recent years, but, in our opinion, akin of the capabilities that these new techniques could bring tothis area of research.
Some attention have been given to imaging problems as in Jin et al.
[2017] and Ongie et al.
Under deterministic IP, the paper Li et al.
[2020] studies the regularization andconvergence rates of penalized neural networks when solving regression problems.
See also Adlerand Öktem [2017] and Bai et al.
Other important references regarding SGD for deterministic IP are Jin et al.
[2021], Tang et al.
[2019], Jin et al.
The main examples we bring in our paper is the class of Functional Linear Regression (FLR).
Thisproblem has drawn the attention of the statistical, econometric and computer science communities inthe past decade, see Cai and Hall [2006], Yao et al.
[2005], Hall and Horowitz [2007].
The usualmethodology applied to this problem is the well-known FDA.
For example, one could consider aprespecified basis functions to regularize the regression problem Goldsmith et al.
[2011] or one coulduse the Functional Principal Component (FPC) basis, Morris [2015].
More recently, methods inspiredin machine learning for standard linear regression problems were also extended to the FLR setting, seefor instance James et al.
[2009], Fan et al.
[2015] for methods that are suitable for high dimensionalcovariates or interpretable in the LASSO sense.
In this work we show how our modification to the
SGD algorithm can be seen as an averaging of boosting estimators and can also be used to estimate
FLR models in the high-dimensional setting.
Problem Formulation
We start by fixing a probability space (Ω , A , P ) and a vector space X of inputs.
We denote the randominput by X ∈ L 2 (Ω , A , P ) taking values in X and consider the space L 2 ( X ) of functions g : X −→ R dwith inner product ⟨ g 1 , g 2 ⟩ L 2 ( X ) =
E [ ⟨ g 1 ( X ) , g 2 ( X ) ⟩ ] and norm ∥ g ∥ 2 L 2 ( X ) =
E [ ∥ g ( X ) ∥ 2 ] < + ∞ ,where ⟨· , ·⟩ and ∥ · ∥ are the inner product and norm of R d .
Similarly, we define the space L 2 ( W ) of functions from W taking values in R k square-integrable withrespect to the measure space ( W , B , µ ) .
Finally, we consider an operator A : L 2 ( W ) −→ L 2 ( X ) .
This operator defines a direct problem and we assume that it is known.
Given f ∈ L 2 ( W ) , we usethe notation
A [ f ] ∈ L 2 ( X ) , i.e. A [ f ] is a square-integrable function
A [ f ] : X −→ R d .
We are interested in solving the statistical inverse problem related to A : jointly to observing samples of X taking values in X , we observe noisy samples of A [ f ◦ ]( X ) , for some fixed, unknown f ◦ ∈ L 2 ( W ) ,which we denote by Y :
[Equation]
where ϵ is a zero-mean random noise.
The problem we will pore over in this paper is the estimationof f ◦ based on this given sample.
Let ℓ : R d × R d → R + be a point-to-point loss function as, for example, the squared loss ℓ ( y , y ′ )
=12 ∥
y − y ′
∥ 2 .
We define the populational risk as:
[Equation]
and we would like to solve:
[Equation]
where F ⊂ L 2 ( W ) is a linear subspace with f ◦ ∈ F .
We will denote by ∂ 2 the partial derivativewith respect to the second argument.
Given a sample, we will study how to control the excess risk of a functional estimator ˆ f of f ◦ :
[Equation]
Instead of taking the standard route of solving the Empirical Risk Minimization problem and laterestablishing results for (3), in 4 we show how our Algorithms allows us to control directly to tackle(3) directly by constructing stochastic gradients directly for the populational risk.
Examples: motivation
Before we formalize our results, we first motivate the study of Eq.
(1) with a few of applications.
Each of those problems have a myriad of solutions on their own.
For more information on those IPs,see, for instance, Vogel [2002].
Deconvolution .
This type of inverse problems relate the values of Y and X through the following convolution equation:
[Equation]
where X = W = R d and the kernel k is known.
In this case, we define the operator A as:
[Equation]
Functional Linear Regression .
Consider the scalar, multivariate response model of functional linear regression: let X
=
L 2 ([0 , T ]) taking values in R d , W = [0 , T ] , µ is the Lebesgue measure in [0 , T ] and Y is giving by the following model
[Equation]
where f ∈ L 2 ( W ) is taking values in R .
Here we changed the notation from w to s in order to keep the classical notation from FLR.
In this case,
[Equation]
The model can be easily extended to deal with Y taking label values such as in a classificationproblem as we will show in the numerical studies.
Due to space constraints, we provide examples in the FLR setting.
In the supplementary material, wedemonstrate how the Algorithms presented in Section 4.2 also in deconvolution problems.
Theoretical Results and Algorithms
In this paper, we consider the following set of assumptions.
Assumption 4.1.
A : L 2 ( W ) −→ L 2 ( X ) is a linear, bounded operator;
ℓ is a convex and C 2 function in its second argument;
There exists θ 0 > 0 such that, for all f, g ∈ L 2 ( W ) ,
[Equation]
E [ ∂ 22 ℓ ( Y, A [ f ]( X ) + θA [ g ]( X ))
( A [ g ]( X )) 2 ] < ∞ ; 4. f ◦ ∈ arg min f ∈F R A ( f ) and R A ( f ◦ ) >
−∞ ;
sup f,f ′ ∈F
∥ f − f ′
∥
L 2 ( W ) =
D < ∞ .
Assumption 1 is our strongest one, since it imposes that our operator is linear and bounded.
Nevertheless, linear SIPs encompass a wide class of problems of practical and theoretical interest forengineering, statistics and computer science communities among others, a few of them presented in
Section 3.
Moreover, the nonlinear case could be similarly studied with more cumbersome notationand assumptions.
Assumption 2 is standard for gradient based algorithms and is commonly assumedin many learning problems.
Assumption 3 is a mild integrability condition of the loss function commonly satisfied in many practical situations.
For instance, in the squared loss case, this assumptionbecomes E [( A [ g ]( X )) 2 ] < ∞ , which is automatically satisfied since A [ g ] ∈ L 2 ( X ) .
Assumption 4 isneeded so the problem we analyze indeed have a solution.
Assumption 5 is stating that the diameterof the set F is finite.
One should notice that our set of assumptions does include the class of ill-posed (linear) inverseproblems since we do not need to assume that A is bijective.
If that were the case, it is known thatthen
A would have a bounded inverse, and then, the IP would not be ill-posed.
In the next sections we provide our theoretical results.
Instead of following the common approach ofminimizing the Empirical Risk Minimization problem, we show how to compute stochastic gradientsin order to control directly for the excess risk (3) both in expectation and in probability.
4.1 Preliminaries
Our first result allows us to compute the gradient of the populational risk at a given functional parameter f .
Before we present it, note that, by linearity, A : L 2 ( W ) → L 2 ( X ) is differentiable and, for every f, g ∈ L 2 ( W ) , we have that the directional derivative of A [ f ] in the direction g is given by
[Equation]
Note that the directional derivative does not dependent on the point f that we are evaluating the gradient.
Let A ∗ denote the adjoint operator of A defined as the linear and bounded operator A ∗ : L 2 ( X ) −→ L 2 ( W ) such that
[Equation]
The following lemma holds true:
Lemma 4.2.
Under 1, 2 and 3 of Assumption 4.1 we have that
[Equation]
where ϕ ( x ) =
E [ ∂ 2 ℓ ( Y , A
[ f ]( x )) | X = x ] .
Proof.
Firstly, define, for fixed ( X , Y ) ,
[Equation]
Then, we get the directional derivative of the risk function in direction g by applying the Taylorformula for ψ as a function of δ around δ = 0 :
[Equation]
where θ comes from the Taylor formula and it is between − θ
and θ 0 , for some fixed θ 0 > 0 .
Hence,by Assumption 3, we find
[Equation]
By the definition of ϕ and by conditioning in X , we find
[Equation]
Finally, we get that the descent direction ∇R A ( f ) is given by A ∗ [ ϕ ] ∈
L 2 ( W ) .
By the linearity and boundedness of A ∗ , we find the following very useful representation for thegradient of R A :
Lemma 4.3.
Consider the case d = 1 for simplicity of notation.
For each w ∈ W fixed, there exists a kernel Φ( · ; w ) : X −→ R such that
[Equation]
Proof.
First, notice that φ w ( g ) =
A ∗ [ g ]( w ) is an element of the dual of L 2 ( X ) .
Hence, by the Riesz Representation Theorem, there exists a kernel Φ( · ; w ) :
X −→ R such that, for all g ∈ L 2 ( X ) ,
[Equation]
as desired.
Corollary 4.4.
If ℓ is the squared loss function ℓ ( y , y ′ )
= 12 ∥ y − y ′
∥ 2 , then the gradient of the riskfunction with respect to f is given by [Equation] w [Equation]
f ]
[Equation]
X
[Equation]
− Y ( X )]
[Equation]
where Y ( x ) =
E [ Y | X = x ] .
Because of the results above, it is possible to construct an unbiased estimator for the gradient for therisk function for any f .
In fact, considering the case d = 1 , for a given sample ( x , y ) and a fixedfunction f , we define, for any w ∈ W ,
[Equation]
[Equation]
Therefore, conditioning in X , we find
[Equation]
The main benefit is that with only one observation we are able to compute an unbiased estimator forthe gradient of the risk function under the true distribution.
4.2 Proposed Algorithms
Inspired by Lemma 4.2, we propose the following SGD algorithm for SIP problems that we called
SGD-SIP: given an initial guess f 0 , for each step i , we compute, following Eq.
(4), an unbiasedestimator u i for the gradient of the loss function.
Next, we update an accumulated functionalparameter by taking a stochastic gradient step in the direction of u i with step size α i .
In the laststep, we average all the accumulated gradient steps in the same spirit as Polyak and Juditsky [1992].
The choice of the step size needs to satisfy two criteria: (cid:80) ni
=
α i sublinear in n , and nα n → ∞ asn → + ∞ .
We formally justify those desired properties in Theorem 4.5.
input : sample { x i , y i } ni =1 , operator A , initial guess foutput :
ˆ f n ˆ g 0 = f 0 ; for 1 ≤ i ≤ n do
Compute u
i ( w ) =
Φ( x i ; w )( A [ˆ g i − 1 ]( x i ) − y i ) ; ˆ g i
= ˆ g i − 1 − α i u
i ; end
Set
ˆ f n = 1 n (cid:80) ni
=
ˆ g i ;
Algorithm 1 uses only one sample at a time in order to estimate the gradient of the true risk function.
In order to preserve this property, we make the number of iterations equal to the sample size; it cannotbe larger.
This connects with the stopping rules in iterative algorithms in IP.
Algorithm 1 has a limitation common to many approaches to Inverse Problems: one cannot hope tocompute u i for every possible w i and some discretization of the operator A is needed, see Kaipioand Somersalo [2007].
Since the SGD-SIP algorithm only computes the stochastic gradient in thepoints of discretization, it risks overfitting the data and provides non-smooth estimators.
Next, wemotivate Algorithm 2 in order to overcome the discretization problem by leveraging machine learningmethods.
Consider that the space W was discretized in a grid of size n w .
In order to fully estimate the functionˆ f n ( w ) for every w ∈ W , we consider a hypothesis class H and, in each step, we fit a function ˆ h ⋆i onthe stochastic gradient u i in the discretized grid of W .
Note that in this case, F will be given by thelinear span of the class H .
Each of these functions h ⋆i can be seen as a base-learner in the same spiritof Boosting estimators, widely used in standard regression problem in the context of SIP Mason et
al.[1999], Friedman [2001].
Next we present our algorithm ML-SGD.
Algorithm 2: for 1 ≤ i ≤ n do for 1 ≤ j ≤ n w do Compute end end
Set
The goal of ML-SGD is twofold.
First, it allows us to interpolate the function
h
⋆j to points wnot used in the discretization grid.
Second, the ML procedure smooths the noise in each gradientstep calculation leading to smoother approximations that helps avoiding over-fitting.
We show in
Section 5 and in the supplementary material the benefits of such an approximation when estimatingthe functional parameter f ◦ in both simulated and empirical examples.
3 Main
Result
Our main result is a finite sample bound for the expected excess risk of Algorithm 1.
The result alsoextends to Algorithm 2 in the case where the base learner are also unbiased estimators.
Theorem 4.5.
Under Assumption 4.1 and if the kernel Φ satisfies C = sup x ∈ X ∥ Φ( x , · ) ∥ 2 < + ∞ 2 ,we have the following performance guarantee for Algorithm 1:
[Equation]
where M ( A, F ) =
C ( E [ | Y | 2 ] + ∥ A ∥ 2 D 2 ) < ∞ .
The proof of the theorem is provided in the supplementary material.
Theorem 4.5 implies that ifwe pick the decreasing sequence {
α i } ni
=1 so that nα n → ∞ ( α n cannot decrease too fast) but fastenough so that 1 n (cid:80) ni
=
α i → 0 , then we get the convergence result.
For instance, one could takeα i =
η/ √ i for some fixed number η normally taken to be in (0 , 1) .
In this case, the excess riskdecreases in expectation with rate O (1 / √ n ) .
Theorem 4.5 also implies that the excess risk converges to zero in probability.
For α i =
η/ √
i it is straightforward to check that
[Equation]
Finite sample bounds with high probability can also be provided under stronger assumptions aboutthe stochastic gradients.
See for instance Nemirovski et al.
Functional Linear Regression: numerical results
In this section, we provide two applications of the Functional Linear Regression problem.
We firstdemonstrate the performance of both algorithms in simulated data and next we provide an examplefor generalized linear models, when predicting the presence or not of Multiple Sclerosis (MS) afterreceiving as input a tract profile of corpus callosum (CCA) obtained by Diffusion Tensor Imaging.
As we have seen in Section 3, the operator in the FLR case is given by
[Equation]
Remember that in this example we are denoting w by s .
Hence [Equation] where A ∗ : L 2 ( X ) −→ L 2 ( W ) is given by
[Equation]
Therefore, we have Φ( x ; s ) =
x ( s ) , and using the squared loss, we find, as in Eq.
[Equation]
[Figure]
5.1 Synthetic Data
[Figure] We will consider the simulation study presented in González-Manteiga and Martínez-Calvo [2011].
Specifically, we set W = [0 , 1] , f ◦ ( z ) = sin(4 πz ) , and X simulated accordingly a Brownian motionin [0 , 1] .
We also consider a noise-signal ratio of 0.2.
We generate 100 samples of X and Y withthe integral defining the operator A approximated by a finite sum of 1000 points in [0 , 1] .
For theobserved data used in the algorithm procedure, we consider a coarser grid where and each functionalsample is observed at only 100 equally-spaced times.
For the ML-SGD algorithm, we used smoothingsplines as base learners.
We compared the results with a Functional Penalized Linear Regression(FPLR) with B-splines and cross-validation to select the number of basis expansion.
In order to fit the PFLR mode, we used the package refund Goldsmith et al.
[2021] available in R .
In the supplementarymaterial we can see the estimated observation
A [ ˆ f n ]( x i ) against the true values
A [ f ◦ ]( x i ) usingboth the SGD-SIP and the ML-SGD algorithm essentially recovers the true function, with a noisierfit for the SGD-SIP.
The benchmark algorithm also recovers the true function, with a smaller mse. Additional details of the simulation as well as error bars are provided in the supplementary material.
5.2 Real Data Application
Next we consider a classification problem in the FLR setting.
The data set contains 3000 examples of bitcoin wallets (address) and the respective cumulative credit in each wallet. Each address contain 501 equally spaced observations from April 2011 and April 2017.
For each address, we also have a category describing the category of the address.
In Table 5.2 we present a summary of the data.
We refer the reader to the supplementary material for more information about the data set used that we make available online.
Here we use the cumulative credit curve at each point in time as the explanatory variables X ∈ X
= L 2 ([0 , 1]) and Y ∈ {− 1 , 1 } as the predicted outcome for the indicator variable that the category is darknet (addresses associated with ilegal activities).
We propose the following model with a the log-likelihood of the negative binomial as the loss function: [Equation] [Table]
In Figure 1 we have the cumulative credit for 3 legal and 3 illegal accounts.
We compare the SGD-SIPand ML-SGD algorithm with PFLR methods.
For the ML-SGD algorithm, we use two types of base [Figure] learner, regression trees and cubic splines.
The step sizes are taken to be equal of the form O (1 / √ i ) ,where i = 1 , · · · , n is the current step of the algorithm and n is the total number of steps/sample.
Forthe PFLR algorithm, we use cubic splines with different degrees of freedom and quadratic penaltyterm.
We highlight that those choices of splines and penalty term are widely used in the literature.
In Table 5.2 we provide 3-fold cross validation for the accuracy and kappa metrics.
The SGD-SIP Algorithm achieved the best average performance in terms of accuracy.
The same performance isachieved by the Functional PLR with cubic splines with number of knots equal to 30 and penalizationfor the derivative of the estimate.
The ML-SGD algorithm with smooth splines with 30 degrees offreedom also achieved similar performance with a smoother estimator.
In Figure 2 we show the finalfunction fit for each of the methods listed in Table 5.2.
We refer the reader to the supplementarymaterial for results under different choices of step size, number of knots and base functions for the PFLR model, other metrics and confusion matrices.
In order to fit the PFLR mode, we used thepackage refund Goldsmith et al.
[2021] available in R .fold_ fold_ fold_ avg_accuracy ML-SGD-spline(k = 30) 0. ML-SGD-spline(k = 10) 0. ML-SGD-tree(depth = 20) 0.
SGD
[Table]
Conclusion
In this work, we provided a novel numerical method to solve SIP based on stochastic gradientswith theoretical guarantees for the excess risk.
Moreover, we have shown how one can improvealgorithmic performance by estimating base-learners for each stochastic gradient in the same spiritas boosting algorithms.
Our “framework" can be applied in a variety of settings ranging fromdeconvolution problems to FLR problems and others.
We demonstrate the performance of our methodwith numerical studies and also with a real world application data and comparing with widely usedtechniques in the FLR setting.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions
and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] We did both in the problem formulation
and in the numerical studies.
(c) Did you discuss any potential negative societal impacts of your work?
[No] We do not believe
that the methods proposed here could cause a negative impact in the society of any kind and this discussion was not necessary.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] Please refer to
(b) Did you include complete proofs of all theoretical results?
[Yes] Some of the proofs are in the main text.
Due to space constraints, some of the proofs were moved to the supplementary
material.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental
results (either in the supplemental material or as a URL)?
[Yes] The codes and data used are
available online and in the supplementary material.
Please, refer to the supplemental material.
(b) Did you specify all the training details (e
g., data splits, hyperparameters, how they were chosen)?
[Yes] Please refer to Section 5 and the supplementary material.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments
multiple times)?
[Yes] In the main text we reported results with 3folds cross-validation.
In the
online supplementary material we report results with synthetic data with error bars for the seed
and additional validation.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs,
internal cluster, or cloud provider)?
[N/A] The algorithms are simple and all the methods run in
standard computers.
Cloud computer and GPUs were not used.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing
[Table]
[Table]
[Table] [Table] (c ) Did you include any new assets either in the supplemental material or as a URL?
[No] The
existing assets are already available online.
(d
) Did you discuss whether and how consent was obtained from people whose data you’re us-
ing/curating?
[No] The assets used are already available for public use.
(e
) Did you discuss whether the data you are using/curating contains personally identifiable infor-
mation or offensive content?
[No] The data is not identifiable or offensive.
Please, refer to the
description provided in Section
If you
used
crowdsourcing or conducted research with human subjects..
[Table]
full [Table] instructions
[Table]
screenshots
[Table]
[Table] [Table]
you
[Table]
any
[Table]
participant risks, with
[Table]
to
Institutional
Review Board
[Table]
approvals if applicable
N
[Table]
[Table]
Did you include the estimated hourly
wage
paid to participants and[-using all the samples available in all the iterations. We start withf 
 ( z ) = 
 in all-] the total amount spent on
participant
compensation?
[N
[Table]
[-To bound the second term, notice that 
-]
[-∥ 2 < + ∞ , we find 
-]