Continuously-Tempered PDMP samplers
Anonymous Author(s)
AffiliationAddress email
Abstract
New sampling algorithms based on simulating continuous-time stochastic processes called piece-wise deterministic Markov processes (PDMPs) have shownconsiderable promise.
However, these methods can struggle to sample from multimodal or heavy-tailed distributions.
We show how tempering ideas can improvethe mixing of PDMPs in such cases.
We introduce an extended distribution definedover the state of the posterior distribution and an inverse temperature, which interpolates between a tractable distribution when the inverse temperature is 0 andthe posterior when the inverse temperature is 1.
The marginal distribution of theinverse temperature is a mixture of a continuous distribution on [0 , 1) and a pointmass at 1: which means that we obtain samples when the inverse temperature is 1,and these are draws from the posterior, but sampling algorithms will also exploredistributions at lower temperatures which will improve mixing.
We show how
PDMPs, and particularly the Zig-Zag sampler, can be implemented to sample fromsuch an extended distribution.
The resulting algorithm is easy to implement andwe show empirically that it can outperform existing PDMP-based samplers onchallenging multimodal posteriors.
Introduction
Recently there has been considerable interest in developing sampling algorithms based on simulatingcontinuous time stochastic processes called a piecewise deterministic processes (PDMPs) Davis[1993].
These sampling algorithms are qualitatively similar to Hamiltonian Monte Carlo algorithms Neal [2011].
To simulate from some targst distribution π ( x ) , they work with an augmented statespace ( x , v ) , where the x component can be viewed as position, and the v component as a velocity.
The motivation for this is that these processes will encourage exploration of π ( x ) by simulatingcontinuous velocity paths in between random event times at which the velocity changes.
Examples ofsuch sampling algorithms include the Bouncy Particle Sampler Peters and de With [2012], BouchardCôté et al. [2018], the Zig-Zag algorithm Bierkens et al.
[2019], the Coordinate Sampler Wu and Robert [2020] and the Boomerang Sampler Bierkens et al.
See Fearnhead et al.
[2018] foran overview of these methods.
Importantly, it is simple to write down the event rate, and how thevelocity should change at each event, to ensure these PDMPs have π as their invariant distribution.
These depend on π only through the gradient of log π , and thus π only needs to be known up to aconstant of proportionality.
These PDMP samplers have a number of advantages, including non-reversible dynamics (which isknown to improve mixing relative to reversible processes Diaconis et al. Bierkens [2016]),and the ability to reduce computation-per-iteration by either leveraging sparsity structure in the model Bouchard-Côté et al. [2018], Sutton and Fearnhead [2021] or using only sub-samples of the data toapproximate the log-likelihood at each iteration (whilst still guaranteeing sampling from the target
Bierkens et al.
However, like other MCMC algorithms, particularly those that use gradient information, these PDMP samplers can struggle to mix for multi-modal target distributions, or forheavy-tailed targets Vasdekis and Roberts [2021].
One of the more successful techniques for enabling an MCMC algorithm to sample from challenging,e.g. multi-modal, target distributions is to use tempering.
There are various forms of tempering, buteach is based on defining either a discrete set or continuum of distributions that interpolate between adistribution that is simple to sample from (viewed as at high temperature) and the target distribution(at a low temperature).
The idea is that allowing mixing across this set will improve mixing, asmoving between modes will be easier for the distributions at higher temperatures.
Examples of suchalgorithms include parallel tempering Swendsen and Wang [1986], simulated tempering Marinariand Parisi [1992], and continuous tempering Graham and Storkey [2017].
In this paper we show how continuous tempering ideas can be used with PDMP samplers.
We havechosen continuous tempering, as opposed to the alternative tempering approaches, as it is the methodthat can most naturally benefit from the continuous-time nature of PDMP dynamics.
To the best ofour knowledge, this is the first attempt at using tempering ideas to improve PDMP samplers.
Thegeneral idea is to define a joint distribution on the state of the PDMP, z
= ( x , v ) , and the inversetemperature, β .
The target distribution of interest is the x -marginal of this joint distribution whenβ = 1 .
We define the joint distribution so that it has a point mass at β = 1 – thus simulating from itwill lead to a proportion of the resulting samples being from the target.
We then use a PDMP samplerto simulate from this joint distribution.
Constructing the appropriate dynamics of the PDMP sampleris non-trivial in this case as we have to deal with its behaviour as it transitions from the continuousdistribution on β ∈ [0 , 1) and the point mass at β = 1 .
We use recent ideas for PDMP samplers withdiscontinuities Chevallier et al. [2021, 2020] to solve this challenge.
While we express ideas based on and similar to Graham and Storkey [2017], a key difference isthe inclusion of a point-mass at β = 1 , this means we can obtain samples from π ( x ) rather thanhaving to resort to importance sampling to correct samples drawn at different temperatures.
It iseasy to introduce a point mass into the dynamics of the PDMP sampler due to its continuous samplepaths: we simulate paths for β ∈ [0 , 1) until the process hits β = 1 – we then simulate paths with βfixed to 1 for an exponential period of time before returning to β ∈ [0 , 1) .
By comparison, using apoint-mass within a Hamiltonian Monte Carlo sampler is difficult as the discretised sample pathswill not hit β = 1 with probability 1.
Yao et al.
[2020] consider an indirect approach to samplingwith a pointmass at β = 1 using a continuous link function.
Our approach is distinct and more direct,allowing one to exploit the unique advantages of PDMP samplers — such as the ability to performsubsampling without altering the ergodic distribution and application to sampling transdimensionaldistributions.
The benefits of introducing the point mass at β = 1 are numerically investigated, and practicalconsiderations related to choosing tuning parameters to encourage a desired proportion of time atthe target distribution are presented (Section 3.4).
We find that, analogously to standard methods indiscrete-time, the tempered counterparts of PDMP samplers outperform vanilla PDMP on challengingsampling problems.
The Zig-Zag sampler
For the purposes of brevity and ease of exposition, we focus specifically on a continuous-temperedversion of the Zig-Zag sampler.
However, we stress that the underlying ideas can easily be applied toany PDMP sampler, for example, the Bouncy Particle Sampler Bouchard-Côté et al.
[2018] or the
Boomerang Sampler Bierkens et al.
[2020]; see the comments at the end of Section 3.2.
Consider the problem of sampling from a target density defined for x ∈ X := R d by
[Equation]
where U : X → R is a continuous differentiable function referred to as the potential and Z isthe, potentially unknown, normalising constant Z = (cid:82) X exp( − U ( x ))
d x .
We will denote the unnormalised target density by q , so π ( x ) =
q ( x ) /Z .
The Zig-Zag process is a continuous-time piece-wise deterministic Markov process, which can bedefined so as to have π ( x ) as its invariant distribution.
The process is defined on an extendedstate-space that can be viewed as consisting of a position, x and a velocity component, v .
For the
Zig-Zag process, the velocity is restricted to be ± 1 in each axis direction.
Thus the extended space is
E = R d × {− 1 , 1 } d .
We write z = ( x , v ) for z ∈ E from here on.
We will use subscripts to denotetime, and superscripts to denote components.
So z t will be the state at time t , while x it will be the i thcomponent of the position at time t .
For an event at time t we use the notation z t − to be the state immediately before the event, andz t the state immediately after it.
The dynamics of the Zig-Zag process are deterministic betweena set of random event times.
At each event time the direction of one component of the velocity isswitched.
The deterministic dynamics are specified by a constant velocity model.
So, if there are noevents between times t and t + h , for h > 0 , the change of state is given by z t
+ h = ( x t + h , v t + h ) =(
x t + h v t , v t ) .
The events occur with a rate that depends on the current state.
For the Zig-Zag process we haved types of event, each of which results in the flipping of one of the d components of the velocityprocess.
To ensure that we have π ( x ) as the x -marginal of the process’s invariant distribution, theserates are defined to be, for i = 1 , . . .
, d , [Equation] with the transition at the corresponding event being that v it = − v it − , and all other elements of thestate are unchanged.
Pseudo-code for simulating the Zig-Zag process is given in Algorithm 1.
When we simulate such aprocess, the output of the algorithm is the set of event times, positions and velocities after the eventtimes.
This set is known as the PDMP skeleton and with the deterministic dynamics of the process itdefines the continuous time path of the process.
We can use such a simulated path to give us drawsfrom π ( x ) , by discarding some suitably chosen initial path time as burn-in, and then evaluating thex component of the process at a set of evenly spaced discrete time-points (see e.g. Fearnhead et al.[2018] for alternative approaches that use the continuous-time paths).
Algorithm
▷ Update position
▷ Update velocity
Continuously-tempering with a point mass3.1 Continuous-tempering
Continuous-tempering is an approach to improve mixing of MCMC and related sampling algorithms.
It introduces a second distribution π 0 , called the base distribution, which is assumed to have a densityπ 0 ( x ) = q 0 ( x ) Z 0 .
The idea is that this density will be simple to simulate from.
For any β ∈ [0 , 1]it is now possible to define a distribution which interpolates between π and π 0 in that its densityis π ( x ) β π 0 ( x ) (1 − β ) .
Continuous-tempering then defines a joint distribution on β and x .
Thisdistribution has un-normalised density q ( x , β ) , on R d × [0 , 1] , defined to beq ( x , β ) =
q 0 ( x ) 1 − β q ( x ) β .
In practice tempering methods often introduce a prior, p ( β ) on β ∈ [0 , 1] , and sample from thedistribution proportional to p ( β ) q
( x , β ) .
The idea of introducing the prior is that it can be tuned togive a marginal distribution on β that has reasonable mass across all of the interval [0 , 1] .
Without aprior, the distribution q is likely to put almost all mass on β close to 0 or on β close to 1.
By construction, conditional on satisfying β = 1 , samples drawn from q ( x , β ) are distributedaccording to π .
However, if we use a continuous prior, samples will almost-surely not lie in that set, and we have to use importance sampling to correct for β ̸ = 1 .
This leads to a weighted sample fromπ , with the Monte Carlo accuracy of the resulting approach depending greatly on the variability ofthe weights introduced by importance sampling.
To overcome this issue, we use a prior p ( β ) that introduces a point mass at β = 1 .
Specifically, for α ∈ R , and an arbitrary probability density function κ on the interval [0 , 1) , we define p [Equation] β [Equation]
=
[Equation]
+
[Equation]
With the above prior, we can define the augmented target
[Equation]
We can extend the Zig-Zag process to sample from such a distribution.
The next subsection describesthe extension.
The usual advantages of subsampling and transdimensional sampling using PDMPsapply for the tempered extension.
3.2 Continuously-tempered Zig-Zag
Unlike most MCMC samplers, Zig-Zag is a continuous-time process.
This makes it simpler todeal with the point-mass at β = 1 , using ideas from Chevallier et al.
[2020]: we simulate paths forβ ∈ [0 , 1) until β hits the boundary at β = 1 ; when this happens, we set the velocity componentfor β , denoted as v d +1 t , to zero so that the Zig-Zag process explores the distribution ω restricted toβ = 1 ; the Zig-Zag process stays with β = 1 until the first event in a new Poisson process, at whichpoint we set the velocity component for β to − 1 to allow exploration of the distribution for β ∈ [0 , 1) .
We define the rate of the events when we leave β = 1 to be η .
Algorithm 2 provides a sketch of therelevant modifications of the standard Zig-Zag process for events concerning the inverse temperaturevariable β .
β t ∈ [0 , 1) until first time t at which trajectory hits β =2: v d +1 t ←▷ Kill the velocity component for β 3: Simulate an event-time τ with rate η 4: Run the Zig-Zag process with β = 1 , for time τ π 5: v d +1 t + τ ← −▷ Reintroduce β 6: Goto Step 1.
An appropriate choice of rate to ensure that the resulting algorithm generates a process with ω as itslimiting distribution is given in the following result (the proof of is located in the supplement).
Theorem 1 Assuming κ ( β ) is continuous and the rate function in Algorithm 2 is chosen as
[Equation]
Then, the resulting Zig-Zag process is ω -ergodic.
The theorem statement above considers the case where κ is continuous but we note in the proof thatmore general choice is possible with a slight adjustment to the rate.
Importantly, the rate η ( x ) isconstant, and thus simulating the time at which we transition from β = 1 to β < 1 is independent ofthe path of the process and simulating the event time is simple.
While we have described the use ofcontinuous tempering as an extension of the Zig-Zag process, the same construction readily extendsto other PDMP samplers — one simply replaces the Zig-Zag dynamics for β ∈ [0 , 1) and β = 1 in
Algorithm 2 with the dynamics of the corresponding PDMP sampler.
3.3 Importance sampling estimator
By construction, continuously-tempered Zig-Zag will spend a substantial amount of time in stateswith β = 1 , and thus the states at those times can give draws from π ( x ) .
We can use importancesampling ideas from Graham and Storkey [2017] to re-weight samples when 0 ≤ β < 1 to giveweighted samples from π ( x ) ; but this only applies if κ ( β ) ∝ ξ 1 − β for some constant ξ .
The idea of importance sampling is that, for β ∈ [0 , 1) we can marginalise out β from ω , and themarginal distribution for x is
[Equation]
Defining ∆( x )
= log q 0 ( x ) + log ξ − log q ( x ) , the above is equal to
[Equation]
Thus if we have this as our proposal distribution, then the corresponding importance sampling weightswill be w ( x ) =
∆( x ) / [exp { ∆( x ) } − 1] .
This will involve additional post-sampling computation ofthe importance weights at the samples taken from the Zig-Zag trajectory.
3.4 Calibration of κ ( β )
The efficiency of continuous-tempering depends on an appropriate choice of κ and α .
Ideally, theyshould be chosen so to balance a non-negligible amount of time at β = 1 , while simultaneouslyallowing occasional excursions to lower inverse temperatures to help mix, e.g. between modes.
For β ∈ [0 , 1] define, Z ( β ) =
(cid:82) R d q ( x , β ) d x .
The induced β -marginal of ω for β 0 ∈ [0 , 1) is
[Equation]
The above suggests that an appropriate choice is κ ( β ) ∝ Z ( β ) − 1 , as this would both induce β to bemarginally uniform under ω for β ∈ [0 , 1) , and cause the parameter α to directly represent the probability that β
= 1 under ω .
When not using the important sampling strategy, the proposed approach isthus to choose κ ( β ) ∝ (cid:98) Z ( β ) − 1 , where (cid:98) Z ( β ) =
exp (cid:110)(cid:80) m − 1 k =
a k β k (cid:111) for some coefficients { a k } m − 1 k
=0that are estimated by regressing point estimates of log Z ( β ) obtained via the path sampling identity. In practice, an estimate (cid:98) Z ( β ) can be obtained from a pilot run of the algorithm with α ≪ 1 or usingmethods described in Gelman and Meng [1998].
Numerical experiments
4.1 Mixture of Gaussians
In our first example, the target corresponds to a mixture of Gaussian distributions with equal weights and variances so our target has unnormalised density
[Equation]
where K = 5 , σ 2 = 0 . 2 and { µ 1 , ..., µ 5 } were generated uniformly on the region [0 , 10] × [0 , 10]and are given in the supplementary material.
Figure 1 plots the PDMP trajectories for the temperedand untempered Zig-Zag sampler, in addition to the trajectories for inverse temperature fixed at β = 1 .
For this example we choose q 0 ( x ) as a Gaussian N ( ν, Σ) centred at ν = (5 , 5) T with covariancematrix Σ
=
I 2 .
Table 1 displays the root-mean-square-error (RMSE) of the Monte Carlo estimates from the Zig-Zagand tempered Zig-Zag averaged over 20 runs.
All methods were simulated for 50,000 event-timeswith the first 40% used as burnin in the standard Zig-Zag and used for both burn-in and estimatingthe polynomial κ ( β ) in the tempered samplers.
The standard Zig-Zag is not able to explore themultiple modes yielding worse estimates of the first and second moments.
We also compare to adirect Zig-Zag analogue of the continuously tempered HMC algorithm Graham and Storkey [2017]where no mass is given to β = 1 and κ ( β ) ∝ 1 , the estimator is based on importance sampling asdefined in Section 3.3.
Further details and boxplots showing variability of the estimated moments aregiven in the supplementary material.
[Table]
We find that the tuning procedure yielded precise control over the time spent at β = [Table] and that for αbetween 0.1 and 0.3 the results are similar.
Tempering with a point mass was found to give moreefficient estimates of the first and second moments.
This may be because the importance samplingestimate spends more time at β ≈ 0 .
In addition to exhibiting lower RMSEs, the tempered versionsof the Zig-Zag sampler have better computational properties.
The thinning efficiency, measured asthe proportion of proposals that result in an event-time simulation is ≈ .
06 at β =
but improvessignificantly when the sampler can transition to lower temperatures.
While the importance samplingestimator for β = 0 has the best thinning efficiency it requires additional post processing to evaluatethe importance sampling weights.
4.2 Transdimensional example
We apply the tempered Zig-Zag to the challenge of sampling a transdimensional distribution.
Suchtarget distributions arise naturally in variable selection problems.
The resulting target (posterior)distribution is a discrete mixture of 2 p models, where p is the number of variables in the dataset.
Such a setting produces a challenge, as typical samplers such as HMC do not extend naturally tosuch spaces.
On the other hand, PDMPs have recently been extended to sample transdimensionaldistributions Chevallier et al. [2020, 2021], Bierkens et al.
This extension employs withinmodel gradient information to efficiently explore the space and jumps between models when aparameter hits zero.
The approach is beneficial as an informative likelihood function’s gradient willdirect less informative variables towards zero and more informative ones away from zero, aidingexploration of the sampler.
This example explores how tempering a target with a point mass can improve performance over thestandard transdimensional Zig-Zag process.
Specifically, we define a family of example problemsof increasing difficulty in the sense that for higher values of an underlying parameter m , separation between the mode of the slab component and the spike at zero is increased.
Allowq (
[Equation]
= 2 (cid:89)
[Equation]
[Equation]
where ϕ ( x i ; m, σ 2 ) is the normal density and w = 0 .
5 is the probability of a variable being included.
We fix σ 2 = 0 .
5 and increase m from m =
to m = 4 .
To enable tempering, define q ( x , β )
= 2 (cid:89) i =
[Equation]
where at β = 0 the spike and slab is centred at zero, encouraging variables to enter and exit the model.
While this is a somewhat artificial example, it is important to note that it encompasses precisely thesituations which transdimensional PDMPs will find challenging, and allows us to explore robustnessfor increasing levels of pathology in the purest setting possible.
Figure 2 (rightmost panel) displays the dynamics of the tempered Zig-Zag for this problem for thechoice of m = 4 .
Note that the standard transdimensional Zig-Zag process (bottom left) becomesstuck in a single model whilst its tempered counterpart is able to transition easily between modelsand thus visit the required areas for which each coordinate is equal to zero. Mean Absolute Error m =m =m =m =m = 4 E [ X 1 ] Zig-Zag 0 .0 .0 .1 .1 .
Zig-Zag CT 0 .0 .0 .0 .0 .
214 P ( | X 1 | > 0) Zig-Zag 0 .0 .0 .0 .0 . Zig-Zag CT 0 .0 .0 .0 .0 . Table 2 gives the absolute error for the Monte Carlo estimates of the marginal inclusion probabilitiesand marginal means.
For lower m , the standard and tempered Zig-Zag perform similarly as the modeof the slab is close enough to ensure regular crossing of the zero-axis for the PDMP trajectories.
However, as m increases, the marginal probabilities of inclusion tend to 1 and the standard Zig-Zagprocess becomes stuck in a single model.
4.3 Boltzmann machine relaxation
Following Nemeth et al.
[2019] and Graham and Storkey [2017], the final example considers samplinga continuous relaxation of a Boltzmann machine distribution.
Full details surrounding the derivation of the distribution can be found in [Nemeth et al., 2019, Supplementary Material, Section D], thoughfor the considered example the target density on R d r is of the form
[Equation]
where q k denotes the k -th row of Q , which is a d b × d r matrix such that QQ ⊤
=
W + D , and D isan arbitrary diagonal matrix that ensures W + D is positive semi-definite.
The above is a continuousrelaxation of the Boltzmann Machine distribution on {− 1 , 1 } d b with probability mass function
[Equation]
The moments up to second order of the relaxation and the original (discrete) Boltzmann Machinedistribution are related via E
[ X ] =
Q ⊤ E
[ S ] and E [ XX ⊤ ] =
Q ⊤ E [ SS ⊤ ] + I .
We employ asimilar experimental setup as in Nemeth et al.
[2019], namely a 28 -dimensional example whichallows for exact computation via enumeration of the moments of S , and hence in turn of X (thelatter being useful for evaluating sampler performance).
Table 3 displays the results.
We find that thebest performance is given by the Zig-Zag sampler with α = 0 .
2 which far outperforms the standard
Zig-Zag α = 1 and importance sampling version of the sampler.
For these experiments the tuning ofκ is notably sub-optimal as the proportion of time spent at zero is not fully controlled by α .
Despitethis, the samplers were able to outperform the importance sampling approach where α = 0 andthe standard Zig-Zag.
Further details on the experiment and specification of κ may be found in thesupplementary material.
machine relaxation averaged over 20 simulations reported to 3 decimal places.
[Table]
Discussion
We present a general strategy to improve the performance of PDMP samplers on challenging targets.
The approach uses an extended distribution that uses an inverse temperature which interpolatesbetween a challenging distribution of interest and a tractable base distribution.
At lower values ofβ the mixing will improve and we allow a pointmass at inverse at β = 1 to attain exact samplesfrom the distribution.
As a proof of concept, a numerical study of these ideas surrounding the Zig-Zag sampler was employed.
This revealed that there are considerable benefits to the proposedextensions, as well as to introducing the point mass at β = 1 .
We compared our tempering with apoint-mass extension of the Zig-Zag to a version based on continuously tempered estimates that aregiven using importance sampling ideas from Graham and Storkey [2017].
The importance samplingapproach requires restrictive choice of κ and gave inferior performance.
Our approach may bereadily applied to other PDMP samplers and can incorporate assets of sampling with a PDMP suchas sampling transdimensional spaces and using subsampling methods.
Another avenue of researchis in incorporating direct simulation from q 0 when the sampler hits β = 0 — such moves would beparticularly useful when q 0 is a tractable multimodal distribution.
Broader impact
Effective sampling methods are instrumental in many probabilistic modelling approaches that accountfor uncertainty quantification (e g., Bayesian Machine Learning approaches).
Thus, it is important toemploy reliable methods in obtaining such estimates, especially if they are used for downstream tasksand decisions.
The methodological innovations discussed herein aim to produce more reliable resultsin the context of PDMP samplers, which are a growing area of research.
The proposed approaches aremore suited to sampling challenging multi-modal distributions with PDMP-based samplers.
As themethods are presented as a proof of concept, and do not feature as an implemented component withinany deployed artificial intelligence system, there are no immediate broader consequences related totheir potential failures.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
[Yes](
b) Did you describe the limitations of your work?
[Yes See Conclusion
c) Did you discuss any potential negative societal impacts of your work?
[Yes] See Broader Impact statement.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] See Theorem
(b) Did you include complete proofs of all theoretical results?
[Yes] See Supplementary Material.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main exper-
imental results (either in the supplemental material or as a URL)?
[Yes] See Supplementary Material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
were chosen)?
[Yes] See Numerical Experiments section.
(c) Did you report error bars (e.g., with respect to the random seed after running ex-
periments multiple times)?
[Yes] See plots included in Supplementary Material for
visualization of variability for multiple runs.
(d) Did you include the total amount of compute and the type of resources used (e.g., type
of GPUs, internal cluster, or cloud provider)?
[Yes] See Supplementary Material.
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] See Section 4.3.
(b) Did you mention the license of the assets?
[Yes] See Supplementary Material.
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
Yes, see Supplementary Material.
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[Yes] See Supplementary Material.
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots , if applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]