ProtoVAE: A Trustworthy Self-Explainable Prototypical Variational Model
Anonymous Author(s)
AffiliationAddress email
Abstract
The need for interpretable models has fostered the development of self-explainableclassifiers.
Prior approaches are either based on multi-stage optimization schemes,impacting the predictive performance of the model, or produce explanations thatare not transparent, trustworthy or do not capture the diversity of the data.
Toaddress these shortcomings, we propose ProtoVAE, a variational autoencoderbased framework that learns class-specific prototypes in an end-to-end mannerand enforces trustworthiness and diversity by regularizing the representation spaceand introducing an orthonormality constraint.
Finally, the model is designed tobe transparent by directly incorporating the prototypes into the decision process.
Extensive comparisons with previous self-explainable approaches demonstrate the superiority of ProtoVAE, highlighting its ability to generate trustworthy and diverse explanations, while not degrading predictive performance.
Introduction
Despite the substantial performance of deep learning models in solving various automated real-worldproblems, lack of transparency still remains a crucial point of concern.
The black-box nature of thesehigh-accuracy achieving models is a roadblock in critical domains such as healthcare [1, 2], law [3], orautonomous driving [4].
This has led to the emergence of the field of explainable artificial intelligence(XAI) which aims to justify or explain a model’s prediction in order to increase trustworthiness,fairness, and safeness in the application of the complex models henceforward.
Consequently, two lines of research have emerged within XAI.
On the one hand, there are generalmethodologies explaining a posteriori black-box models, so-called post-hoc explanation methods[5, 6, 7].
While on the other hand, there are models developed to provide explanations along withtheir predictions [8, 9, 10].
The latter class of models, also known as self-explainable models(SEMs), are the focus of this work.
Recently, many methods have been developed for quantifyingpost-hoc explanations [11].
However, there is still a lack of a concise definition of what SEMs shouldencompass, thus a lack of comparability of recent methods [12].
Methodologically, a large number of SEMs follow the approach of concept learning, analogous to prototype or basis feature learning, where a set of class representative features are learned [8, 9].
In this paper we gauge SEMs through the prism of three properties.
First and foremost, the prototypes should be visualizable in the input space, and these transparent concepts should directly be employedby a glass-box classification model.
Many of the existing approaches try to imitate prototype transparency by using nearest training samples to visualize the prototypes [8, 13], while some flatlyuse training images as prototypes preventing an end-to-end optimization and limiting the flexibilityof the model [9, 14].
Secondly, the prototypes should exhibit both inter-class and intra-class diversity.
Methods failing to ensure this property [9] are prone to prototype collapse into a single point which necessarily undermines their performance.
Finally, SEMs should perform comparable to their black- box counterparts while producing robust and faithful explanations.
Previous approaches have atendency to achieve self-explanability by sacrificing the predictive performance [9, 14, 13].
To address the aforementioned shortcomings of current SEMs, we introduce ProtoVAE, a prototypicalself-explainable model based on a variational autoencoder (VAE) backbone.
The architecture and theloss function are designed to produce transparent , diverse , and trustworthy predictions, as well asexplanations, while relying on an end-to-end optimization.
The predictions are linear combinations ofdistance-based similarity scores with respect to the prototypes in the feature space.
The encoder anddecoder are trained as a mixture of VAEs sharing the same network but each with its own Gaussianprior centered on one of the prototypes.
The latter are enjoined to capture diverse characteristics ofthe data through a class-wise orthonormality constrain.
Consequently, our learned prototypes are trulytransparent global explanations that can be decoded and visualized in the input space.
Further, we areable to generate local pixel-wise explanations by back-propagating relevances from the similarityscores.
Empirically, our model corroborates trustworthiness both in terms of performance as well asthe quality of its explanations.
Our main contributions can be summarized as follows:
• We define three properties for SEMs, based on which we present a novel prototypical selfexplainable model with a variational auto-encoder backbone, equipped with a fully transparentprototypical space.
• We are able to learn faithful and diverse global explanations easily visualizable in the input space.
• We provide an extensive qualitative and quantitative analysis on five image classification datasets,
demonstrating the efficiency and trustworthiness of our proposed method.
Predicates for a self-explainable model
For the benefit of an efficient and comprehensible formalization of SEMs, we here define threeproperties that we consider as prerequisites for SEMs.
Definition
An SEM is transparent if:
(i) its concepts are utilized to perform the downstream task without leveraging a complex black-box
model;
(ii) its
concepts are visualizable in input space.
Definition
An SEM is diverse if its concepts represent non-overlapping information in the latentspace.
Definition
An SEM is trustworthy if:
(i) the performance matches to that of the closest black-box counterpart;(ii)
the explanations are robust, i.e., similar images yield similar explanations.(iii)
the explanations represent the real contribution of the input features to the prediction.
Note that these definitions echo properties and axioms found in other works.
However, the viewof such properties is diverse across the literature which leads to failure of encompassing the wideresearch of SEMs in general.
For example, transparency is known as ‘completeness’ in [15] and‘local accuracy’ in [16].
In the next section, we provide a comparison of existing SEMs based on thefulfillment of the proposed predicates.
Categorization of related self-explainable works
Self-explainable models optimize for both explainability and prediction, making the network inherently interpretable.
As our main contribution is a prototypical model, we review and categoriseexisting prototypical SEMs according to the above-mentioned properties.
SENN [8] introduces a general self-explainable neural network designed in stages to behave locallylike a linear model.
The model generates interpretable concepts, to which sample similaritiesare directly aggregated to produce predictions.
This generalized approach has been followed bymost of the prototypical and concept-based self-explainable methods, and is also mirrored by ourapproach.
SENN, however uses training data to provide interpretation of learned concepts, thereforeapproximating transparency, unlike our model which by-design has a decoder to visualize prototypes.
[Table]
ProtoPNet [9] is a representative of a line of works [9, 14, 18, 19, 20], where a prototypical layer isintroduced before the final classification layer.
For maintaining interpretability, the prototypes areset as the projection of closest training image patches after every few iterations during training.
Ourmethod is closely related to ProtoPNet with the distinction of decode-able learned prototypes yieldinga smooth and regularized prototypical space, thus allowing more flexibility in the model.
TesNet[14] extends ProtoPNet and improves diversity at class level using five loss terms.
Similarly to ourapproach, they distribute the base concepts among the classes and include an orthonormal constraint.
However, the basis concepts are still projections from the nearest image patches, which leads to loss inpredictive performance, similar to ProtoPNet.
SITE [17] incorporates generates class prototypes fromthe input and introduces a transformation-equivariant model by constraining the interpretations beforeand after transformation.
Since the prototypes are dynamic and generated for each test image, thismethod only provides local interpretations and lacks global interpretations.
FLINT [13] introduces aninterpreter model (FLINT- g ) in addition to the original predictor model (FLINT- f ).
Although FLINTf has been introduced by the authors as a framework that learns in parallel to the interpretations, it isnot an SEM on its own.
Therefore, we focus on FLINT- g , henceforward referred to as FLINT.
FLINTtakes as input features of several hidden layers of the predictor to learn a dictionary of attributes.
However, the interpreter is not able to approximate the predictor model perfectly, therefore losingtrustworthiness .
Unlike prior approaches, ProtoVAE is designed to fulfill all three SEM properties.
We summarize the discussed methods and their categorization in Table 1.
ProtoVAE
[Figure]
In this section, we introduce ProtoVAE, which is designed to obey the aforementioned SEM properties.
Specifically, transparency is in-built in the architecture and further enforced along with diversitythrough the loss function.
Also, we describe how our choices ensure the trustworthiness of ourmethod.
4.1 Transparent architecture
In a transparent self-explainable model, the predictions are interpretable functions of conceptsvisualizable in the input space.
To satisfy this property, we rely on an autoencoder-based architectureas backbone and a linear classifier.
In order to have consistent, robust , and diverse global explanations,we consider prototypes in a greater number than classes.
Unlike previous prototypical methods [9,14], which update the prototypes every few iterations with the embedding of the closest trainingimages, ProtoVAE is trained end-to-end to learn both the prototypes in the feature space and theprojection back to the input space.
This gives ProtoVAE the flexibility to capture more generalclass characteristics.
To further alleviate situations where some of the optimized prototypes arepositioned far from the training data in the feature space, possibly causing poor reconstructions andinterpretations, we leverage a variational autoencoder (VAE).
VAEs are known to learn more robustembedding and thus generate better reconstructions from out-of-distribution samples than simpleautoencoders [21].
A schematic representation of the network is depicted in Fig.
Details of the operations The downstream task at hand is the classification into K > 0 classes of theimage dataset X
= { ( x i , y i ) }
Ni =1 , where x i ∈
R p is an image and the one-hot vector y i ∈ { 0 , 1 } K encodes its label.
The network consists of an encoder f : R p → R d × R d , a decoder g : R d → R p( d < p ), M prototypes per class Φ = { ϕ kj } k =1 ..K j =1 ..M , a similarity function sim : R d → R M and aglass-box linear classifier h : R M → [0 , 1] K .
An image x i is first transformed by f into a tuple( µ i , σ i ) =
f ( x i ) which, in the VAE realm, are the parameters of the posterior distribution.
A featurevector z i is then sampled from the normal distribution N ( µ i , σ i ) and is used twice.
On the one hand,it is decoded as ˆ x i =
g
( z i ) .
On the other hand, it is compared to the prototypes.
We use the samesimilarity function as in [9] and obtain the resulting vector s
i ∈ R K × M as:
[Equation]
with 0 < ϵ < 1 .
Finally, s i is used to compute the predictions: ˆ y i
=
h ( s i ) .
Moreover, the similarityvector s i captures the distance to the prototypes but also indicates the influence of each prototype onthe prediction.
4.2 Diversity and trustworthiness
Unlike transparency, diversity cannot be achieved solely through the architectural choices.
It needs tobe further enforced during the optimization.
Our architecture implies two loss terms: a classificationloss and a VAE-loss.
Without further regulation, our model is left vulnerable to the curse of prototypecollapse [14, 22] which would undermine the SEM diversity property.
We prevent such a situationwith a third loss term enforcing orthonormality between prototypes of the same class.
The lossfunction of ProtoVAE can thus be stated as follows:
[Equation]
We detail now each term and discuss how they favor diversity and trustworthiness .
Inter-class diversity through classification Although the prototypes are assigned to a class, theclassifier is blind to that information.
Thus, the prediction problem is a classic classification that wesolve using the cross-entropy loss.
[Equation]
Since h is linear, the loss pushes the embedding of each class to be linearly separable, yielding agreater inter-class diversity of the prototypes.
Intra-class diversity through orthonormalization The inter-class diversity is guaranteed by theprevious terms.
However, without further regularization, the prototypes might collapse to the centerof the class, obviating the possibilities offered by the extra prototypes.
To prevent such a situationand foster intra-class diversity, we enforce the prototype of each class to be orthonormal to each otheras follows:
[Equation]
where I M is the identity matrix of dimension M × M and the column-vectors of matrix ¯ Φ k arethe prototypes assigned to class k minus their mean, i.e., ¯ Φ k = { ϕ kj − ¯ ϕ k , j = 1 . .
M } with¯ ϕ
k = (cid:80) Ml =1 ϕ kl .
Beyond regularizing the norm of the prototype, this term favors the disentanglementof the captured concepts within each class, which is one way to obtain intra-class diversity .
Robust classification and reconstruction through VAE The VAE architecture ensures the robustness of the embedding and of the decoder.
In its original form, the VAE loss considers a singlestandard normal distribution as a prior and is trained to minimize:
[Equation]
[Equation]
where I d is the identity matrix of dimension d × d .
Such an objective enjoins the embedding toorganize as if generated by a single Gaussian distribution, thus making it difficult to split it with thelinear classifier h .
To help the classifier, we consider instead a mixture of VAEs sharing the samenetwork each with a Gaussian prior centered on one of the prototypes.
Since, each prototype has alabel, only data-points sharing that label are involved in the training of the associated VAE.
The lossfunction of our mixture of VAEs is (derivation in the supplementary materials):
[Equation]
In addition to training the decoder, this loss enjoins the embedding to gather closely around theirclass prototypes.
4.3 Visualization of explanations
ProtoVAE incorporates in-built global explanability in the form of decode-able learned prototypes.
Additionally, to generate faithful pixel-wise local explanation maps, we build upon the conceptsof Layer-wise relevance propagation (LRP) [23] which is a model-aware XAI method computingrelevances based on the contribution of a neuron to the prediction.
Following [12], we generate explanation maps, where for each prototype, the similarity of an input to the prototype is backpropagatedto the input image according to LRP’s rules.
For an input image x i , the point-wise similarity betweenthe transformed mean vector µ i with a prototype ϕ kj is first calculated as:
[Equation]
ikj = [Equation] with d ikj = ( µ i − ϕ kj ) ∗ ( µ i − ϕ kj ) , (7) where ∗ is the Hadamard element-wise product and η > 0 .
The similarity γ ikj is then backpropagatedthrough the encoder following LRP composite rules.
See [24] for further details.
Experiments
In this section, we conduct extensive experiments to evaluate ProtoVAE’s trustworthiness, transparency, and ability to capture the diversity in the data.
More specifically, we demonstrate thetrustworthiness of our model in terms of predictive performance in Sec. 5.1.
Qualitative evaluationsare then conducted in Sec.
5.2 to verify the diversity and transparency properties, followed by aquantitative evaluation of the explanations corroborating its trustworthiness.
We further provide anablation study for the terms in Eq.
2 in the supplementary.
Datasets and implementation: We evaluate ProtoVAE on 5 datasets, MNIST [25], FashionMNIST[26] (fMNIST), CIFAR-10, [27], a subset of QuickDraw [28] and SVHN [29].
We use small encodernetworks with 4 convolution layers for MNIST, fMNIST and CIFAR-10, 3 for QuickDraw and 8 for
SVHN.
These convolution layers are followed by 2 linear layers which gives us the tuple ( µ i , σ i ) foreach image i .
The decoder mirrors the encoder’s architecture.
Similar to [9], we fix the prototypes perclass, M , to 5 for MNIST and SVHN and 10 for the other datasets.
Further details about the datasetsand additional implementation details, such as the detailed architecture and hyperparameters, areprovided in the supplementary.
Code is also added in the supplementary and will be made publiclyavailable upon acceptance.
Baselines Since our model is closely related to ProtoPNet [9] and thanks to the availability ofits code, this is our primary baseline. To ensure a fair comparison, we train ProtoPNet by usingour respective encoder architectures as backbones and use the same number of prototypes for eachclass for different datasets as ProtoVAE.
In addition, we provide the results for the predictiveperformance of FLINT, SENN, and SITE as reported in [13, 17].
Further comparisons are included
[Table]
[Figure] [Table] [Figure] [Table]
[Figure] [Table] [Figure] [Table]
in the supplementary material.
Finally, we also compare with the black-box counterpart of ourmodel which consists of the encoder followed by the linear classifier and trained end-to-end with thecross-entropy loss.
This black-box encoder model is thus free from all regularization necessary forself-explainability.
5.1 Evaluation of predictive performance
In the Table 2, we can observe that ProtoVAE surpasses all other SEMs in terms of predictiveperformance on all five datasets, which is based on its increased flexibility in the architecture.
Additionally, while FLINT uses ResNet-18 [30] based architectures as backbones for QuickDraw and CIFAR-10, we still surpass the accuracy while using small encoders. We also report the performanceof our model with a ResNet-18 backbone in the supplementary.
For ProtoPNet, we observe a gap inperformance, which is due to the low number of optimal class-representatives in the actual trainingdata.
This creates a huge bottleneck at the prototype layer and therefore limits its performance.
Further, and more importantly, when compared to the true black-box counterpart, ProtoVAE achievesno loss in accuracy and is even able to perform better on all the datasets.
We believe this is due toan efficient over-clustering of the latent space with the flexible prototypes, as well as the naturalregularizations achieved through the VAE model.
These results strengthens the trustworthiness of ProtoVAE in terms of the predictive performance.
5.2 Evaluation of explanations
Qualitative evaluation
The demonstrated results in this section strengthen the fulfilment of thetransparency property by providing human-understandable explanations for ProtoVAE.
We visualizethe decoded prototypes for different datasets, which act as global explanations for the correspondingclasses, in Fig.
For example, the prototypes for MNIST demonstrate that class 2 consist of 2’s withflat bottom line or with rounded bottom lines.
For fMNIST, the sandals class consists of both heelsand flats.
The class prototypes thus directly help visualizing the components of the classes by lookingat a fixed number of prototypes per class instead of all the training data.
Interestingly, although
SVHN often contains multiple digits of different classes in the same training image, our prototypesefficiently capture only one digit representing its class.
Moreover, a blurring effect is observed in ourprototypes which captures more variability and therefore suggests efficient representation of the true“mean" of a subset of a given class, as opposed to other methods [9, 14, 13] which show the closesttraining images and are therefore sharper.
This behavior supports our claim of more flexibility in thenetwork, therefore enhancing predictive capability along with the ability to provide more faithfulexplanations.
This blurring effect is observed to be more prominent in CIFAR-10, which is due to [Figure] E P r o t o
P N
[Figure]
t [Figure] the high complexity in each class in the dataset and can be reduced by using a larger number ofprototypes per class. Additionally, to provide more clear visualization of the learned transparentprototypical space, we show UMAP representations of the prototypes and the training data for MNISTin Fig.
This visualization further illustrates the inter-class as well as the intra-class diversity of theprototypes.
Moreover, due to the regularized prototypical space, we are efficiently able to interpolatebetween prototypes both within a class and between classes, therefore making the latent space fullytransparent .
In Fig.
3, we interpolate between 2 different prototypes of class ‘2’ and from a prototypeof class ‘2’ to a class ‘7’ prototype.
The local explanability maps for a test image according to the three closest, i.e. most similar,prototypes for both ProtoVAE and ProtoPNet are shown in Fig.
4, along with the correspondingsimilarity scores.
As seen, different prototypes of the same class activate different parts of the sametest image, which therefore helps in achieving better performance.
The ProtoPNet maps are extremelycoarse which therefore makes them challenging to interpret.
Therefore, we overlay the heatmapsover the input image for ProtoPNet.
As observed, the most activated prototypes do not belong tothe same class as the test image.
This might happen because of ProtoPNet focusing on patches inprototypes, therefore losing contextual information.
The 3 closest prototypes shown for the image 0.108 Test Image D og A n t
A pp l e
C a t
Test Image Nearest prototypes from different classes 0. Nearest prototypes from different classes [Figure]
‘apple’ belong to class ‘lion’.
Further, an uninformative training image, which is not seen in the
ProtoVAE prototypes, has been selected by ProtoPNet to represent 5 out of 10 prototypes for class‘lion’.
The remaining 5 prototypes are represented by 1 other same training image.
This effect is seenpredominantly in ProtoPNet where the prototypes of the same class collapse to one point and are thusrepresented by the same training image, therefore dissatisfying the diversity property, as opposed to ProtoVAE.
The prototypes for class ‘lion’ for both the models are included in the supplementary.
We also show the closest prototypes from 3 different random classes and their corresponding explainability maps to demonstrate the behavior of explanations for different class prototypes in Fig.
Interestingly, the ‘dog’ image from the QuickDraw dataset resembles an ‘ant’ prototype for the legs,an ‘apple’ prototype for the face and a ‘cat’ prototype for the ears.
This information provided by thelocal explainability maps thus aligns well with human-understandable concepts.
Finally, to compare the efficacy of the mapping to the input space learned by our decoder, tomethodologies with training-data projection of prototypes [9, 14], we show prototypes along withthe 3 closest training images for different datasets in Fig.
The prototypes are observed to be therepresentative of a subset of the respective class.
For example, the prototype shown for class ‘4’ of MNIST is representing the subset of ‘4’ with an extended bar, while the ‘banana’ prototype representsthe left facing ‘banana’ subset, and the ‘dog’ prototype represents the subset of white dogs on adarker background.
Quantitative evaluation To quantify the trustworthiness of the explanations provided by the 265 proposed model, we calculate the Average Drop (AD) and Average Increase (AI) with respect to localexplanation maps and similarity scores for all prototypes [31, 2].
The AD measures the decrease in 267 similarity scores with respect to each prototype when the 50% least important pixels are removedfrom the images, while AI estimates the ratio of increasing similarity scores.
A low AD and a high 269 AI suggest better performance.
These scores are computed as follows:
[Equation]
where s i ( k, j ) is the similarity score of an image i with prototype j of class k (see Eq.1) ands 50% i ( k, j ) is the similarity score after masking the 50% least activated pixels according to theprototypical explanation map of prototype j .
Also, [[ · ]] are the Iverson brackets which take the value1 if the statement they contain is satisfied and 0 otherwise.
We report AD and AI computed over 100 random test images for ProtoVAE and ProtoPNet in Table 3.
For the grayscale datasets, MNIST and fMNIST, the masked pixels are replaced by 0 .
For {+CIFAR-
 [Table]+} and SVHN, they are replaced by random uniformly sampled values.
ProtoVAE achieves considerablylower AD and higher or comparable AI for all the datasets.
For SVHN, ProtoPNet performs wellwhich we believe is due to the abundance of representative patches in the dataset, thereby improvingits explanations.
Finally, we perform a relevance ordering test [32, 12], where we start from a random image andmonitor the predicted class probabilities while gradually adding a percentage of the most relevantpixels to the random image according to the local explanation maps.
We take 100 random test imagesand report the average results of change in predicted class probability for all the prototypes in themodel.
The rate distortion graphs are shown in Fig.
7 for MNIST, QuickDraw and SVHN.
We alsoinclude two baselines, Random-ProtoPNet and Random-ProtoVAE, where the pixel relevances areordered randomly.
Larger area under the curve indicates better performance.
As shown, ProtoVAE’slocal explanations are able to capture more relevant information than ProtoPNet for all three datasets.
Further, for MNIST, ProtoPNet is performing even worse than Random-ProtoPNet, highlighting thelack of trustworthiness in ProtoPNet’s explanations.
Conclusion and Discussion
In this work, we define three properties that act as prerequisites for efficient development of SEMs,namely, transparency , diversity and trustworthiness .
We then introduce ProtoVAE, a prototypicalself-explainable method, based on a variational auto-encoder backbone, which addresses these threeproperties.
ProtoVAE incorporates a transparent model and enforces diversity and trustworthinessthrough the loss functions.
In addition to providing faithful explanations, ProtoVAE is able to achievebetter predictive performance than its counterpart black-box models.
The main limitation of ProtoVAE is the fixed number of prototypes.
This means that the model has tograsp simple as well as more complex classes with the same number of prototypes.
For example, in MNIST, there are more variations to be captured by the prototypes in the class ‘4’ than in class ‘1’.
Asimple but effective solution is a distance-based pruning procedure, which will be explored in futureworks.
Another approach in sight is to use a prior on the distribution of the prototypical similaritiesand prioritize some prototypes by controlling the frequency with which each prototype is used in thepredictions.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
[Yes](b) Did you describe the limitations of your work?
[Yes] Discussed in section 6.
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] Included in thesupplementary(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?[Yes]2.
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A](b)
Did you include complete proofs of all theoretical results?
[N/A]3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimentalresults (either in the supplemental material or as a URL)?
[Yes] Provided in the supplementary.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they werechosen)?
[Yes] Details reported in the supplementary(c)
Did you report error bars (e.g., with respect to the random seed after running experimentsmultiple times)?
[Yes] Mean and standard deviation for accuracy are reported.(d)
Did you include the total amount of compute and the type of resources used (e.g., type ofGPUs, internal cluster, or cloud provider)?
[Yes] Included in the supplementary.4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a)
If your work uses existing assets, did you cite the creators?
[Yes](b) Did you mention the license of the assets? [Yes] Included in the supplementary(c)
Did you include any new assets either in the supplemental material or as a URL?
[Yes]Provided in the supplementary.
(d) Did you discuss whether and how consent was obtained from people whose data you’reusing/curating?
[N/A] All the datasets used in this work are open-source.
[Table] (e) Did you discuss whether the data you are using/curating contains personally identifiable [Table] information or offensive content?
[N/A]
The datasets used are open-source and do not containpersonally identifiable information or offensive content.5.
If you used crowdsourcing or conducted research with human subjects...
[Table]
(a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A]
[Table] (b) Did you describe any potential participant risks, with links to Institutional Review Board [Table]
(IRB) approvals, if applicable?
[N/A]
[Table]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on [Table] participant compensation?
[N/A]
[Table]