Parameter-Efficient Image-to-Video Transfer Learning
Anonymous Author(s)
AffiliationAddress email
Abstract
Capitalizing on large pre-trained models for various downstream tasks of interesthave recently emerged with promising performance.
Due to the ever-growingmodel size, the standard full fine-tuning based task adaptation strategy becomesprohibitively costly in terms of model training and storage.
This has led to anew research direction in parameter-efficient transfer learning.
However, existingattempts typically focus on downstream tasks from the same modality ( e.g. , imageunderstanding) of the pre-trained model.
This creates a limit because in somespecific modalities, ( e.g. , video understanding) such a strong pre-trained modelwith sufficient knowledge is less or not available.
In this work, we investigatesuch a novel cross-modality transfer learning setting, namely parameter-efficientimage-to-video transfer learning .
To solve this problem, we propose a new SpatioTemporal Adapter (ST-Adapter) for parameter-efficient fine-tuning per video task.
With a built-in spatio-temporal reasoning capability in a compact design, STAdapter enables a pre-trained image model without temporal knowledge to reasonabout dynamic video content at a small ( ∼ 8%) per-task parameter cost, requiringapproximately 20 times fewer updated parameters compared to previous work.
Extensive experiments on video action recognition tasks show that our ST-Adaptercan match or even outperform the strong full fine-tuning strategy and state-of-theart video models, whilst enjoying the advantage of parameter efficiency.
Introduction
In the NLP field, almost all the state-of-arts across a wide range of downstream tasks have beenachieved by adapting from large pretrained models (a.k.a.
foundation models [7]) such as BERT [14]and GPT [52, 8].
The de facto standard approach to adapting a pretrained model to down-stream tasksis fine-tuning either fully or partially ( e g. , linear probing by training the newly added multi-layerperceptron layers on the top alone), subject to the condition of adopting a similar network architectureas the pretrained model.
Nonetheless, given increasingly larger whilst ever stronger foundation models ( e.g. , GPT-3 with 175B parameters), fully fine-tuning the whole model for every single downstream task would become prohibitively expensive and infeasible in terms of training cost and model storage.
This could significantly restrict their deployment and usability in real-world applications.
In this context, a series of NLP works has been introduced towards efficient transfer learning with better trade-offs between parameter and accuracy [24, 23, 38, 35].
This trend has recently motivated the computer vision community.
For example, the CLIP model[53], trained with 400 million web image-text pairs, achieves promising performances on a variety of image recognition and generation tasks.
In the video domain, with significantly more computational cost and resources, Xu et al.
[77] trained a video variant of CLIP but excelled on a smaller number of downstream video tasks.
This is partly attributed to two orders of magnitude more minor training data and limited availability of computing resources, as large video data is notoriously more difficult
ST
[Figure]
Add
[Figure]
FeedForward
Add & Norm
[Figure] [Figure]
Upsample
[Figure]
Spatio-Temporal Adapter
[Figure]
& Norm
[Figure]
Newly added to ViT [
[Figure]
Downsample to collect, manage, and process than image data.
Under these restrictions, large pre-trained imagemodels are arguably still favorable in the selection of model initialization for video tasks.
In this work, we investigate a novel, critical problem of efficiently adapting large pre-trained imagemodels for video downstream tasks , with a focus on the widely influential action recognition task.
Considering that training video models is drastically more expensive in both computing resourceand time than image models [18], this problem becomes particularly more useful and valuable inpractice.
On the other hand, it is also more challenging and non-trivial due to the extra necessityof overcoming the big gap between image and video in transfer learning.
Especially, pre-trainedimage models lack the ability to infer temporal structured information, which however is criticalin video understanding.
In fact, the key design with state-of-the-art video models [10, 40, 6, 9] isusually about learning the temporal dimension based on contemporary image models.
Althoughmodel initialization is still important, they largely go beyond the fine-tuning strategy, as architecturalmodification is often imposed in addition to full model training/fine-tuning per downstream task.
Given that this is a new problem, we first conduct a comprehensive benchmark using both variousfine-tuning methods for image-to-video transfer learning and state-of-the-art video models [6, 9].
Regarding the pretrained image model, we select two Vision Transformer (ViT) [15] models, with onefrom CLIP pre-training [53] and the other pre-trained on ImageNet-21K [13].
ViT is representativein terms of network architecture, pre-training algorithm, and training data scale.
Crucially, wefurther propose an efficient yet effective Space-Time Adapter (ST-Adapter), capable of extracting andleveraging the pre-trained knowledge of a large image model to achieve superior video understandingat a small parameter cost.
Specifically, ST-Adapter is formulated based on a novel parameter-efficientbottleneck with a sequence of operations including feature dimension reduction, spatial-temporalmodeling, and feature dimension recovery.
It is easy to implement and scalable for deployment sinceall the primitive steps are realized with standard operators ( e.g. , fully-connected layer, depth-wise 3Dconvolution).
With such a lightweight design, our bottleneck can be cheaply integrated throughoutthe base network for enabling stronger layer-wise spatio-temporal learning.
As a result, our modelcan be more rapidly optimized using fewer training epochs for significant convergence advantage.
We summarize the contributions as follows.
(1) We investigate a new problem of parameterefficient image-to-video transfer learning .
Our motivation is to advocate the usability and deploymentof increasingly larger whilst ever more powerful pre-trained image models in benefiting morechallenging video understanding tasks.
(2) We establish a benchmark for action recognition tasks bycomprehensively experimenting with a variety of fine-tuning strategies and several state-of-the-artvideo understanding models.
(3) We introduce a novel parameter-efficient Spatio-Temporal Adapter(ST-Adapter) for more effectively capitalizing a large pre-trained image model in video understanding.
By grounding all the primitives on standard operators, ST-Adaptor is easy to implement and friendlyto deployment.
(4) Extensive experiments on action recognition datasets show that our ST-Adapteroutperforms not only existing parameter-efficient alternatives and the full fine-tuning strategy, butalso state-of-the-art video methods with the same network architecture and model initialization.
Related Work
Parameter-efficient transfer learning Driven by the wider application of large pre-trained languagemodels across a diversity of downstream tasks, the topic of efficient tuning has received increasingattention in NLP.
Existing efficient tuning methods fall broadly into three categories.
The firstcategory is to introduce task-specific adapters [24, 23, 49, 48].
Specifically, an adapter consistsof lightweight modules inserted between layers of a pre-trained model.
To be parameter-efficient,only those newly added adapter modules need to be updated during task fine-tuning, whilst all theparameters of the large pre-trained model, which takes the majority proportion of the whole solution,are frozen.
The second category is prompt tuning [38, 50, 30, 60, 41].
Instead of manipulating thenetwork architecture, these methods prepend a set of learnable tokens at the input point of the modelor intermediate layers.
Similarly, only these added tokens need to be optimized for each downstreamtask.
The third category is learning weight approximation [26].
In particular, only the low-rankmatrices for approximating the weights need to be updated during training.
Early works for efficient transfer learning in vision focus on parameter sharing in the context ofmultitask learning [81, 55, 54].
Recently, there are several works for extending the efficient tuningidea from NLP to vision tasks.
CoOp [82] and CoCoOp [83] apply prefix tuning for adapting the CLIPmodel to various image recognition tasks.
VL-Adapter [63] achieves the performance comparableto full fine-tuning on challenging vision-language tasks.
Commonly, their design focuses are allrestricted to the text encoder of the CLIP model.
More recently, [28] and [4] introduce the idea ofprompt learning to visual backbones.
They obtained favorable results on various image recognitionbenchmarks.
Moving a step further, in this work, we consider the more challenging adaptationproblem from a pre-trained image model without temporal knowledge to video understanding tasks.
Video action recognition Action recognition in the unconstrained video has largely been dominatedby deep learning methods, thanks to the availability of large video datasets, e.g. , Kinetics [10–12]and Something-Something [21].
As a key component, the model architectures adopted by existingvideo methods has expanded from CNNs [31, 66, 18, 17, 75, 67, 70, 40, 44] to Transformers[16, 39, 37, 43, 2, 6].
As temporal information is important for modeling the dynamics, a varietyof motion learning techniques has been introduced [73, 29, 47].
Further, different training methodshave also been explored, e.g. , unsupervised learning [65, 19, 74], and video-text contrastive learning[62, 77, 76, 64].
New opportunities for stronger video models are created following the introductionof large pretrained foundation models [53, 27, 79].
For example, Wang et al.
[72] equipped the CLIPwith temporal modules and good performance can be achieved after the model is fully fine-tuned onvideo datasets.
Ju et al.
[30] adopted the CLIP model for video recognition tasks by learning videospecific prompts.
In contrast, in this work, we explore the potential of the large pre-trained imagemodels with the parameter-efficient adapter strategy.
Importantly, despite the simplicity, we bringabout more significant advantages in performance along with a new benchmark on parameter-efficientimage-to-video transfer learning.
Methodology
To capitalize a large pre-trained image model for more challenging video understanding such as actionrecognition in a cross-modality manner, it is necessary to fill the intrinsic gap between image andvideo.
For easier understanding, we start with an intuitive baseline based on temporal aggregation.
Temporal aggregation
A straightforward baseline method of exploiting a pre-trained image modelfor video understanding is to temporally aggregate per-frame feature representations ( e.g. , averagepooling).
Concretely, given an input video clip V ∈ R T × H × W , where T, H, W are the number offrames, height and width respectively.
Following [15], we first split each frame into N = H × W/P 2patches of size P × P .
Then, we flatten these patches and project them into a sequence of patchtokens Z t =
[ z 1 , ...
z s , ..., z N ] , z s ∈ R d where d = 3 × P 2 with t = 1 , ..., T .
The sequence offeature vectors is then enhanced with the positional embedding by element-wise addition, along witha trainable class token concatenated.
Subsequently, we feed each sequence with N + 1 tokens to astack of self-attention based blocks individually.
For each sequence we keep only the classificationtoken z clst .
We further perform temporal average pooling on the class
tokens z final = 1 T (cid:80) t z clstto yield a compact representation for the whole clip.
We obtain the prediction by passing z finalthrough a classifier.
As the sptial information is only naively averaged over time, it is also known as
Space-Only TimeSformer [6].
Spatio-temporal attention For more dedicated structural modeling in the time dimension with
ViTs, a mainstream approach in the video domain is to develop various spatio-temporal attentionmechanisms by further imposing temporal attention on top [6, 2, 3, 9, 80, 22].
We choose tworepresentative video ViT models, TimeSformer [6] and XViT [9], in our performance benchmark.
However, state-of-the-art video ViT models often need to fully fine-tuned per task, which is parameterinefficient , given that in this way we have to keep a separate copy of the whole fine-tuned modelparameters for every single task.
3.1 Preliminaries
Our method is inspired by the Adapter [24] designed for parameter-efficient transfer learning in
NLP.
Specifically, the adapter module is composed of a down-projection linear layer followed bya non-linear activation function and an up-projection linear layer.
Formally, given an input featurematrix X ∈ R N × d at the i -th layer, the feature adaptation process can be written as:
[Equation]
where W down ∈ R d × r refers to the down projection layer, W up ∈ R
r × d the up-projection layer, andf ( · ) the activation function.
Note, that a residual summation is applied for preserving the informationin input as required.
The idea of Adapter has been remarkably successful in NLP due to severaladvantages: (1) High parameter efficiency across tasks since only a small number of parameters aretask-specific; (2) Reaching on-par performance compared to full fine-tuning; (3) Taking significantlysmall training costs; (4) Avoiding the catastrophic forgetting limitation of full fine-tuning.
We aim to propagate the success of Adapter from NLP to computer vision particularly the imageto-video transfer learning problem as discussed earlier.
To that end, we introduce a novel Adaptertailored specially for spatio-temporal reasoning – a key capability for video understanding which,however, existing NLP Adapter variants lack.
3.2 Spatio-Temporal Adapter (ST-Adapter)
Typically, an image model only considers the ability of spatial modeling.
The objective of our
Spatio-Temporal Adapter (ST-Adapter) is to enable a pre-trained image model to reason aboutspatial and temporal information of video in a parameter efficient principle.
In design, we considera couple of practically-crucial criteria: (1) Smaller parameter size : The parameter cost for eachdownstream task should be small – the essential criterion for parameter efficiency.
(2) Developmentfriendliness : This is critical for real-world development and deployment.
In practice, it is necessarythat a model can be easily implemented using the standard highly optimized deep learning toolboxes( e.g. , PyTorch, TensorFlow, TensorRT, and TorchScript), without tedious per-toolbox specialization.
This also facilitates the realization of high inference efficiency across a diversity of running platformsdue to the best usage of built-in software and hardware resources.
Under these considerations, we formulate the proposed ST-Adapter by sticking to commonly-adoptedprimitive operators alone .
Starting with the above Adapter (Eq.
(1)) originally developed for
NLP tasks, we further introduce a spatio-temporal operator realized by a standard depth-wise 3Dconvolution layer [17] between the bottlenecks (Figure 1).
In particular, our spatio-temporal operator enables layer-wise temporal inference efficiently, because it only operates in a compressed lowdimensional ( e.g. , 128D) feature space and the depth-wise convolution is highly efficient bothin parameter and computation [25].
As a result, this yields an introduction of tiny extra ( ∼ 2%)parameters and ( ∼ 0.3%) computation.
Formally, our ST-Adapter can be expressed as:
[Equation]
where DWConv3D denotes the depth-wise 3D-convolution for spatio-temporal reasoning we introduce.
It is noteworthy that before applying DWConv3D , the down-projected feature representations will befirst reshaped from X ′ ∈ R T × N × d to X ′′ ∈ R T × h × w × d (where N = h × w ) to have the spatial andtemporal dimensions prepared for reasoning.
With this highly integrated design, our ST-Adapterenjoys the same efficiency and flexibility as the NLP Adapter, while uniquely being able to conductspatio-temporal modeling.
3.3 ST-Adapter Integration
For proper adaptation, the adapter modules are often integrated between layers of a Transformer.
In NLP, a variety of integrating designs have been investigated.
For example, [24] deploys twoadapter modules per layer with one following the Multi-Head Self-Attention (MHSA) and the otherfollowing the Feed-Forward Networks (FFN) [24].
On the other hand, [61, 5] suggest that addingonly one adapter after the FNN suffices.
Similarly, our ST-Adapter can be also integrated generally atdistinctive positions.
Empirically, we find that a decent performance can be achieved in case a single
ST-Adapter is placed before the MHSA of each transformer block (Figure 1(a) and Table 4c).
Experiments
4.1 Experiments Setup
Datasets For the benchmark experiments, we use two popular video action recognition datasets.
Kinetics-400 (K400) :
The Kinetics [32] dataset contains ∼ 240k training videos and 20k validationvideos labeled with 400 action categories.
Most videos have a length of 10s or about 300 frames.
While there is a great diversity in these videos, they are largely biased to spatial appearance [58].
Something-Something-v2 (SSv2) :
The SSv2 [21] dataset consists of 220,487 videos covering 174human actions.
The video length ranges from 2 to 6 seconds.
In contrast to K400, SSv2 presentsricher temporal information with much higher significance [58].
Pre-trained models In all experiments, we use the standard ViT [15] as our base backbone model.
We conduct most of our experiments with the ViT-B/16 variant with 12 layers and 86M parameters,taking as input a sequence of patches at size 16 × 16 .
What was learned during pre-training directly decides the knowledge that can be transferred todownstream tasks, thus also the effectiveness upper bound of transfer learning methods.
To thisend, we benchmark the same backbone under two different pre-training strategies: pre-training withweb-scale raw data that has been recently proposed by CLIP [53] (400M image-text pair) and classicalsupervised pre-training on annotated data from ImageNet-21K (21k classes and 14M images).
Implementation details.
Implementation details, including training and testing settings and moduleinstantiation details, are provided in the appendix.
Competitors We provide several transfer learning approaches in our benchmark for efficient imageto-video transfer learning.
Note that the parameters of the linear classifier are always updated duringtraining for all approaches.
(1) Full Fine-tuning : Fully updating all the parameters when adapting for a specific target task.(2) Partial Fine-tuning : Only update the last ViT layer while keeping the rest of the parameter fixed.(3) Temporal Fine-tuning : We only tune the temporal attention modules ( i.e., TA) in the SA+TA
architecture.(4) Linear Probing :
Freezing all the parameters except those in the linear classification layer.
(5) Prompt Tuning [28]: Prepending a sequence of learnable prompt tokens to the input visual patchtokens.
During fine-tuning, only these newly added prompts are upadated.
(6) Attention Pooling Head : Replacing the original temporal average pooling with a temporalattention pooling layer (similar to the one used in [9]) before the classification head.
These approaches above do not incorporate temporal modeling to the image
ViT.
Hence, we furtherconsider temporally augmented ViT architectures as introduced in state-of-the-art video methods:
(a) Spatial Attention Only (SA) : Space-Only TimeSformer [6].
(b) Spatial Attention + Temporal Attention (SA+TA) :
The default TimeSformer [6] with dividedspace-time attention (Fig. 1a).(c)
Spatial Attention + Temporal Shift (SA+TS) : XViT
Note that not all fine-tuning protocols are compatible with each of these video ViT variants.
Take
SA+TS for example, the original model behavior is altered with channel shift, as a result, it is notcompatible with Linear Probing that requires freezing all the parameters of the backbone.
4.2 Main Results and Analysis
Cross-modality fine-tuning benchmark.
Table 1 presents the results of fine-tuning a ViT-B/16pre-trained with CLIP and ImageNet-21K. All baselines are built by combining existing efficientfine-tuning methods with three state-of-the-art ViT-based action recognition models.
From the resultswe can see that:
(i) For CLIP pre-trained model, ST-Adapter performs on par with Full Fine-tuning ( 82.0 vs. 81.7for
K400 and 66.3 vs. 66.1 for SSv2) while updating far less parameters ( 7.2M vs. 121.57M).ST-Adapter significantly outperforms all other efficient fine-tuning methods.
We see that baselineslike Prompt Tuning and Partial Fine-tuning can provide non-trivial gain in performance compared toLinear Probe, but are still behind our ST-Adapter.
(ii) CLIP pre-train models dominate over ImageNet
-21K pre-train ones.
These results well match theshift of paradigm in current AI research [7], where pre-training no longer needs limiting to curateddata and annotations to deliver good performance on downstream tasks, but can take advantage ofbroader scale web raw data.
Interestingly, we observe that SSv2, a motion-centric dataset in design, also benefits from strongerappearance (image) pre-training.
We think this may attribute to that raw textual description canprovide a much richer description ( i.e. , human-object relations) of the image than curated limitedcategorical labels.
Full fine-tuning on SA+TS (XViT) performs slightly worse with CLIP pretrainthan ImageNet-21k pretrain.
We conjecture this is because the channel shift operation breaks theknowledge in the pre-training weights, and thus does not benefit from CLIP pre-training.
Comparison to the state-of-the-art models.
We compare ViT with ST-Adapter to other state-of-thearts methods on both K400 dataset [32] and SSv2 dataset [21] in Table 2 and Table 3 respectively.
We can observe that:
(i) With the proper adaptation method, we can simply turn a large image foundation model into a good
video model by only tuning a few parameters.
Our results are comparable to or better than previous
methods tailored for such tasks.
Our largest model with ViT-L backbone set a new state-of-the-art inK400 by achieving 86.7% top-1 accuracy.
(ii) It is noteworthy that, our method takes significantly fewer frames as input compared to othermethods (8 vs. 16, 32, 64, 96).
It is also reflected in terms of GFlops.
Saying that the ViT was notdesigned for efficiency purposes like [37, 9, 42, 16] but the adapted CLIP ViT has achieved similaraccuracy-efficiency trade-offs.
(iii)
The paradigm of pre-training and fine-tuning has been widely adopted in most state-of-art
methods to achieve good performance.
Between them, most of the approaches start from imagepre-trained models, and only a few can afford video pre-training.
Note
that for the SomethingSomething dataset, except MViT [16] pre-trained on video data from scratch, the rest of methods arestill initialized from image pre-trained weights.
A good image pre-trained model with rich appearanceinformation can facilitate temporal modeling in temporally challenging datasets like SSv2.
[Table] [Table]
[Figure]
4.3 Ablations
Unless otherwise specified, we use ViT-B/16 backbone and 8 input frames in all ablation experiments,and we use one ST-Adapter with bottleneck width 384 before MHSA in each Transformer block.
Where to insert ST-Adapter By default, we insert a ST-Adapter to every Transformer block in thebackbone, but we also show the performance impact of using fewer ST-Adapters.
As shown in Table4b, while more ST-Adapters tend to do better, ST-Adapters at deeper layers boost performance morethan those at shallower layers.
This observation is useful when we insert ST-Adapters into deepermodels and having an Adapter for each block might be too expensive.
We also show the performance
[Table]
when inserting ST-Adapters to different positions within a block.
As shown in Table 4c
[Table]
while theperformance is relatively insensitive to the position of the Adapters
[Table]
using multiple adapters in oneblock may substantially boost performance on some datasets, like SSv2 in our case.
Training parameter efficiency We experiment with a different number of channels in the middle ofthe bottleneck design.
As shown in Table 4a and Fig.
2a, our method is effective with a wide range ofbottleneck width: even with a channel reduction to 64, our ST-Adapters still obtain relatively goodperformance, outperforming all baselines in Table 1 except for Full Fine-tuning (SA + TA).
Even witha bottleneck width of 768, our ST-Adapters are still very parameter efficient, introducing only about1/6 new parameters to a Transformer encoder block.
In contrast to the inverted bottleneck designcommonly used with depthwise convolutions [57], ST-Adapters work best with regular bottlenecks.
The success of transfer learning with such low-rank projections again shows the rich knowledge andstrong potential of modern foundation models.
Training time efficiency In Fig.
2b we show an enlarged difference between full fine-tuned modelsand our ST-Adapters with low training budgets.
When we reduce the number of training steps, theaccuracy of full fine-tuned models drops significantly faster than models with ST-Adapters.
Thisshows the advantage of our proposed modules when backbone models are large or computationalresources are limited.
Training data efficiency Fig.
2c showcases the impact of training data size on action recognitionaccuracy.
Even with the same pre-trained weights, ST-Adapters tend to obtain higher performancethan full fine-tuning especially on smaller datasets: the margin between the two models increaseswith the shrinkage of data.
This shows that ST-Adapters are powerful tools to transfer to downstreamtasks where only a small amount of labeled data is available.
[Table] [Table]
Conclusions
In this work, we have presented a simple yet effective Spatio-Temporal Adapter (ST-Adapter) forenabling a less studied parameter-efficient image-to-video transfer learning.
Fully using commonlyadopt primitive operators, ST-Adapter is particularly designed to be both lightweight and easy toimplement for friendly usability and deployment.
This cross-modality adaptation is a practicallycritical capability considering that it is dramatically challenging and more costly to build a sufficientlystrong large video model in reality.
Encouragingly, extensive experiments on video action recognitionshow that our ST-Adapter can match or surpass both the full fine-tuning strategy as well as fullytrained state-of-the-art video models, whilst having the benefit of (20 times less updated parameters)parameter-efficiency.
Further, our method is also faster to train and consumes less computingresources with economic and environmental superiority.
We believe this work is inspiring for theresearch of other video understanding tasks such as action localization and video summarization.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[No]
(c) Did you discuss any potential negative societal impacts of your work?
[No]
(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs‘ of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] Code will be
provided on GitHub after blind review.
(b) Did you specify all the training details (e
g., data splits, hyperparameters,
how they
were chosen)?
[Yes] See ??
(c) Did you report error bars (e.g., with respect to the random seed after running ex-
periments multiple times)?
[No] The experiments are too expensive to repeat many
times.
(d) Did you include the total amount of compute and the type of resources used (e.g., type
of GPUs, internal cluster, or cloud provider)?
[Yes] See ??
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] All are mentioned in
(b) Did you mention the license of the assets?
[No]
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
Code will be provided on GitHub after blind review.
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[No]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[No]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
[Table]
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]