Look More but Care Less in Video Recognition
Anonymous Author(s)
AffiliationAddress email
Abstract
Existing action recognition methods typically sample a few frames to representeach video to avoid the enormous computation, which often limits the recognitionperformance.
To tackle this problem, we propose Ample and Focal Network(AFNet), which is composed of two branches to utilize more frames but with lesscomputation .
Specifically, the Ample Branch takes all input frames to obtainabundant information with condensed computation and provides the guidance for
Focal Branch by the proposed Navigation Module; the Focal Branch squeezes thetemporal size to only focus on the salient frames at each convolution block; inthe end, the results of two branches are adaptively fused to prevent the loss ofinformation.
With this design, we can introduce more frames to the network butcost less computation.
Besides, we demonstrate AFNet can utilize fewer frameswhile achieving higher accuracy as the dynamic selection in intermediate featuresenforces implicit temporal modeling.
Further, we show that our method can beextended to reduce spatial redundancy with even less cost.
Extensive experimentson five datasets demonstrate the effectiveness and efficiency of our method.
Introduction
Online videos have grown wildly in recent years and video analysis is necessary for many applicationssuch as recommendation [4] and surveillance [3].
These applications require not only accurate butalso efficient video understanding algorithms.
With the introduction of deep learning networks [2] invideo action recognition, there has been rapid progress in the performance of the methods in this area.
Though successful, these deep learning based methods are often at the cost of huge computation,making them hard to be deployed in the real world.
In video recognition, we need to sample a certain number of frames to represent each video whichmakes the computational cost scale proportionally to the number of sampled frames.
In most cases, asmall proportion of all the frames is sampled for each input, which only contains limited informationof the original video.
A straightforward solution is to sample more frames to the network but thecomputation expands proportionally to the number of sampled frames.
There are some works proposed recently to dynamically sample salient frames [27, 15].
The selectionstep of these methods is made before the frames being sent to the classification network, whichmeans the information of those unimportant frames is totally lost and it consumes a considerabletime for the sampling procedure.
Some other methods proposed to address the spatial redundancy inaction recognition by adaptively resizing the resolution based on the importance of each frame [21],or cropping the most salient patch for every frame [26].
However, these methods still completelyabandon the information that the network recognizes as unimportant and introduce a policy network tomake decisions for each sample which leads to extra computation and complicates training strategies.
In our work, we go from another perspective compared with previous works.
We propose a methodwhich makes frame selection within the classification network.
Shown in Figure 1, we design an
[Figure]
Most loss design frames with different computational resources within the network preserves all input architecture called Ample and Focal Network (AFNet) which is composed of two branches: theample branch takes a glimpse of all the input features by lightweight computation as we downsamplethe features and reduce the channel size; the focal branch receives the guidance from the proposednavigation module to squeeze the temporal size by only computing on the selected frames to save cost;in the end, we adaptively fuse the features of these two branches to prevent the loss of information.
With this design, the two branches are both very lightweight and we enable AFNet to look broadlyby sampling more frames and stay focused on the important information for less computation.
Considering these two branches in a uniform manner, on the one hand, we can avoid the loss ofinformation compared to other dynamic methods as the ample branch preserves the information of allthe input; on the other hand, we can restrain the noise from the unimportant frames by deactivatingthem in each convolutional block.
Further, we have demonstrated that the dynamic selection strategyin intermediate features is beneficial for temporal modeling as it implicitly implements frame-wiseattention which can enable our network to use fewer frames while achieving higher accuracy.
Inaddition, instead of introducing a policy network to select frames, we design a lightweight navigationmodule which can be easily plugged into the network so that our method can be trained in an end-toend fashion.
It is worth noting that AFNet is compatible with spatial adaptive works which can helpto further reduce the computations of our method.
The main contributions are summarized as follows:
• We propose an adaptive two-branch framework which enables 2D-CNNs to process more frames with less computational cost .
With this design, we not only prevent the loss of information but strengthen the representation of essential frames.
• We propose a lightweight navigation module to dynamically select salient frames at each convolution block which can be easily trained in an end-to-end way.•
The selection strategy at intermediate features not only empowers the model with strongflexibility as different frames will be selected at different layers, but also enforces implicittemporal modeling which enables AFNet to obtain higher accuracy with fewer frames .
• We have conducted extensive experiments on five video recognition benchmarks.
The resultsshow the superiority of AFNet compared to state-of-the-art methods .
Related Work
2.1 Video Recognition
The development of deep learning in recent years serves as a huge boost to the research of videorecognition.
A straightforward method for this task is using 2D-CNNs to extract the features ofsampled frames and use specific aggregation methods to model the temporal relationships acrossframes.
For instance, [25] proposes to average the temporal information between frames.
While [18]shifts channels along the temporal dimension to exchange the information among neighboring frames.
Another approach is to use 3D-CNNs to model the temporal and spatial information jointly, such as
C3D [24], I3D [2] and ResNet3D [10].
Though being shown effective, methods based on 3D-CNNsare computationally expensive, which brings great difficulty in real-world deployment.
While the two-branch design has been explored by SlowFast [6], but the motivation and detailedstructure are different in the following ways: (1) network category: SlowFast is a static 3D model,but AFNet is a dynamic 2D network; (2) motivation: SlowFast is designed to capture semanticinformation and changing motion with branches at different temporal speed for better performance,while AFNet is aimed to dynamically skip frames to save computation and the design of two-branchstructure is to prevent the information loss; (3) specific design: AFNet is designed to downsample features for efficiency at ample branch while SlowFast processes features in the original resolution;(4) temporal modeling: SlowFast applies 3D convolutions for temporal modeling, AFNet is a 2Dmodel which is enforced with implicit temporal modeling by the designed navigation module.
2.2 Redundancy in Data
The efficiency of 2D-CNNs has been broadly studied in recent years.
While some of the works aimat designing efficient network structure [12], there is another line of research focusing on reducingthe intrinsic redundancy in image-based data [29, 9].
In video recognition, people usually samplelimited number of frames to represent each video to prevent numerous computational costs.
Eventhough, the computation for video recognition is still a heavy burden for researchers and a commonstrategy to address this problem is reducing the temporal redundancy in videos as not all frames areessential to the final prediction.
[30] proposes to use reinforcement learning to skip frames for actiondetection.
There are other works [27, 15] dynamically sampling salient frames to save computationalcost.
As spatial redundancy widely exists in image-based data, [21] adaptively processes frames withdifferent resolutions.
[26] provides the solution as cropping the most salient patch for each frame.
However, the unselected regions or frames of these works are completely abandoned.
Hence, therewill be some information lost in their designed procedures.
Moreover, most of these works adopt apolicy network to make dynamic decisions, which introduce additional computation somehow andcannot be trained in an end-to-end fashion.
In contrast, our method adopts a two-branch design,allocating different computational resources based on the importance of each frame and preventingthe loss of information.
Besides, we design a lightweight navigation module to guide the networkwhere to look, which can be incorporated to the backbone network and trained in an end-to-end way.
Moreover, we demonstrate that the dynamic frame selection at intermediate features will not onlyempower the model with strong flexibility as different frames will be selected at different layers, but result in learned frame-wise weights which enforce implicit temporal modeling.
Methodology
Intuitively, considering more frames enhances the temporal modeling but results in higher computational cost.
To efficiently achieve the competitive performance, we propose AFNet to involve moreframes but wisely extract information from them to keep the low computational cost.
Specifically,we design a two-branch structure to treat frames differently based on their importance and processthe features in an adaptive manner which can provide our method with strong flexibility.
Besides,we demonstrate that the dynamic selection of frames in the intermediate features results in learnedframe-wise weights which can be regarded as implicit temporal modeling.
3.1 Architecture Design
As is shown in Figure 2, we design our Ample and Focal (AF) module as a two-branch structure:the ample branch (top) processes abundant features of all the frames in a lower resolution and asqueezed channel size; the focal branch (bottom) receives the guidance from ample branch generatedby the navigation module and makes computation only on the selected frames.
Such design can beconveniently plugged into existing CNN architectures to build our network.
Ample Branch.
The ample branch is designed to involve all frames with cheap computation, whichserves as 1) a guidance to select salient frames to help focal branch to concentrate on importantinformation; 2) a complementary stream with focal branch to prevent the information loss via aspecially designed fusion strategy.
Formally, we denote video sample i as v i , containing T frames as v i = (cid:8) f i 1 , f i 2 , ..., f iT (cid:9) .
Forconvenience, we omit the superscript i in the following sections if no confusion arises.
We denote theinput of ample branch as v x ∈ R T × C × H × W , where C represents the channel size and H × W is theresolution.
The features generated by the ample branch can be written as:
[Equation]
where v y a ∈ R T × ( C o / 2) × ( H o / 2) × ( W o / 2) represents the output of ample branch with channel, height,width at focal branch denoted as C o ,
H o , W o , respectively, and F a stands for a series of convolutionblocks.
The stride of the first convolution block is set to 2 to downsample the resolution of this branch
NavigationModule
Vanilla Convolution
Sparse Convolution Residual Connection … H
W C
T frames …
H
W C n …
C 0 /2 W/
H/
C n /2 W/
H/ … H
W C
NavigationModule … … …
L 0 Temporal Mask
L n Temporal Mask
Ample Branch
Focal Branch Multiplication Addition 𝜃 1−𝜃
Downsample
Upsample
DynamicFusion
Navigation Module.
The proposed navigation module is designed to guide the focal branch where tolook by adaptively selecting the most salient frames for video v i .
Specifically, the navigation module generates a binary temporal mask
L n based on the output fromthe n -th convolution block in ample branch v y an .
At first, average pooling is applied to v y an to resizethe spatial dimension to 1 × 1 , then convolution is performed to transform channel size to 2:
[Equation]
where ∗ stands for convolution and W 1 denotes the weights of the 1 × 1 convolution.
After that, wereshape the dimension of feature ˜ v y an from T × 2 × 1 × 1 to 1 × (2 × T ) × 1 × 1 so that we canmodel the temporal relations for each video from channel dimension by:
[Equation]
where W 2 represents the weights of the second 1 × 1 convolution and it will generate a logit p tn ∈ R 2for each frame t in the output frame map which denotes whether to select it.
However, directly sampling from such discrete distribution is non-differentiable.
In this work,we apply Gumbel-Softmax [13] to resolve this non-differentiability.
Specifically, we generate anormalized categorical distribution by using Softmax:
[Equation]
and we draw discrete samples from the distribution π as:
[Equation]
where G j = − log( − log U j ) is a standard Gumbel distribution with U j sampled from a uniformdistribution Unif(0,1).
As arg max cannot be differentiated, we relax the discrete sample L inbackpropagation via Softmax:
[Equation]
the distribution ˆ l will become a one-hot vector when the temperature factor τ → 0 and we let τdecrease from 1 to 0.01 during training.
Focal Branch.
The focal branch is guided by the navigation module to only consider the selectedframes, which diminishes the computational cost and potential noise from redundant frames.
The features at the n -th convolution block in this branch can be denoted as v y fn ∈ R T × C o × H
o × W o .
Based on the temporal mask L n from the navigation module, we select frames which have corresponding non-zero values in the binary mask for each video and apply convolutional operations onlyon these extracted frames v ′ y fn ∈ R T
l × C o × H
o × W o :
[Equation]
where F fn is the n -th convolution blocks at this branch and we set the group number of convolutionsto 2 in order to further reduce the computations.
After the convolution operation at n -th block, wegenerate a zero-tensor which shares the same shape with v y fn and fill the value by adding v ′ y fn andv y fn − 1 with the residual design following [11].
At the end of these two branches, we generate a weighting factor θ by pooling and linear layers tofuse the features from two branches:
[Equation]
where ⊙ denotes the channel-wise multiplication.
3.2 Implicit Temporal Modeling
While our work is mainly designed to reduce the computational cost in action recognition like [26, 22],we demonstrate that AFNet enforces implicit temporal modeling by the dynamic selection of frames inthe intermediate features.
Considering a TSN[25] network which adapts vanilla ResNet[11] structure,the feature at the n -th convolutional block in each stage can be written as v n ∈ R T × C × H × W .
Thus,the feature at n + 1 -th block can be represented as:
[Equation]
[Equation]
where F n +1 is the n + 1 -th convolutional block and we define ∆ v n +1 as the coefficient learned fromthis block.
By that we can write the output of this stage v N as:
[Equation]
[Equation]
Similarly, we define the features in ample and focal branch as:
[Equation] [Equation]
where L n is the binary temporal mask generated by Equation 5 and v y 1 denotes the input of thisstage.
Based on Equation 8, we can get the output of this stage as:
[Equation]
As L n is a temporal-wise binary mask, it will decide whether the coefficient ∆ v y fn will be calculatedin each frame at every convolutional block.
Considering the whole stage is made up of multipleconvolutional blocks, the series multiplication of focal branch’s output with the binary mask L n willapproximate soft weights. This results in learned frame-wise weights in each video which we regardas implicit temporal modeling.
Although we do not explicitly build any temporal modeling module,the generation of L n in Equation 3 has already taken the temporal information into account so thatthe learned temporal weights equal performing implicit temporal modeling at each stage.
3.3 Spatial Redundancy Reduction
In this part, we show that our approach is compatible with methods that aim to solve the problemof spatial redundancy.
We extend the navigation module by applying similar procedures with the
[Figure]
W
H
W
[Figure]
temporal mask generation and the work [9] to generate a spatial logit for the n -th convolution blockwhich is shown in Figure 3:
[Equation]
where W 3 denotes the weights of the 3 × 3 convolution and W 4 stands for the weights of convolutionwith kernel size 1 × 1 .
After that, we still use Gumbel-Softmax to sample from such discretedistribution to generate spatial mask M n and guide the focal branch to merely focus on the salientregions of the selected frames to further reduce the cost.
3.4 Loss functions
Inspired by [25], we average the predictions of each frame to represent the final prediction of thecorresponding video and our optimization objective is minimizing:
[Equation]
The first term is the cross-entropy between predictions P ( v ) for input video v and the correspondingone-hot label y .
We denote r in the second term as the ratio of selected frames in each batch and
RT as the target ratio which is set before the training( RS is the target ratio when applying spatialselection module).
We let r approximate RT by adding the second loss term and control the trade-offbetween accuracy and efficiency by introducing a factor λ which balances these two terms.
Empirical Validation
In this section, we conduct extensive experiments to validate our method.
We first compare ourmethod with plain 2D CNNs to demonstrate that our AF module implements implicit temporalwise attention which is beneficial for temporal modeling.
Then, we validate AFNet’s efficiency byintroducing more frames but costing less computation compared with other methods.
Further, we show
AFNet’s strong performance compared with other efficient action recognition frameworks.
Finally,we provide comprehensive ablation results and qualitative analysis to demonstrate the effectivenessof our proposed navigation module and two-branch design.
Datasets.
Our method is evaluated on five widely-used video datasets: (1) ActivityNet-v1.3 [1] islabeled with 200 action categories.
It contains 10,024 training videos and 4,926 validation videosin total and the average duration is 117 seconds; (2) Jester is a hand gesture recognition datasetintroduced by [20].
The dataset consists of 27 classes, with 119k videos for training and 15k videosfor validation; (3) Something-Something V1 & V2 [8] are two human action datasets with strongtemporal information, including 98k and 194k videos for training and validation respectively; (4)
Mini-Kinetics [21, 22] is a subset of Kinetics [14] which selects 200 classes from Kinetics, with 121kvideos for training and 10k videos for validation.
Data pre-processing.
We uniformly sample 8 frames for each video on Jester, Mini-Kinetics, and 12frames for ActivityNet and Something-Something to compare with existing works.
During training,the training data is randomly cropped to 224 × 224 following [32], and we perform random flippingexcept for Something-Something.
At inference stage, we use one clip per video for efficiency and allframes are center-cropped to 224 × 224.
Implementation details.
Our method is bulit on ResNet-50 [11] and we replace the first three stagesof the network with our proposed AF module.
We first train our two-branch network from scratch on
ImageNet for fair comparisons with other methods, then add the proposed navigation module andtrain it along with the backbone network on video datasets.
In our implementations, RT denotes theratio of selected frames while RS represents the ratio of select regions.
Due to limited space, moreimplementation details can be found in supplementary material.
4.1 Comparisons with Existing Methods
Baselines.
In this part, we first compare AFNet with TSN [25] on Jester and Something-Something
V1 to demonstrate that our AFNet can achieve Less is More by using less frames while obtaininghigher accuracy.
Further, we build our framework on TSM [18] and compare with other worksto validate that AFNet can achieve More is Less by seeing more frames with less computations.
Then we empirically compare AFNet with competing efficient video recognition methods, including
AdaFrame [27], LiteEval [28], ListenToLook [7], SCSampler [15], AR-Net [21], VideoIQ [23],
AdaFoucs [26] on the dataset of ActivityNet and Mini-Kinetics.
[Table]
Less is more.
At first, we implement AFNet on
Something-Something V1 and Jester datasets. We compare it with the baseline method TSN asboth methods do not explicitly build temporalmodeling module.
In Table 1, our method showssimilar performance with TSN when selectingall the frames.
Nonetheless, when we selectfewer frames in AFNet, it exhibits much higheraccuracy compared to TSN.
The results mayseem counterintuitive as seeing more frames isusually beneficial for video recognition.
The explanation is that the two-branch design of AFNetcan preserve the information of all input frames and the intermediate selection of salient framesimplements implicit temporal modeling compared to TSN which simply averages the logits of allthe frames at the end of the network.
The results also indicate that the navigation module not onlyreduces the computational cost but strengthens the representation of salient frames as well.
Whenwe decrease the ratio of selected frames from 0.5 to 0.25, there is no accuracy drop on Jester andthe accuracy even increases by 0.9 % on Something-Something V1, which further demonstrates theeffectiveness of our navigation module.
[Table]
More is less.
In this segment, we incorporate our method with temporal shift module (TSM [18])to validate that AFNet can further reduce the redundancy of such competing methods and achieve
More is Less by seeing more frames with less computation.
We implement our method on thedatasets of Something-Something V1 & V2 and Jester which contain strong temporal informationand relevant results are shown in Table 2.
Compared to TSM which samples 8 frames, our methodshows significant advantages in performance as we introduce more frames.
Yet, our computationcost is much less than TSM as we allocate frames with different computation resources by thistwo-branch design.
Compared to other competitive dynamic methods like AdaFuse and AdaFocus,our method shows an obviously better performance both in accuracy and computations.
When costingsimilar computation, AFNet outperforms AdaFuse and AdaFocus by 3.1 % and 1.8 % respectively on
Something-Something v1.
Moreover, we compare our method with TSM when both sampling 12frames and AFNet still outperforms TSM which achieves Less is More on the TSM structure as thenavigation module can restrain the noise of unimportant frames and enforces frame-wise attentionwhich can be regarded as implicit temporal modeling.
AFNet achieves the best trade-off compared to existing works
GFLOPs refers
[Table]
achieves the best trade off
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
0 . 8 , 1 . 0 } and a fixed ratio of selected regions RS ∈ { 0 . 4 } .
Bottleneck Block Index
[Figure] [Figure]
[Figure] [Figure]
R T
[Figure] [Figure]
Ratio 0.
Comparisons with state-of-the-art methods.
Then, we implement our method on the dataset of MiniKinetics, ActivityNet-v1.3, and compare AFNet with other efficient video recognition approaches.
At first, we evaluate our method on Mini-Kinetics and our method shows the best performance bothin accuracy and computations compared with other efficient methods.
To demonstrate that AFNetcan further reduce spatial redundancy, we extend AFNet by adding the spatial selection module on
ActivityNet.
We move the navigation module to the first layer of the network to avoid huge variancein features when incorporating spatial selection module and note that we only apply this procedurein this part.
As shown in Table 4, our method achieves the best performance while costing the leastcomputation compared to other works.
Moreover, we change the ratio of selected frames and plot themean Average Precision and computational cost of various methods in Figure 4.
One can observethat AFNet exhibits a better trade-off between accuracy and efficiency compared to other works.
4.2 Visualizations and Ablation Study
In this part, we implement our method on ActivityNet with 12 sampled frames to analyze the resultsand conduct extensive ablation study to prove the effectiveness of our design.
Visualization results.
We show the distribution of RT among different convolution blocks underdifferent selection ratios in Figure 5 and use 3rd-order polynomials to estimate the trend of distribution(shown in dash lines).
One can see a decreased trend in RT for all the curves with the increasedindex in convolution blocks and this can be explained that earlier layers mostly capture low-levelinformation which has relatively large divergence among different frames.
While high-level semanticsbetween different frames are more similar, therefore AFNet tends to skip more at later convolutionblocks.
In Figure 6, we visualize the frames selected by the 3rd-block of our AFNet with RT =0.5 on
[Figure]
[Table]
the validation set of Something-Something V1 where we uniformly sample 8 frames.
Our navigationmodule effectively guides the focal branch to concentrate on frames which are more task-relevantand deactivate the frames that contain similar information.
Effect of two branch design.
We further incorporate our navigation module into ResNet50 andcompare it with AFNet to prove the strength of our designed two-branch architecture.
From Table 5,
AFNet shows substantial advantages in accuracy under different ratios of select frames.
Aside fromit, models which adopted our structure but with a fixed sampling policy also show significantly betterperformance compared with the network based on single branch which can further demonstrate theeffectiveness of our two-branch structure and the necessity to preserve the information of all frames.
Effect of navigation module.
In this part, we compare our proposed navigation module withthree alternative sampling strategies in different selection ratios: (1) random sampling; (2) uniformsampling: sample frames in equal step; (3) normal sampling: sample frames from a standard gaussiandistribution.
As shown in Table 5, our proposed strategy consistently outperforms other fixed samplingpolicies under different selection ratios which validates the effectiveness of the navigation module.
Moreover, the advantage of our method is more obvious when the ratio of selected frames is smallwhich demonstrates that our selected frames are more task-relevant and contain essential informationfor the recognition.
Further, we evaluate the extension of navigation module which can reduce spatialredundancy, and compare it with: (1) random sampling; (2) center cropping.
Our method showsbetter performance compared with fixed sampling strategies under various selection ratios.
Conclusion
This paper proposes an adaptive Ample and Focal Network (AFNet) to reduce temporal redundancyin videos with the consideration of architecture design and the intrinsic redundancy in data.
Ourmethod enables 2D-CNNs to have access to more frames to look broadly but with less computationby staying focused on the salient information.
AFNet exhibits promising performance as our twobranch design preserves the information of all the input frames instead of discarding part of theknowledge at the beginning of the network.
Moreover, the dynamic temporal selection within thenetwork not only restrains the noise of unimportant frames but enforces implicit temporal modelingas well.
This enables AFNet to obtain higher accuracy when using fewer frames compared with staticmethods.
We further show that our method can be extended to reduce spatial redundancy by onlycomputing important regions of the selected frames.
Extensive experiments have shown that ourmethod outperforms existing works both in accuracy and computational efficiency.
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information onhow to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .
You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes] See Section ?? .•
Did you include the license to the code and datasets?
[No] The code and the data areproprietary.•
Did you include the license to the code and datasets?
[N/A]
Please do not modify the questions and only use the provided macros for your answers.
Note that the
Checklist section does not count towards the page limit.
In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
[Yes](b) Did you describe the limitations of your work?
[Yes] See supplementary material.(c)
Did you discuss any potential negative societal impacts of your work?
[Yes] Seesupplementary material.(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]2.
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A](b)
Did you include complete proofs of all theoretical results?
[N/A]3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] See Section 4.(b)
Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
[Yes] See Section 4.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[No]
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] See Section 4.4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] See Section 4.
(b) Did you mention the license of the assets?
[Yes] See Section 4.
(c) Did you include any new assets either in the supplemental material or as a URL?
[No]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[No]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[No]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]