Original: Implicit Warping for Animation with Image Sets
Revised: Implicit Warping for Animation with Image Sets

Original: Anonymous Author(s)

Original: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: We present a new implicit warping framework for image animation using sets ofsource images through the transfer of the motion of a driving video.
Revised: We present a new implicit warping framework for image animation using sets of source images through the transfer of the motion of a driving video.

Original: A single crossmodal attention layer is used to find correspondences between the source imagesand the driving image, choose the most appropriate features from different sourceimages and warp the selected features.
Revised: A single crossmodal attention layer is used to ﬁnd correspondences between the source images and the driving image, choose the most appropriate features from different source images and warp the selected features.

Original: This is in contrast to the existing methodsthat use explicit flow-based warping, which is designed for animation using asingle source and does not extend well to multiple sources.
Revised: This is in contrast to the existing methods that use explicit ﬂow-based warping, which is designed for animation using a single source and does not extend well to multiple sources.

Original: The pick-and-choosecapability of our framework helps it achieve state-of-the-art results on multipledatasets for image animation using both single and multiple source images.
Revised: The pick-and-choose capability of our framework helps it achieve state-of-the-art results on multiple datasets for image animation using both single and multiple source images

Original:  Introduction
Revised:  Introduction

Original: We study the task of generating videos by animating a set of source images of the same subject usingthe motions of a driving video possibly containing a different subject.
Revised: We study the task of generating videos by animating a set of source images of the same subject using the motions of a driving video possibly containing a different subject.

Original: This is more general than thesetting of animating a single source image studied in the prior works [26, 27, 41, 50] as it is a specialcase when the set contains just one image.
Revised: This is more general than the setting of animating a single source image studied in the prior works [26, 27, 41, 50] as it is a special case when the set contains just one image.

Original: Moreover, having the capability of animating a set ofimages, especially those representing different views when available, is of great practical value.
Revised: Moreover, having the capability of animating a set of images, especially those representing different views when available, is of great practical value.

Original: Asingle image often cannot fully describe the subject due to occlusions, limited pose information, etc .
Revised: A single image often cannot fully describe the subject due to occlusions, limited pose information, etc .

Original: Diverse source images provide more appearance information and reduce the burden of hallucinationthat an image generator has to perform.
Revised: Diverse source images provide more appearance information and reduce the burden of hallucination that an image generator has to perform.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 1, multiple source images provide morecomplete information, such as the color of the eyes, the texture of the background, etc .
Revised: 1, multiple source images provide more complete information, such as the color of the eyes, the texture of the background, etc .

Original: This allowsfor potentially generating an output image that is more faithful to the source setting.
Revised: This allows for potentially generating an output image that is more faithful to the source setting.

Original: The single-source-based prior works [26, 27, 41, 50] ubiquitously rely on explicit flow-based warpingof the source image conditional on the pose of the driving image.
Revised: The single-source-based prior works [26, 27, 41, 50] ubiquitously rely on explicit ﬂow-based warping of the source image conditional on the pose of the driving image.

Original: Due to this architectural choice,they often have to be modified in ad-hoc ways to take advantage of multiple source images.
Revised: Due to this architectural choice, they often have to be modiﬁed in ad-hoc ways to take advantage of multiple source images.

Original: Onescheme is to train an additional pre-processing network to select the most appropriate source imagefor the given driving image.
Revised: One scheme is to train an additional pre-processing network to select the most appropriate source image for the given driving image.

Original: This would, however, not allow for the use of features from multiplesource images at a time.
Revised: This would, however, not allow for the use of features from multiple source images at a time.

Original: The other possibility is to warp each source image to the driving pose andthen average the now-aligned warped features for the generator input.
Revised: The other possibility is to warp each source image to the driving pose and then average the now-aligned warped features for the generator input.

Original: But as is visible in Fig.
Revised: But as is visible in Fig.

Original: 1 andlater measured in Section 4, this leads to sub-optimal results due to the misalignment of warpedfeatures and inconsistent predictions across views.
Revised: 1 and later measured in Section 4, this leads to sub-optimal results due to the misalignment of warped features and inconsistent predictions across views.

Original: In this work, we introduce a novel implicit warping mechanism that overcomes the above drawbacks—(1) It operates on a set of images and naturally scales from a single source image to multiple sourceimages; and (2) Without predicting explicit per-source flow, a single layer finds correspondencesbetween the source images and the driving image, chooses the aptest features from different sourceimages, and performs warping of the selected features. 
Revised: In this work, we introduce a novel implicit warping mechanism that overcomes the above drawbacks— (1) It operates on a set of images and naturally scales from a single source image to multiple source images; and (2) Without predicting explicit per-source ﬂow, a single layer ﬁnds correspondences between the source images and the driving image, chooses the aptest features from different source images, and performs warping of the selected features. Our novel layer and the ability to pick-andchoose appropriate features from a set of source images allows our method to achieve state-of-the-art image animation quality in the multiple source image setting, as well as the single source image setting. The beneﬁts of our framework are obvious in Fig.

Original: Our novel layer and  the ability to pick-andchoose appropriate features from a set of source images allows our method to achieve state-of-the-art
Revised: 1, in the image reconstruction and cross-identity motion transfer tasks. In the top row, the inability of prior works of FOMM [26] and face-vid2vid [41] to pick-and-choose features results in an incorrect output eye color. Only our on Systems (

Original: Sources
Revised: Sources

Original: Driving video
Revised: Driving video

Original: FOMM [26] fv2v [41]
Revised: FOMM [26] fv2v [41]

Original: [Figure]
Revised: [Figure]

Original: image animation quality in the multiple source image setting, as well as the source image setting.

Original: The benefits of our framework are obvious in Fig.

Original: 1, in the reconstruction

Original: Only

Original:  Related Work
Revised:  Related Work

Original: Image animation.
Revised: Image animation.

Original: Recent works on image animation can be roughly categorized into subjectdependent and subject-agnostic works.
Revised: Recent works on image animation can be roughly categorized into subjectdependent and subject-agnostic works.

Original: For subject-dependent, the framework is trained on a specificsubject and can only animate that specific person [2, 8, 30–33, 36, 46].
Revised: For subject-dependent, the framework is trained on a speciﬁc subject and can only animate that speciﬁc person [2, 8, 30–33, 36, 46].

Original: For example, NerFACE [8]trains a neural radiance field to model a target face using a short clip of the subject.
Revised: For example, NerFACE [8] trains a neural radiance ﬁeld to model a target face using a short clip of the subject.

Original: On the other hand,subject-agnostic frameworks only need a single 2D image to perform the animation on the targetperson, so they are usually more general and applicable [1, 3, 5, 6, 9–11, 16, 21, 23, 26, 27, 29, 37,40, 41, 44, 50, 51, 54].
Revised: On the other hand, subject-agnostic frameworks only need a single 2D image to perform the animation on the target person, so they are usually more general and applicable [1, 3, 5, 6, 9–11, 16, 21, 23, 26, 27, 29, 37, 40, 41, 44, 50, 51, 54].

Original: For example, Siarohin et al .
Revised: For example, Siarohin et al .

Original: [26] predict local affine transformations usinglearned latent keypoints.
Revised: [26] predict local afﬁne transformations using learned latent keypoints.

Original: Zakharov et al .
Revised: Zakharov et al .

Original: [50] focus on inference speed and propose to decomposehigh and low-frequency components for better efficiency.
Revised: [50] focus on inference speed and propose to decompose high and low-frequency components for better efﬁciency.

Original: Wang et al .
Revised: Wang et al .

Original: [41] learn 3D latent keypointsand their decomposition to predict warping flows and gain better controllability.
Revised: [41] learn 3D latent keypoints and their decomposition to predict warping ﬂows and gain better controllability.

Original: Doukas et al .
Revised: Doukas et al .

Original: [6] adopt 3D morphable models to generate a dense flow field to warp the source image and then takethe warped input along with audio features to synthesize the final output.
Revised: [6] adopt 3D morphable models to generate a dense ﬂow ﬁeld to warp the source image and then take the warped input along with audio features to synthesize the ﬁnal output.

Original: While these frameworksachieve promising results in image animation, none of them are designed for using multiple referencesource images as input.
Revised: While these frameworks achieve promising results in image animation, none of them are designed for using multiple reference source images as input.

Original: In contrast, our framework is designed to take advantage of complementaryinformation available in different input images for high-quality image animation.
Revised: In contrast, our framework is designed to take advantage of complementary information available in different input images for high-quality image animation.

Original: Attention in computer vision.
Revised: Attention in computer vision.

Original: The self-attention layer, which uses the same input as the query, key,and value, was popularized by the work of Vaswani et al .
Revised: The self-attention layer, which uses the same input as the query, key, and value, was popularized by the work of Vaswani et al .

Original: [35] for the task of machine translation.
Revised: [35] for the task of machine translation.

Original: Since then, a number of works have explored using the self-attention layer in conjunction withconvolutional layers for the tasks of image and video recognition [13, 14, 38, 39, 43, 45, 47].
Revised: Since then, a number of works have explored using the self-attention layer in conjunction with convolutional layers for the tasks of image and video recognition [13, 14, 38, 39, 43, 45, 47].

Original: Afew works have also explored cross-modal attention in which the queries, keys, and values arecomputed from two different domains, such as vision and text [48, 49], audio and text [19], etc .
Revised: A few works have also explored cross-modal attention in which the queries, keys, and values are computed from two different domains, such as vision and text [48, 49], audio and text [19], etc .

Original: In our cross-modal attention layer, the queries and keys are from the domain of a compact imagerepresentation, such as keypoints, while the values are extracted from the dense image features.
Revised: In our cross-modal attention layer, the queries and keys are from the domain of a compact image representation, such as keypoints, while the values are extracted from the dense image features.

Original: Networks fully composed of self-attention layers, also known as transformers, have recently obtainedstate-of-the-art numbers in the tasks of image classification [18, 20, 34, 42], as well as generation [7,15, 17, 22, 24, 25].
Revised: Networks fully composed of self-attention layers, also known as transformers, have recently obtained state-of-the-art numbers in the tasks of image classiﬁcation [18, 20, 34, 42], as well as generation [7, 15, 17, 22, 24, 25].

Original: Our generator uses a hybrid architecture, with a single cross-modal attention layerfor warping the features of the source image conditional on the pose of the driving image.
Revised: Our generator uses a hybrid architecture, with a single cross-modal attention layer for warping the features of the source image conditional on the pose of the driving image.

Original: This isfollowed by a decoder made up of convolutional residual blocks to produce an output image.
Revised: This is followed by a decoder made up of convolutional residual blocks to produce an output image.

Original: While we do not predict explicit flows to warp feature maps, related work by Xu et al .
Revised: While we do not predict explicit ﬂows to warp feature maps, related work by Xu et al .

Original: [47] has usedefficient 1-D attention layers for predicting explicit optical flow.
Revised: [47] has used efﬁcient 1-D attention layers for predicting explicit optical ﬂow.

Original: We use global 2-D attention, butsuch factored 1-D attention layers and spatial-reduction attention layers [42] can be used to reducethe run-time further.
Revised: We use global 2-D attention, but such factored 1-D attention layers and spatial-reduction attention layers [42] can be used to reduce the run-time further.

Original: To the best of our knowledge, we are the first to propose a way of findingcorrespondences and warping features via a single cross-modal attention layer for image animation.
Revised: To the best of our knowledge, we are the ﬁrst to propose a way of ﬁnding correspondences and warping features via a single cross-modal attention layer for image animation.

Original:  Image Animation with Implicit Warping
Revised:  Image Animation with Implicit Warping

Original: The idea behind image animation using a source image is to warp features of the source imageconditional on the given driving image so as to reconstruct the driving image or transfer the poseof the driving image in the case of cross-identity motion transfer.
Revised: The idea behind image animation using a source image is to warp features of the source image conditional on the given driving image so as to reconstruct the driving image or transfer the pose of the driving image in the case of cross-identity motion transfer.

Original: Such image animation methodstypically consist of 4 components: (1) Source image representation extraction, (2) Driving imagerepresentation extraction, (3) Source image feature warping, and (4) Warped image feature decoding.
Revised: Such image animation methods typically consist of 4 components: (1) Source image representation extraction, (2) Driving image representation extraction, (3) Source image feature warping, and (4) Warped image feature decoding.

Original: In this work, we focus on the third component—source image feature warping.
Revised: In this work, we focus on the third component—source image feature warping.

Original: The other componentsin our method are largely based on prior works [26, 27, 41].
Revised: The other components in our method are largely based on prior works [26, 27, 41].

Original: In the following sections, we describeour framework and contributions in more detail.
Revised: In the following sections, we describe our framework and contributions in more detail.

Original: Explicit flow-based warping.
Revised: Explicit ﬂow-based warping.

Original: Prior works on image animation, such as Monkey-Net [27], First
Revised: Prior works on image animation, such as Monkey-Net [27], First

Original: Order Motion Model (FOMM)
Revised: Order Motion Model (FOMM)

Original: [26], and face-vid2vid [41], use a flow-based warping technique todeform the features extracted from the source image.
Revised: [26], and face-vid2vid [41], use a ﬂow-based warping technique to deform the features extracted from the source image.

Original: They generate a warping field using the compactrepresentations extracted from the source and driving images, usually in the form of keypoints andadditional related information such as Jacobian matrices.
Revised: They generate a warping ﬁeld using the compact representations extracted from the source and driving images, usually in the form of keypoints and additional related information such as Jacobian matrices.

Original: This field is used to warp the source imagefeatures, which are subsequently mapped to the output image space by a decoder network.
Revised: This ﬁeld is used to warp the source image features, which are subsequently mapped to the output image space by a decoder network.

Original: The priorworks, specifically FOMM [26] and face-vid2vid [41], perform warping using the scheme illustratedin Fig.
Revised: The prior works, speciﬁcally FOMM [26] and face-vid2vid [41], perform warping using the scheme illustrated in Fig.

Original: 2a.
Revised: 2a.

Original: Given the locations of K unsupervisedly-learned keypoints in the source image S anddriving image D , they first generate K per-keypoint flows w 1 , ¨ ¨ ¨ , w K .
Revised: Given the locations of K unsupervisedly-learned keypoints in the source image S and driving image D , they ﬁrst generate K per-keypoint ﬂows w 1 , ¨ ¨ ¨ , w K .

Original: By using the source image(or extracted features) warped with each of the K flows, a motion estimator network predicts how tocombine the individual flows in the flow composition mask m .
Revised: By using the source image (or extracted features) warped with each of the K ﬂows, a motion estimator network predicts how to combine the individual ﬂows in the ﬂow composition mask m .

Original: Combining the K per-keypoint flowswith this mask gives the final composite flow mask w , which is applied to the source image featuresf S to obtain the warped feature w p f S q .
Revised: Combining the K per-keypoint ﬂows with this mask gives the ﬁnal composite ﬂow mask w , which is applied to the source image features f S to obtain the warped feature w p f S q .

Original: Due to the per-keypoint warping steps, the time complexityof these methods is directly proportional to the number of keypoints K .
Revised: Due to the per-keypoint warping steps, the time complexity of these methods is directly proportional to the number of keypoints K .

Original: As mentioned in Section 1, the prior works have significant drawbacks when extending them tomultiple source images.
Revised: As mentioned in Section 1, the prior works have signiﬁcant drawbacks when extending them to multiple source images.

Original: We either have to average the per-source image warped features or employa heuristic to choose the most appropriate source image given the driving image.Neither of these [Figure] [Figure]
Revised: We either have to average the per-source image warped features or employ a heuristic to choose the most appropriate source image given the driving image.

Original: [Figure]

Original: [Figure]

Original: [Figure]

Original: [Figure] [Figure] [Figure] two solutions is ideal as the former can lead to blurry outputs and the latter leads to flickering whenswitching between sources.
Revised: Neither of these two solutions is ideal as the former can lead to blurry outputs and the latter leads to ﬂickering when switching between sources.

Original: A more desirable way is to have an end-to-end trainable pick-and-mixfeature combination mechanism that can aggregate information from multiple source images.
Revised: A more desirable way is to have an end-to-end trainable pick-and-mix feature combination mechanism that can aggregate information from multiple source images.

Original: In this work, we introduce ‘ Implicit Warping ’—a new and simpler way to warp source features.
Revised: In this work, we introduce ‘ Implicit Warping ’—a new and simpler way to warp source features.

Original: Thisresults in an image animation method that is easily and directly extensible to using multiple source images.
Revised: This results in an image animation method that is easily and directly extensible to using multiple source images.

Original: Our method relies on a cross-modal attention layer illustrated in Fig.
Revised: Our method relies on a cross-modal attention layer illustrated in Fig.

Original: 2b.
Revised: 2b.

Original: It neither producesexplicit per-keypoint warping flows nor requires per-source image warping of features.
Revised: It neither produces explicit per-keypoint warping ﬂows nor requires per-source image warping of features.

Original: Implicit Warping using attention.
Revised: Implicit Warping using attention.

Original: The process of warping the source image features conditional onthe driving image consists of two parts: (1) Identifying dense correspondences between thesourceand driving images, and (2) Warping the source image features based on the dense correspondences.
Revised: The process of warping the source image features conditional on the driving image consists of two parts: (1) Identifying dense correspondences between the source [Figure] [Figure] [Figure]

Original: The key intuition behind our method is that these two steps can be represented using a single scaleddot-product attention layer popularized by Vaswani et al .
Revised: [Figure] [Figure] [Figure] [Figure]

Revised: [Figure]

Revised: and driving images, and (2) Warping the source image features based on the dense correspondences.

Revised: The key intuition behind our method is that these two steps can be represented using a single scaled dot-product attention layer popularized by Vaswani et al .

Original: [Equation]
Revised: [Equation]

Original: where Q , K , and V are made up of vector-valued queries, keys, and values, respectively, and c isa scalar-valued scaling factor.
Revised: where Q , K , and V are made up of vector-valued queries, keys, and values, respectively, and c is a scalar-valued scaling factor.

Original: For a given query vector Q of size p 1 ˆ d q , the softmaxed value ofthe product Q ˆ K T can be interpreted as the similarity S of the given query vector with the setof all available keys K of size p k ˆ d q .
Revised: For a given query vector Q of size p 1 ˆ d q , the softmaxed value of the product Q ˆ K T can be interpreted as the similarity S of the given query vector with the set of all available keys K of size p k ˆ d q .

Original: After the softmax normalization, the key with the highestdot-product with the query will obtain the highest similarity.
Revised: After the softmax normalization, the key with the highest dot-product with the query will obtain the highest similarity.

Original: After computing the similarity S of sizep 1 ˆ k q and given values V of size p k ˆ d 1 q , the product S
Revised: After computing the similarity S of size p 1 ˆ k q and given values V of size p k ˆ d 1 q , the product S

Original: ˆ V gives a query-conditional weightedaverage of values of size p 1 ˆ d 1 q .
Revised: ˆ V gives a query-conditional weighted average of values of size p 1 ˆ d 1 q .

Original: This scheme naturally extends to multiple queries Q of size p q ˆ d q .
Revised: This scheme naturally extends to multiple queries Q of size p q ˆ d q .

Original: In our particular application, the queries Q come from the driving keypoints, and the keys K andvalues V come from the source image(s) keypoints and features, respectively.
Revised: In our particular application, the queries Q come from the driving keypoints, and the keys K and values V come from the source image(s) keypoints and features, respectively.

Original: Using additional sourceimages only leads to an increase in the number of available keys and values.
Revised: Using additional source images only leads to an increase in the number of available keys and values.

Original: We refer to the above procedure as implicit warping , as illustrated in Fig.
Revised: We refer to the above procedure as implicit warping , as illustrated in Fig.

Original: 2b.
Revised: 2b.

Original: This is opposed toexplicit warping shown in Fig.
Revised: This is opposed to explicit warping shown in Fig.

Original: 2a, where an explicit flow map is first computed from the source anddriving keypoints and then applied to source features to produce warped outputs.
Revised: 2a, where an explicit ﬂow map is ﬁrst computed from the source and driving keypoints and then applied to source features to produce warped outputs.

Original: There is no explicitflow map in our scheme; the similarity between the queries and keys produces an implicit flow map,which is then applied to the values to produce the warped outputs.
Revised: There is no explicit ﬂow map in our scheme; the similarity between the queries and keys produces an implicit ﬂow map, which is then applied to the values to produce the warped outputs.

Original: In the following, we describe howwe learn queries, keys, and values for the novel implicit warping based on cross-modal attention.
Revised: In the following, we describe how we learn queries, keys, and values for the novel implicit warping based on cross-modal attention.

Original: Learning queries, keys, and values.
Revised: Learning queries, keys, and values.

Original: Similar to the prior works [26, 27, 41], we use learned keypointsand associated information as a compact representation of the source and driving images.
Revised: Similar to the prior works [26, 27, 41], we use learned keypoints and associated information as a compact representation of the source and driving images.

Original: Our imageencoder that predicts keypoints reuses the architecture from FOMM [26].
Revised: Our image encoder that predicts keypoints reuses the architecture from FOMM [26].

Original: While reconstructingthe driving image, we only have access to the driving keypoints but have access to both the sourceimage(s) and the source keypoints.
Revised: While reconstructing the driving image, we only have access to the driving keypoints but have access to both the source image(s) and the source keypoints.

Original: Given the K keypoints predicted from an image, we create aspatial representation of K channels where each channel represents a keypoint.
Revised: Given the K keypoints predicted from an image, we create a spatial representation of K channels where each channel represents a keypoint.

Original: For the k th keypoint,we place a Gaussian of a fixed mean and variance in channel k , centered at the predicted keypointlocation.
Revised: For the k th keypoint, we place a Gaussian of a ﬁxed mean and variance in channel k , centered at the predicted keypoint location.

Original: In addition to the K keypoints, we predict a scalar per-keypoint with values in r 0 , 1 s , servingas the keypoint strength score.
Revised: In addition to the K keypoints, we predict a scalar per-keypoint with values in r 0 , 1 s , serving as the keypoint strength score.

Original: We multiply this predicted scalar with the channel corresponding tothe keypoint.
Revised: We multiply this predicted scalar with the channel corresponding to the keypoint.

Original: This allows us to modulate the keypoint strength , which is related to its visibility, in thegiven source and driving images.
Revised: This allows us to modulate the keypoint strength , which is related to its visibility, in the given source and driving images.

Original: The predicted keypoint strengths are later discussed in Section 4 andvisualized in Fig.
Revised: The predicted keypoint strengths are later discussed in Section 4 and visualized in Fig.

Original: Both the keypoint locations and the scalar factors are learned in an unsupervisedmanner.
Revised: Both the keypoint locations and the scalar factors are learned in an unsupervised manner.

Original: Unlike FOMM [26], we do not predict the Jacobian matrix per keypoint in addition to thekeypoint location, making our intermediate representation more compact ( 2 ` 1 “ 3 values perkeypoint instead of 2 ` 4 “ 6 values per keypoint in the prior work).
Revised: Unlike FOMM [26], we do not predict the Jacobian matrix per keypoint in addition to the keypoint location, making our intermediate representation more compact ( 2 ` 1 “ 3 values per keypoint instead of 2 ` 4 “ 6 values per keypoint in the prior work).

Original: Unlike face-vid2vid [41], wedo not assume rigid motion and find applications beyond talking-head animation.
Revised: Unlike face-vid2vid [41], we do not assume rigid motion and ﬁnd applications beyond talking-head animation.

Original: We use a U-net to encode the above spatial driving keypoint representations, with a final point-wiseprojection layer to map it to the dimension d selected for the queries and the keys.
Revised: We use a U-net to encode the above spatial driving keypoint representations, with a ﬁnal point-wise projection layer to map it to the dimension d selected for the queries and the keys.

Original: A similar U-netis used to encode the source image, except that its input consists of both the source image and thespatial source keypoint representation.
Revised: A similar U-net is used to encode the source image, except that its input consists of both the source image and the spatial source keypoint representation.

Original: Both these networks accept inputs and produce outputs at 1 { 4resolution of the source image, i.e . 64 for 256 ˆ 256 , 96 for 384 ˆ 384 , etc .
Revised: Both these networks accept inputs and produce outputs at 1 {resolution of the source image, i.e . 64 for 256 ˆ 256 , 96 for 384 ˆ 384 , etc .

Original: Assuming a source imagesize of 256 ˆ 256 , the two U-net networks produce a total of p 64 ˆ 64 ˆ d q , or p 4096 ˆ d q queries andkeys , for each driving and source image, respectively.
Revised: Assuming a source image size of 256 ˆ 256 , the two U-net networks produce a total of p 64 ˆ 64 ˆ d q , or p 4096 ˆ d q queries and keys , for each driving and source image, respectively.

Original: Since the output of these U-nets is of a fixeddimension d , the design of the subsequent components is independent of the number of keypoints used.
Revised: Since the output of these U-nets is of a ﬁxed dimension d , the design of the subsequent components is independent of the number of keypoints used.

Original: In order to obtain values from the source image, we pass it through two downsampling convolutionlayers and obtain values of size p 4096 ˆ d 1 q .
Revised: In order to obtain values from the source image, we pass it through two downsampling convolution layers and obtain values of size p 4096 ˆ d 1 q .

Original: Note that there is a one-to-one spatial correspondencebetween the keys and the values.
Revised: Note that there is a one-to-one spatial correspondence between the keys and the values.

Original: Additional details can be found in the supplementary material.
Revised: Additional details can be found in the supplementary material.

Original: However, there is one possible pitfall to the scheme described so far.
Revised: However, there is one possible pitfall to the scheme described so far.

Original: If none of the keys have ahigh dot-product with the query, the key with the highest dot-product will still be assigned the highsimilarity after the softmax, ensuring that the outputs sum up to 1.
Revised: If none of the keys have a high dot-product with the query, the key with the highest dot-product will still be assigned the high similarity after the softmax, ensuring that the outputs sum up to 1.

Original: While it is the key most similarto the query in the given set of keys, it may not be similar enough to produce a good output.
Revised: While it is the key most similar to the query in the given set of keys, it may not be similar enough to produce a good output.

Original: Forexample, suppose the source image has a face with lips closed, while the driving image has onewith lips open and teeth exposed.
Revised: For example, suppose the source image has a face with lips closed, while the driving image has one with lips open and teeth exposed.

Original: In this case, there will be no key (and value) in the source imageappropriate for the mouth region of the driving image.
Revised: In this case, there will be no key (and value) in the source image appropriate for the mouth region of the driving image.

Original: We overcome this issue by allowing ourmethod to learn additional image-independent key-value pairs, which can be used in the case ofmissing information in the source image.
Revised: We overcome this issue by allowing our method to learn additional image-independent key-value pairs, which can be used in the case of missing information in the source image.

Original: These additional keys and values are concatenated to thekeys and values obtained from the source image.
Revised: These additional keys and values are concatenated to the keys and values obtained from the source image.

Original: In our experiments, we add 400 extra keys andvalues, approximately 10% of the number of keys and values obtained from a single source image ofsize 256 ˆ 256 .
Revised: In our experiments, we add 400 extra keys and values, approximately 10% of the number of keys and values obtained from a single source image of size 256 ˆ 256 .

Original: We use semantic dropout while training our framework to encourage the learningand use of the added keys and values.
Revised: We use semantic dropout while training our framework to encourage the learning and use of the added keys and values.

Original: This is achieved by randomly dropping out keys and valuescorresponding to different facial regions, such as eyebrows, eyes, lips, teeth, etc .
Revised: This is achieved by randomly dropping out keys and values corresponding to different facial regions, such as eyebrows, eyes, lips, teeth, etc .

Original: This is possible asan attention layer treats its input keys and values as a set.
Revised: This is possible as an attention layer treats its input keys and values as a set.

Original: In contrast, convolutional layers assumea specific spatial order, and spatial locations cannot be randomly dropped out.
Revised: In contrast, convolutional layers assume a speciﬁc spatial order, and spatial locations cannot be randomly dropped out.

Original: As shown later in
Revised: As shown later in

Original: Section 4 and Table 4, these extra keys and values help improve output quality.
Revised: Section 4 and Table 4, these extra keys and values help improve output quality.

Original: The cross-modal attention module.
Revised: The cross-modal attention module.

Original: After obtaining queries, keys, and values, the cross-modalattention module computes the warped source features conditional on the driving image via implicitwarping.
Revised: After obtaining queries, keys, and values, the cross-modal attention module computes the warped source features conditional on the driving image via implicit warping.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 3, the queries kp d obtained from the driving image are of size p q ˆ d qafter flattening.
Revised: 3, the queries kp d obtained from the driving image are of size p q ˆ d q after ﬂattening.

Original: The keys kp s and values f s obtained from the one or more source images are of sizep k ˆ d q and p
Revised: The keys kp s and values f s obtained from the one or more source images are of size p k ˆ d q and p

Original: k ˆ d 1 q respectively.
Revised: k ˆ d 1 q respectively.

Original: We add learnable position embeddings to the queries kp d and keyskp s .
Revised: We add learnable position embeddings to the queries kp d and keys kp s .

Original: The dot product Q ˆ K T after scaling and softmax gives the attention matrix A of size p q ˆ k q .
Revised: The dot product Q ˆ K T after scaling and softmax gives the attention matrix A of size p q ˆ k q .

Original: By multiplying A with values V of size p k ˆ d 1 q , we obtain the warped output feature of size p q ˆ d 1 q .
Revised: By multiplying A with values V of size p k ˆ d 1 q , we obtain the warped output feature of size p q ˆ d 1 q .

Original: Additionally, we concatenate the image pixels with the keys and align them with the driving image bymultiplying them with the computed attention map, which is further concatenated with the queries andpassed through an MLP (purple-shaded part of Fig. 3).
Revised: Additionally, we concatenate the image pixels with the keys and align them with the driving image by multiplying them with the computed attention map, which is further concatenated with the queries and passed through an MLP (purple-shaded part of Fig. 3).

Original: The intuition behind using the warped keys andqueries is to recover any information, such as skew or rotation, which a weighted average of valuesmay not easily capture.
Revised: The intuition behind using the warped keys and queries is to recover any information, such as skew or rotation, which a weighted average of values may not easily capture.

Original: Further, including the pixel values helped by aiding color consistency.
Revised: Further, including the pixel values helped by aiding color consistency.

Original: Addingthis residual connection helped improve the quality of outputs, as shown in Section 4 and Table 4.
Revised: Adding this residual connection helped improve the quality of outputs, as shown in Section 4 and Table 4.

Original: While our end-to-end framework is reasonably fast ( „ 10 FPS on 512 ˆ 512 images), it is possible toreduce the run-time of the attention operations by utilizing the factored 1-D attention layers [47] orthe spatial-reduction attention (SRA) layer [42], which we leave for the future work.
Revised: While our end-to-end framework is reasonably fast ( „ 10 FPS on 512 ˆ 512 images), it is possible to reduce the run-time of the attention operations by utilizing the factored 1-D attention layers [47] or the spatial-reduction attention (SRA) layer [42], which we leave for the future work.

Original: After training,we observed that the learned attention masks are very sparse.
Revised: After training, we observed that the learned attention masks are very sparse.

Original: This indicates that we can furtherreduce the inference time by selecting just the top-few values per row from the product A “ Q ˆ K T ,and multiplying them with corresponding values of V , thereby saving computation.
Revised: This indicates that we can further reduce the inference time by selecting just the top-few values per row from the product A “ Q ˆ K T , and multiplying them with corresponding values of V , thereby saving computation.

Original: We show results on multiple datasets in the single and multiple source image settings with comparisonto strong baselines in the next section, and try to provide insights into the workings of the model.
Revised: We show results on multiple datasets in the single and multiple source image settings with comparison to strong baselines in the next section, and try to provide insights into the workings of the model.

Original:  Experiments
Revised:  Experiments

Original: Datasets.
Revised: Datasets.

Original: We perform our ablations on the TalkingHead-1KH [41] dataset at the 256 ˆ 256 resolutionand compare with baselines at the full 

Revised: We perform our ablations on the TalkingHead-1KH [41] dataset at the 256 ˆ 256 resolution and compare with baselines at the full

Original: ˆ 512 resolution.
Revised: ˆ 512 resolution.

Original: We also report results on the 256 ˆ[-
-]
Revised: We also report results on the 256 ˆ[-
-]

Original: VoxCeleb2 [26] dataset.
Revised: VoxCeleb2 [26] dataset.

Original: Additionally, to demonstrate the generality of our method, we report resultson the more challenging TED Talk [28] dataset of moving upper bodies at a resolution of 384 ˆ 384 .
Revised: Additionally, to demonstrate the generality of our method, we report results on the more challenging TED Talk [28] dataset of moving upper bodies at a resolution of 384 ˆ 384 .

Original: Metrics.
Revised: Metrics.

Original: We measure the fidelity of driving image reconstruction using PSNR, L 1 , LPIPS [53], and
Revised: We measure the ﬁdelity of driving image reconstruction using PSNR, L 1 , LPIPS [53], and

Original: FID [12].
Revised: FID [12].

Original: The quality of motion transfer is measured using average keypoint distance (AKD) betweenkeypoints predicted using MTCNN [52] for faces and OpenPose [4] for upper bodies.
Revised: The quality of motion transfer is measured using average keypoint distance (AKD) between keypoints predicted using MTCNN [52] for faces and OpenPose [4] for upper bodies.

Original: For upper-bodymotion transfer, we also report the missing keypoint ratio (MKR), at a chosen keypoint predictionconfidence threshold C , together denoted as (AKD, MKR)@ C .
Revised: For upper-body motion transfer, we also report the missing keypoint ratio (MKR), at a chosen keypoint prediction conﬁdence threshold C , together denoted as (AKD, MKR)@ C .

Original: For human evaluation, MTurk userswere shown videos synthesized by two different methods and asked to choose the more realistic one.
Revised: For human evaluation, MTurk users were shown videos synthesized by two different methods and asked to choose the more realistic one.

Original: Ò larger is better, Ó smaller is better.

Original: See section 4 for details about metrics.

Original: [Table]

Original: [Table]

Original: Baselines.
Revised: Baselines.

Original: On the face datasets, we compare our method against two state-of-the-art methods:
Revised: On the face datasets, we compare our method against two state-of-the-art methods:

Original: First-Order Motion Model (FOMM)
Revised: First-Order Motion Model (FOMM)

Original: [26], and face-vid2vid (fv2v) [41].
Revised: [26], and face-vid2vid (fv2v) [41].

Original: On the upper-body dataset,we compare against FOMM [26] and the state-of-the-art AA-PCA [28].
Revised: On the upper-body dataset, we compare against FOMM [26] and the state-of-the-art AA-PCA [28].

Original: We use the pretrained modelprovided for AA-PCA, while we train all other methods from scratch.
Revised: We use the pretrained model provided for AA-PCA, while we train all other methods from scratch.

Original: As previously mentioned in
Revised: As previously mentioned in

Original: Section 1, prior works are unable to handle multiple source images, despite the immense practicalityof the setting.
Revised: Section 1, prior works are unable to handle multiple source images, despite the immense practicality of the setting.

Original: We adapt these methods, where possible, to multiple source images by warping featuresfrom each provided source image, and then averaging them to obtain a single warped feature map.
Revised: We adapt these methods, where possible, to multiple source images by warping features from each provided source image, and then averaging them to obtain a single warped feature map.

Original: All methods use K “ 20 keypoints in the following experiments.
Revised: All methods use K “ 20 keypoints in the following experiments.

Original: Additional details about datasets,metrics, and training hyper-parameters for all methods are provided in the supplementary material.
Revised: Additional details about datasets, metrics, and training hyper-parameters for all methods are provided in the supplementary material.

Original: Driving image reconstruction with a single source image.
Revised: Driving image reconstruction with a single source image.

Original: This is the standard evaluation settingused in prior work [26–28, 41].
Revised: This is the standard evaluation setting used in prior work [26–28, 41].

Original: The first frame of each video is chosen as a source image, and thefull-length videos are reconstructed.
Revised: The ﬁrst frame of each video is chosen as a source image, and the full-length videos are reconstructed.

Original: Quantitative evaluation of predictions is presented in Table 1.
Revised: Quantitative evaluation of predictions is presented in Table 1.

Original: Our method outperforms competing methods in almost all metrics on both datasets.
Revised: Our method outperforms competing methods in almost all metrics on both datasets.

Original: We obtain a lower
Revised: We obtain a lower

Original: FID and average keypoint distance (AKD) on both datasets, indicating good visual quality as well asmotion reproduction in the predictions.
Revised: FID and average keypoint distance (AKD) on both datasets, indicating good visual quality as well as motion reproduction in the predictions.

Original: Fig.
Revised: Fig.

Original: 4 highlights a case where our attention-based methodis able to provide the correct reconstruction while other methods fail.
Revised: 4 highlights a case where our attention-based method is able to provide the correct reconstruction while other methods fail.

Original: As attention is non-local andglobal in nature, it can borrow image features from regions that are spatially far away, e.g . the left andright hair and ears.
Revised: As attention is non-local and global in nature, it can borrow image features from regions that are spatially far away, e.g . the left and right hair and ears.

Original: Prior works based on explicit flow prediction would have to predict large valuesof flow for such newly disoccluded regions.
Revised: Prior works based on explicit ﬂow prediction would have to predict large values of ﬂow for such newly disoccluded regions.

Original: Fig.
Revised: Fig.

Revised: 5 compares results between the various methods

Revised: Ò larger is better, Ó smaller is better.

Original: 5 compares results between the various methodswhen using a single source image on the TED Talk dataset.
Revised: See section 4 for details about metrics. [Table] [Table] when using a single source image on the TED Talk dataset.

Original: Our method demonstrates better handand face reconstruction than prior works.
Revised: Our method demonstrates better hand and face reconstruction than prior works.

Original: Driving image reconstruction with multiple source images.
Revised: Driving image reconstruction with multiple source images.

Original: We now compare results in the case ofdriving image reconstruction when multiple source images are provided.
Revised: We now compare results in the case of driving image reconstruction when multiple source images are provided.

Original: This is a setting of greatpractical value as a single source image often does not have all the details necessary to reconstructa driving image with any arbitrary pose and expression.
Revised: This is a setting of great practical value as a single source image often does not have all the details necessary to reconstruct a driving image with any arbitrary pose and expression.

Original: We present results obtained when usingmultiple source images in Table 2.
Revised: We present results obtained when using multiple source images in Table 2.

Original: Here, we use sequences of length at most 180 frames and select [Figure] Ours
Revised: Here, we use sequences of length at most 180 frames and select regularly-spaced source images in addition to the ﬁrst frame.

Original: FOMM [26] fv2v [41]
Revised: Similar to the single source image setting, our method outperforms all prior work. As the number of source images increases, our method obtains better reconstructions as indicated by the improving scores on all metrics. However, reconstructions by prior work get worse as the number of source images increases, contrary to expectation.

Original: Our output [Figure] [Figure] FOMM [26]
Revised: Upon viewing the qualitative results shown in Fig.

Original: AA
Revised: 1, the issue with prior works becomes clear—even though features from each source image are warped to the same driving pose, there exist misalignments between the warped features.

Original: [Figure] PCA [28]
Revised: Further, they do not have the ability to choose features from only a speciﬁc source image.

Original: [Figure] this issue due to global attentionover all available images and their features.
Revised: Our method does not suffer from this issue due to global attention over all available images and their features.

Original: As a result, our method is able to predict the correcteye color and sharp background, as seen in Fig.
Revised: As a result, our method is able to predict the correct eye color and sharp background, as seen in Fig.

Original: Fig.
Revised: Fig.

Original: 4 shows regions of the source image that thehighlighted regions of the output borrow features from.
Revised: 4 shows regions of the source image that the highlighted regions of the output borrow features from.

Original: This demonstrates our picking-and-choosingmechanism—regions t 1 , 2 , 4 , 7 , 9 u visible in both sources contribute equally to the output, while therest, only visible in one of the sources, is used exclusively.
Revised: This demonstrates our picking-and-choosing mechanism—regions t 1 , 2 , 4 , 7 , 9 u visible in both sources contribute equally to the output, while the rest, only visible in one of the sources, is used exclusively.

Original: User preference scores are provided in
Revised: User preference scores are provided in

Original: Table 3.
Revised: Table 3.

Original: Users prefer our method in all settings, and the preference increases further when moresource images are used.
Revised: Users prefer our method in all settings, and the preference increases further when more source images are used.

Original: Additional comparisons are available in the supplementary material.
Revised: Additional comparisons are available in the supplementary material.

Original: A closer look at the architecture.
Revised: A closer look at the architecture.

Original: Having shown the superiority of our method on image reconstruction in the single and multi-source image settings, we now examine the contribution of architectural[Table] [Table] choices to the output quality.
Revised: Having shown the superiority of our method on image reconstruction in the single and multi-source image settings, we now examine the contribution of architectural choices to the output quality.

Original: As shown in Table 4, adding the residual connection visualized by thepurple region in Fig.
Revised: As shown in Table 4, adding the residual connection visualized by the purple region in Fig.

Original: 3, improves all metrics.
Revised: 3, improves all metrics.

Original: This indicates that the warped image and keys, alongwith the queries provide additional information to the values.
Revised: This indicates that the warped image and keys, along with the queries provide additional information to the values.

Original: Adding extra learnable keys and values,as described in Section 3 further improves all metrics by allowing the network to share features wheninformation is missing in the source image(s).
Revised: Adding extra learnable keys and values, as described in Section 3 further improves all metrics by allowing the network to share features when information is missing in the source image(s).

Original: Fig.
Revised: Fig.

Original: 6 visualizes predicted keypoints and strengths forvarious configurations of a person’s face.
Revised: 6 visualizes predicted keypoints and strengths for [Figure] Ours

Original: For the same arrangement of keypoints (both or single eyeclosed), we see different keypoint strengths based on the appearance. We also see large changes inkeypoint strengths when certain keypoints are occluded, e.g . when the face is rotated to an extremeangle. This single scalar in place of a 4-valued Jacobian matrix per keypoint helps us achieve a morecompact representation useful for applications such as video compression and conferencing [41] 
Revised: FOMM [26] fv2v [41] Our output

Original: [Table]
Revised: [Figure]

Original: Discussion

Original: Referenceimage BotheyesopenForwardfacingMouthclosed [Equation] [Table]
Revised: [Figure]

Original: [Figure]

Original: [Table] [Figure] [Table] [Figure]
Revised: various conﬁgurations of a person’s face.

Original: [Table]

Original: [Figure] [Table] [Figure] [Table]
Revised: For the same arrangement of keypoints (both or single eye closed), we see different keypoint strengths based on the appearance.

Original: [Figure] [Table] [Figure] [Table]
Revised: We also see large changes in keypoint strengths when certain keypoints are occluded, e.g . when the face is rotated to an extreme angle.

Original: [Figure] [Table] [Figure] [Table]
Revised: This single scalar in place of a 4-valued Jacobian matrix per keypoint helps us achieve a more compact representation useful for applications such as video compression and conferencing [41].

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original:  [Table] [Figure] [Table] [Figure]
Revised:  Discussion

Original: [Table]

Original: [Figure] [Table] [Figure] In this work, we presented the novelimplicit warping framework andshowed its application to image animation.
Revised: In this work, we presented the novel implicit warping framework and showed its application to image animation.

Original: Through the use of a single cross-modal attention layer, ourmethod is able to deform source image features conditional on the driving image, and can scale to multiple source images as-is.
Revised: Through the use of a single cross-modal attention layer, our method is able to deform source image features conditional on the driving image, and can scale to multiple source images as-is.

Original: Unlikeprior work, our method does not predict explicit per-source image flows,and can pick-and-choose from multiple source image features.
Revised: Unlike prior work, our method does not predict explicit per-source image ﬂows, and can pick-and-choose from multiple source image features.

Original: It convincingly beats prior state-of-the-artmethods on multiple datasets in boththe single and multiple source imagesettings, while using a more compact intermediate representation ofkeypoint locations and strengths.
Revised: It convincingly beats prior state-of-the-art methods on multiple datasets in both the single and multiple source image settings, while using a more compact intermediate representation of keypoint locations and strengths.

Original: Limitations.
Revised: Limitations.

Original: Our method can fail in cases where it is expected to hallucinate a lot of missinginformation.
Revised: Our method can fail in cases where it is expected to hallucinate a lot of missing information.

Original: For example, given just a source image of the back of someone’s head, it cannot generatethe correct frontal face.
Revised: For example, given just a source image of the back of someone’s head, it cannot generate [Table] [Table] [Table]

Revised: the correct frontal face.

Original: Since it is a data-driven approach, it might fail for extreme expressions notpresent in the training dataset.
Revised: Since it is a data-driven approach, it might fail for extreme expressions not present in the training dataset.

Original: Additional data and augmentations might help alleviate such issues.
Revised: Additional data and augmentations might help alleviate such issues.

Original: Societal impact.
Revised: Societal impact.

Original: Our method has the potential for negative impact if used to create deepfakes .
Revised: Our method has the potential for negative impact if used to create deepfakes .

Original: Viathe use of cross-identity transfer and speech synthesis, a malicious actor can create faked videos ofa person, resulting in identity theft or dissemination of fake news.
Revised: Via the use of cross-identity transfer and speech synthesis, a malicious actor can create faked videos of a person, resulting in identity theft or dissemination of fake news.

Original: However, in controlled settings,the same technology can also be used for entertainment purposes.
Revised: However, in controlled settings, the same technology can also be used for entertainment purposes.

Original: Our method can be used inlow-bandwidth video compression in the same-identity video reconstruction setting.
Revised: Our method can be used in low-bandwidth video compression in the same-identity video reconstruction setting.

Original: This can helpimprove video quality, especially for regions with poor connectivity.
Revised: This can help improve video quality, especially for regions with poor connectivity.

Original: As our method generates outputsusing source image features, we have not observed and do not expect it to demonstrate any racial bias.
Revised: As our method generates outputs using source image features, we have not observed and do not expect it to demonstrate any racial bias.

Original: Checklist
Revised: Checklist

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] See section 5.(c)
Revised: [Yes] See section 5.

Original:  Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes] Seesection 5.
Revised: [Yes] See section 5.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A](b) Did you include complete proofs of all theoretical results?
Revised: [N/A] (b) Did you include complete proofs of all theoretical results?

Original: [N/A]3.
Revised: [N/A] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [No] The datasets used are publically available and URLs are in the supplementary materials.
Revised: [No] The datasets used are publically available and URLs are in the supplementary materials.

Original: We willrelease the code after acceptance.
Revised: We will release the code after acceptance.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] See provided supplementary material.
Revised: [Yes] See provided supplementary material.

Original: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
Revised: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] We have reported standard deviations for ablations insupplementary material.
Revised: [Yes] We have reported standard deviations for ablations in the supplementary material.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g.,type of GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] See provided supplementarymaterial.
Revised: [Yes] See provided supplementary material.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes](b) Did you mention the license of the assets?
Revised: [Yes] (b) Did you mention the license of the assets?

Original: [Yes] , when and where provided.
Revised: [Yes] , when and where provided.

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No][Table] No new assets are being used/released as part of this work.
Revised: [No] No new assets are being used/released as part of this work.

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re[Table] using/curating?
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: [N/A]
Revised: [N/A]

Original: [Table] (e) Did you discuss whether the data you are using/curating contains personally identifiable [Table] information or offensive content?
Revised: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?

Original: [No] All datasets we use were obtained from public[Table] YouTube videos under the Creative Commons license.
Revised: [No] All datasets we use were obtained from public YouTube videos under the Creative Commons license.

Original: If you used crowdsourcing or conducted research with human subjects...[Table] (a) Did you include the full text of instructions given to participants and screenshots, if [Table]
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: applicable?

Original: [Yes] See provided supplementary material.
Revised: [Yes] See provided supplementary material.

Original: [Table] (b) Did you describe any potential participant risks, with links to Institutional Review[Table] Board (IRB) approvals, if applicable?
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A][Table] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [Yes] See provided supplementary material.
Revised: [Yes] See provided supplementary material.

Original: [Table]