Dense Interspecies Face Embedding
Anonymous Author(s)
AfﬁliationAddress email
Abstract
Dense Interspecies Face Embedding (DIFE) is a new direction for understanding faces of various animals by extracting common features among animal facesincluding human face.
There are three main obstacles for interspecies face understanding: (1) lack of animal data compared to human, (2) ambiguous connectionbetween faces of various animals, and (3) extreme shape and style variance.
Tocope with the lack of data, we utilize multi-teacher knowledge distillation of CSEand StyleGAN2 requiring no additional data or label.
Then we synthesize pseudopair images through the latent space exploration of StyleGAN2 to ﬁnd implicitassociations between different animal faces.
Finally, we introduce the semanticmatching loss to overcome the problem of extreme shape differences betweenspecies.
To quantitatively evaluate our method over possible previous methodologies like unsupervised keypoint detection, we perform interspecies facial keypointtransfer on MAFL and AP-10K. Furthermore, the results of other applications likeinterspecies face image manipulation and dense keypoint transfer are provided.
Introduction
Face understanding is a longstanding research topic in computer vision, and there have been extensivestudies and applications such as face recognition, facial expression estimation, face photo manipulation, and virtual humans [6, 43, 4, 47, 3].
The vast majority of successful works on faces have focusedon human faces due to its practical implication for a variety of applications and the accessibility toa large amount of annotated data.
There have been few attempts for understanding faces of otheranimal species to facilitate more applications such as ecological research, pet identiﬁcation, andfarm automation [13, 31, 11].
However, the task of understanding animal faces is more challengingthan that for the human face due to the lack of sufﬁcient data with annotations.
In the literatureon full-body understanding, joint learning of interspecies data has been studied as a remedy bytransferring the knowledge of well-annotated domain such as human to other species [30, 23, 24].
However, the existing approaches are focused on full-body data, therefore their performance on facialdata is limited.
In this work, we are interested in the problem of learning the representation of interspecies facialsemantics, which is a common representation shared across interspecies facial data.
Learning suchrepresentation allows us to extend face understanding beyond the human face, as it enables discoveringfacial semantics of other species without enough annotations through transferring knowledge fromwell-annotated human data.
As many animal species share similar face geometry, the representationcan also beneﬁt from learning multiple species jointly.
Since sharing photos of animals through socialnetworks has become more popular, learning the interspecies face representation can be used forvarious applications.
For instance, one can search animal photos using a photo of human face as aquery to ﬁnd animal faces with similar pose.
An animal face animation can be synthesized by usingthat of a human as a template, which can be useful for media production.
[Figure]
To achieve our goal, we propose to learn Dense Interspecies Face Embedding (DIFE) as shown in
Fig. 1(a).
Our approach is inspired by CSE [24], which was proposed to learn continuous surfaceembedding (CSE) of full-body shared across interspecies data.
While the interspecies embedding in[24] is promising, it is difﬁcult to use it for faces as the embedding in the facial region is too coarseto discriminate facial landmarks accurately.
To overcome this problem, we propose a multi-teacherknowledge distillation method for learning DIFE.
Our key idea is to enforce a student network to learna ﬁne-grained interspecies facial embedding from two teacher networks: CSE and StyleGAN2 [18].
Since the CSE has a useful representation of interspecies data, we use it as a teacher to have thestudent network learn a common embedding space shared in multiple species.
For a ﬁne-grainedface embedding, we simultaneously exploit StyleGAN2 as the secondary teacher.
StyleGAN2 isoriginally trained to generate high-quality face images [18].
Recent studies have demonstrated thatthe features of StyleGAN2 have abundant information on facial attributes such as postures andexpressions [12, 33, 34].
Thus, the goal of our knowledge distillation is to transfer such informationto the student network for learning ﬁne-grained facial semantics for multiple species.
To further improve the performance of DIFE, we propose a data augmentation method to synthesizepseudo-paired images, and the semantic matching loss to enhance the semantic correspondencebetween images.
For the data augmentation, we synthesize images, where the face poses of thesynthesized images are similar to that of input images using StyleGAN2.
This can be achieved inour framework as DIFE is tied to the latent features of multiple StyleGAN2 networks in differentdomains.
Thus, DIFE of the input image can be used as a face geometry condition for the latent spaceexploration of StyleGAN2 to generate images for both intra- and inter-domain pairs.
In addition,inspired by [36], a semantic matching loss is proposed to use the pseudo-paired data to enforce the
DIFE to learn pixel-level ﬁne-grained matching.
We demonstrate the effectiveness of our method on the cross-domain keypoint transfer task usinginterspecies data such as people, dogs, cats, and wild animals in MAFL [46] and AP-10K [41].
Weevaluate our method on both sparse and dense keypoint transfers.
In addition, we apply our methodfor an image editing task of overlaying a virtual object on various animal faces.
Our contribution can be summarized as follows:
• We present a cross-domain face understanding study that considers not only human faces
but also the faces of animals.
Our approach makes it possible to avoid tedious and expensive
animal data collection through domain adaptation, setting the stage for a variety of cross-
domain applications.
• We present a novel multi-teacher knowledge distillation paradigm that extracts and combines
the information from models with different architectures and data domains.
Our framework
is designed to learn continuous face embedding across interspecies data from CSE and
ﬁne-grained face embedding from StyleGAN2.
Combining the knowledge from the two,our method learns dense interspecies face embedding (DIFE).
• We further introduce a method for synthesizing paired data for learning the semantic
matching using the latent space exploration of StyleGAN2.
With the novel approach, we
can reﬁne the representation of our face embedding for semantic correspondence between
various instances and species.
Related Works2.1 Domain-speciﬁc Face Understanding
We ﬁrst review works that aimed to identify the visual correspondence between various instancesand scenes within one data category, human or animal.
For faces, detecting facial landmarks suchas the eyes and the nose has been studied extensively to ﬁnd correspondences between subjects.
Initial works [35, 22, 42, 45, 39, 5] ﬁnd facial landmarks using facial landmark annotations in a fullysupervised way.
Regarding animal faces, Yang et al.
[40] detected sheep facial landmarks based on 6annotated keypoints.
Although most works show impressive results under extreme conditions, thelack of annotated animal data limits the application compared to human faces.
Recently, there have been active researches [37, 44, 14, 36, 15, 7, 16] on ﬁnding landmarks froma collection of images in an unsupervised way, which is applicable to various domains, includinghuman body, face, etc.
This stream of works relies on the fact that landmarks are spatially equivariantto the transformation.
Some works learn landmarks by comparing warped images with the originalones [37, 44], and others cast the problem into a conditional image generation using video framesdepicting the same instances in different poses [14, 15].
The most related work to our method is
DVE [36] in that they consider not only equivariance between the same instances but also intracategory variations.
All of aforementioned methods can detect landmarks within the same objectcategories, but the landmarks are not convertible between different object categories (e.g. humansand cats).
2.2 Interspecies Understanding
There are studies that aim to discover the relationship between animal species, whose structures arerelated but geometrically diverse.
To the best of our knowledge, [29] is the only work that conductedsemantic matching between animal faces.
Animal facial landmark detector is trained by transferringthe knowledge from the pre-trained human landmark detector.
The network is ﬁnetuned using horsesand sheep data with 5 keypoints annotations.
In comparison, we propose an unsupervised approachfor interspecies facial semantic matching with the higher granularity of the keypoints.
Closely relevant to our work is the stream of works in [30, 23, 24], as they ﬁnd the relationship ofbody poses between animals.
Sanakoyeu et al.
[30] transfers the knowledge of the human body tothe chimpanzee using DensePose [9].
The CSE [23] utilizes the manually annotated 3D meshes ofeach category and continuous surface embedding to relate several animal categories ﬂexibly.
In thefollowing work [24], they utilize the cycle-consistency between meshes and images to further theperformance.
Although they show impressive results on the complete body, they have difﬁculty inachieving a ﬁne-level sharpness on faces, making it impossible to discern different parts of faces.
While our work also takes the advantage of CSE [24] to detect interspecies correspondences, weimprove the accuracy of facial correspondence with the help of the latent features of StyleGAN2 [18]without additional annotations.
2.3 StyleGAN2 Utilization
Our work is also related to StyleGAN utilization works in that we use features of pretrained StyleGAN2 [18] to obtain ﬁne-grained facial features.
StyleGANs [17, 18] show highly photo-realisticresults in the unconditional image generation problem with the power of well disentangled latentspaces.
There have been abundant works to exploit the pre-trained StyleGAN for various tasks,including image manipulation [38, 12, 34, 32], 3D reconstruction [27], image segmentation [20, 2],and semantic matching [28].
Because StyleGAN2 has plenty of face geometry many works try to extract the information withknowledge distillation [2, 26 ? ].
Labels4Free [2] and Segmentation in style [26] utilizes the featureof StyleGAN2 to train segmentation model in unsupervised manner.
In StyleGAN2 distillation [ ? ],they generate images and labels with StyleGAN2 generator and face attribute classiﬁer, and imagemanipulation model is trained with synthesized data.
However we can tell no studies have used
StyleGAN models learned in different data domains at the same time, while our work learns therelationship between separate StyleGAN2 models, which have different data domains.
[Figure]
Method
Let x d ∈ R
H
× W × 3 denote an image of a face in a domain d ∈ { 0 , 1 , ..., D − 1 } , where H and Ware the height and the width of the image and D denote the number of domains.
The goal of our taskis to obtain a dense interspecies face embedding E ∈ R H (cid:48)
× W (cid:48) × n of the image x d through an encoderΦ , which is described as E =
Φ( x d ) .
Here, n is the embedding dimension for dense interspecies faceembedding.
To train the encoder Φ in an unsupervised manner, we propose two training approaches:the multi-teacher knowledge distillation, and the semantic matching through pseudo-paired datasynthesis.
Since all images and labels are synthesized by the teacher models whose training data arenot suitable for our task, the training can be considered as an unsupervised learning.
In the following,we describe our dense interspecies face embedding (DIFE) and its training in detail.
3.1 Multi-Teacher Knowledge Distillation
Fig. 2(a) illustrates the overview of the multi-teacher knowledge distillation on multiple domainsconsisting of humans, dogs, and cats.
Desirable properties of the interspecies face embedding aretwo folds.
First, the representation is shared by various face images of different domains.
Faces inmultiple domains should have similar embeddings if their geometry is similar.
Second, the embeddingcontains ﬁne-grained details of facial semantics, so that different parts of a face are distinguishable.
To learn such properties, we ﬁrst train DIFE by distilling knowledge from CSE [24].
As therepresentation of CSE has high-quality co-embeddings of diverse animal species, we exploit it tolearn a common space for cross-domain faces.
Speciﬁcally, we optimize a knowledge distillation lossthat minimizes the distance between the DIFE and the CSE embedding of the same input.
Formally,the loss is described as
[Equation]
where C d is the CSE embedding of x d .
Note that x d is an image synthesized by StyleGAN2 describedas dotted arrow in Fig.
2(a).
Although the above loss enforces our embedding to learn a shared space for multiple domains offaces, the representation of CSE is coarse in the region of the face since it is originally learned todistinguish different parts of a full body.
To further learn ﬁne-grained details of faces, we add anotherknowledge distillation loss using the features of StyleGAN2.
Speciﬁcally, we exploit the semanticinformation of faces extracted from a pre-trained StyleGAN2 to enhance the representation of DIFE.
Using synthetic images generated by the StyleGAN2 as input, we minimize the distance between theembedding of the input and its corresponding StyleGAN2 feature map, which is formulated as
[Equation]
where F d is the StyleGAN2 feature map, and D d is the domain-speciﬁc embedding of x d for thedomain d .
Since the dimension of DIFE is different from that of StyleGAN2 and each feature of
StyleGAN2s is in the different domain, we convert DIFE to a domain-speciﬁc embedding usinga domain converter Ψ d , as follows: D d = Ψ d ( E d ) .
When learning from multiple domains, weoptimize L K 1 = (cid:80) Dd L dK 1 and L K
= (cid:80) Dd L
dK 2 for all of the domains.
3.2 Semantic Matching
Pseudo-paired data synthesis.
Fig.
2(b) shows how to synthesize pseudo-paired data x d (cid:48) , whichhas the identical face geometry with the original image x d .
The paired domain d (cid:48) can be differentfrom the original domain d .
From a given human face image x d generated by StyleGAN2, weﬁrst extract DIFE using the encoder Φ .
Then, with the domain converter Ψ d (cid:48) , we transform theinterspecies embedding into a domain-speciﬁc embedding D d (cid:48) .
In Fig.
2(b)
, we convert DIFE intothe dog-speciﬁc embedding.
Note that the domain-speciﬁc embedding lies in the similar vector spacewith StyleGAN2, since the domain converter is trained by the StyleGAN2 knowledge distillation.
Finally, we synthesize a pseudo-paired image x d (cid:48) by feeding the domain-speciﬁc embedding D d (cid:48)
to
StyleGAN2.
Rather than directly feeding the embedding, we introduce a latent space explorationmethod, which generates more realistic images by adjusting the latent space of the domain-speciﬁcembeddings into the natural image manifold.
In the latent space exploration, given a domain-speciﬁc embedding D d (cid:48) , the goal is to ﬁnd a latentcode w d (cid:48) ∈ W d (cid:48) , which satisﬁes the following two conditions.
Here, W d (cid:48) is an latent space of
StyleGAN2 trained on the data domain d (cid:48) .
First, the pseudo-paired data x d (cid:48) generated from the foundlatent code w d (cid:48) should have the same face geometry with the original image x d .
This is achieved byﬁnding w d (cid:48) with small difference between the StyleGAN2 feature F d (cid:48)
( w d (cid:48) )
that is generated fromw d (cid:48) , and the domain-speciﬁc embedding D d (cid:48) .
Second, the pseudo-paired data needs to be realisticand natural.
To this end, following [1], we take the mean of latent codes ¯ w , and enforce our latentcode to be similar to ¯ w .
¯ w is calculated by sampling 4096 latent codes.
With the regularization, thegenerated image looks natural by locating the latent code in the realistic image manifold.
Finally, thelatent code w d (cid:48) is found by optimizing the following equation:
[Equation]
Here, the feature map F d (cid:48) ( w ) is extracted from StyleGAN2 by feeding the latent code w to optimize.
After the optimization, the pseudo-paired data x d (cid:48) is synthesized by StyleGAN2 with w d (cid:48) .
Semantic matching loss.
The purpose of the semantic matching loss is to boost the matchingperformance of DIFE at pixel level with the synthesized pseudo-paired data.
The pixel location u ∈ Ωon 2D cartesian coordinate system Ω = { 0 , 1 , ..., H (cid:48) − 1 } × { 0 , 1 , ..., W (cid:48) − 1 } is used to explainthe loss.
Theoretically, the pixel embedding E d u at location u in DIFE of the original image has thesame semantic meaning with the corresponding pixel embedding E d (cid:48) u of the pseudo-paired image.
However, the shape difference between different domains makes it infeasible to exactly match thepixel locations.
For example, the eyes of the original image x d and the eyes of the pseudo-paired datax d (cid:48) in Fig. 2(b) do not perfectly match.
To alleviate this issue, we introduce a soft distance measure similar to [36].
Intuitively, we compare the embedding from a pixel on the original image to theweighted embedding of a small region on the pseudo-paired image, as described as the contour in
Fig. 2(b).
We ﬁnd the region to compare based on the similarity σ ( E d , E d (cid:48) , u i , u j ) between the pixelembeddings as:
[Equation]
where u i and u j are the pixel coordinates of E d and E d (cid:48) , respectively.
Finally, the semantic matchingloss L d,d (cid:48)
M is deﬁned as follows:
[Equation]
When learning from multiple domains, we optimize intra semantic matching loss L M 1 = (cid:80) Dd L d,dMand inter semantic matching loss L M 2 = (cid:80) Dd (cid:80) Dd (cid:48) (cid:54) =
d L d,d (cid:48) M for all permutations of the domains.
3.3 Training Objective
We can formulate our ﬁnal training objective with hyper-parameters as:
[Equation]
The matching losses L M 1 and L M 2 cannot be applied in the early stage of training, since the domainspeciﬁc embedding is different from the features of StyleGAN2s.
Therefore, we ﬁrst impose L K 1 and
L K 2 on the encoder and domain converter, and after initialization, we start to generate pseudo-paireddata and utilize semantic matching losses L M 1 and L M 2 .
Experiments
4.1 Interspecies Keypoint Transfer
Given an image in the source domain with keypoints, the goal of interspecies keypoint transfer isto ﬁnd the corresponding locations of the keypoints on another image in the target domain.
Wemainly evaluate our method on this task to demonstrate that our embedding is not only a high qualityrepresentation of cross-domain faces, but also beneﬁcial for knowledge transfer to unlabeled domains.
The task is basically achieved by ﬁnding the pixel-level correspondence between the embeddings ofthe source and the target images.
The performance is quantitatively evaluated by calculating the errorbetween the positions of the transferred keypoints and their corresponding ground truth.
We furtherdescribe mathematical formulations of the evaluation in detail in the supplementary material.
Datasets.
At the training time, DIFE is trained with synthetic images and their target labels generated by pre-trained CSE and StyleGAN2 networks.
We use two datasets for evaluation: MAFL [46]and AP-10K [41].
The MAFL is a dataset of human faces that is composed of 20k images.
The testset of MAFL excluding the shared images with the training set of CelebA is used (400 images).
The
AP-10K dataset is composed of 10k images of various animal species which is split into three disjointsubsets, i.e., train, validation, and test sets, with the ratio of 7:1:2 per animal species.
As the imagescontain the full-body of animals, we crop the region of their faces by computing the bounding boxusing facial landmarks in the dataset.
For the evaluation, we make pairs of two different species byrandomly selecting images from each domain.
As a result, we collect 100 pairs of humans and dogs,89 pairs of human and cats, 47 pairs of humans and wild animals, and 89 pairs of dogs and cats.
Implementation details.
We pre-train CSE with DensePose-COCO [21] and DensePose-LVIS
[10],which are the datasets for full-body keypoints of human and animal respectively.
StyleGAN2 ispre-trained with FFHQ [18] for human and AFHQ [8] for animals.
DIFE encoder uses the hourglassnetwork [25] for a fair comparison with DVE [36], and the domain converter consists of threeconsecutive 7 × 7 conv layers.
We use Adam optimizer [19] with the learning rate of 10 − 3 , batch sizeof 12, and max training step of 1M with early stopping.
The lambda values for each loss functionsare ﬁxed to λ 1 = 10 − 2 , λ 2 = 10 0 , λ 3 = 10 − 2 , and λ 4 = 10 − 2 in all experiments.
All experimentsare carried out on one NVIDIA Titan XP and our source code will be made publicly available.
Method
[Table]
[Figure]
Baselines.
We compare our method with two baselines: CSE [24] and DVE [36].
Even though
CSE is originally learned for full-body embeddings of interspecies data, we compare to the methodto demonstrate the effectiveness of our method on face images.
DVE is an unsupervised learningapproach for learning pixel-level facial embeddings, which is designed for human faces in the originalwork.
For the comparison, we train the method with interspecies data.
Both the training set of
AP-10K and synthesized images of StyleGAN2 were used, and the latter is selected for the betterperformance.
We implemented the baselines using the original source code available on their websiteswith the default hyperparameters.
Quantitative results.
Table 1 presents the quantitative results of CSE, DVE, and DIFE on humans,dogs, cats, and wild animals showing that our method achieves better performance in all data domains.
As can be seen in the table, transferring between human and other animals is more challenging dueto the larger shape differences between faces of human and animals.
Compared to other baselines,
DIFE makes a signiﬁcant improvement with the help of the semantic loss.
The errors in all methodsare lower for Dog+Cat, with DIFE performing better than other baselines.
We have also tested achallenging case of the keypoint transfer between three domains by training the models at once withhuman, dogs, and cats, in which DIFE also outperforms other baselines.
For the Human+Wild case,we utilized the model that was trained with human+cat, because it was difﬁcult to train with humanand wild animals due to the extreme shape and style variance.
Since the wild animal domain includesanimals that resemble cats, such as tigers, lions, and cheetahs, using our model trained with humanand cats performed reasonably, showing that our method is applicable for out-of-domain cases.
Qualitative results.
Fig.
3 shows some qualitative results of the keypoint transfer.
As expected,
CSE has difﬁculty in ﬁnely distinguishing different parts of faces as it has limited face prior foranimals due to the sparse annotation of DensePose-LVIS.
While DVE tends to recognize the eyesand the noses to some extent, it does not properly overcome the shape difference, showing somemismatches.
DIFE, with its ability to understand cross-domain association and face embedding, canﬁgure out exact visual correspondence on interspecies faces.
[Figure] [Figure]
[Table] [Table]
Fig.
4 visualizes the embeddings of different methods.
CSE shows similar values across domains,but less distinction between different parts of a face.
On the other hand, StyleGAN2 features exhibitdistinctiveness between different parts of a face, but corresponding parts between different domainsshow a large difference in values, making it difﬁcult to align between different domains.
DIFE makesthe best of both worlds, showing distinctive values within a face, while having similar values betweencorresponding face parts between different domains.
4.2 Ablation Study
The effect of loss functions.
Table 2 shows an ablation study to verify the role of each loss.
Pixelerrors are reduced by adding each loss
[Table]
and any absence of a loss leads to the performance drop.
Aswe cannot synthesize pseudo-paired data without the knowledge distillation of StyleGAN2, the resultfor the absence of L K 2 is not provided.
The absence of L K 1 leads to a large performance drop of
DIFE
[Table]
and we ﬁnd that the role of knowledge distillation of CSE is essential in the early stages oflearning.
The semantic matching loss also helps to increase the performance by learning more ﬁnevisual correspondence across the data domains.
Which layer in StyleGAN2 is good for the knowledge distillation?
Table 3 shows the result ofselecting a different layer of StyleGAN2 for acquiring the features.
Fig.
5 shows that the featuresfrom the higher layer is more ﬁne-grained but domain-speciﬁc, which makes the domain converterto become confused.
Also, because the dimension of the domain-speciﬁc embedding follows thedimension of StyleGAN2 features n d , selecting the ninth layer limits the representation power of thedomain embedding.
On the other hand, the feature from the lower layer is easy for the knowledgedistillation with the abundant representation power with large n d , but the information about theface is limited because of its coarseness.
With this ablation study, we selected the seventh layer of
StyleGAN2 to learn face prior of each data domain.
4.3 Applications
Image manipulation.
Fig.
6 presents image manipulation examples based on DIFE.
Putting objectslike glasses and masks requires the positions of face parts, and the result shows that DIFE ﬁnds exactlocations to put the objects onto the images, even on animal faces.
[Figure]
Pseudo-paired data synthesis.
Fig.
7 shows examples of pseudo-paired data with various datadomains and postures.
Given original images, we perform pseudo-paired data synthesis on theintra-domain and the inter-domain.
We can see that the generated paired images are in similar poseswith the original ones, while having minor texture variations.
Dense keypoint transfer.
Although DIFE extracts embeddings from all pixels, we cannotaccurately evaluate all visual correspondencesdue to the lack of annotations.
However, we canﬁgure out more dense visual correspondencequalitatively by transferring generated grid keypoints .
In Fig.
8, the result of transferring densekeypoints from a human source to another human, dog, cat, and wild animal targets can befound.
Conclusion
[Figure]
In this paper, we introduced a study for extracting common features from faces of severalspecies, including humans, as a dense embedding.
In this process, we presented a mechanism forextracting and learning desired information from separate teacher models, even though the teachermodels have different architectures and data domains.
Furthermore, we proposed a new data augmentation technique with a latent space exploration of StyleGAN2 and the semantic matching lossto overcome the extreme shape difference between species.
Various experiments and applicationswere offered for the evaluation, and our approach has been shown to be superior to other baselines.
For the future research directions, we need to deal with multiple animals more than three domainsefﬁciently, which is not focused on this paper due to absence of pre-trained StyleGAN2.
A researchon the generalization method for the out-of-domain data is needed to address the situation where datacannot be obtained or the animal species is unknown.
As the ﬁrst work to tackle the difﬁcult problemof interspecies face embedding, more follow-up studies are necessary to overcome current limitations.
Nevertheless, we believe that this work has set the stage for a new research topic of cross-domainstudy for interspecies face understanding.
Broader Impact.
This study aims to understand the interspecies face, which can be potentially extended to variousapplications.
For example, our method can assist to ﬁnd lost pets, check the health status of farmanimals through analyzing the facial expressions.
Identifying wild animals for ecological experimentsis another expected application of our method.
Moreover, the proposed method can potentially beextended to the automatic motion capture, that can be used in the ﬁlm production or content creationfor AR/VR.
We note that the studies using DIFE could potentially violate the personal privacy oranimal rights.
The studies extending DIFE should carefully consider the above potential misuses, andbring positive impacts to the society.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s
contributions and scope?
[Yes] See Abstract and Section
(b) Did you describe the limitations of your work?
[Yes] See Section
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] SeeSection
(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes] See Section
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main exper-
imental results (either in the supplemental material or as a URL)?
[Yes] The zipped code, data and README.md is provided as supplemental material
(b) Did you specify all the training details (e
g., data splits, hyperparameters, how theywere chosen)?
[Yes] See Section 4.
(c) Did you report error bars (e g., with respect to the random seed after running experiments multiple times)?
[Yes] See Appendix D.
(d) Did you include the total amount of compute and the type of resources used (e
g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] See Section 4.
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] See Appendix C.
(b) Did you mention the license of the assets?
[Yes] See Appendix C.
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes
[Table]
The pre-trained weight of DIFE is provided as supplemental material
[Table] (d) Did you discuss whether and how consent was obtained from people whose data you’re [Table]
using/curating?
[Yes] See Appendix C.
[Table] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable [Table] information or offensive content?
[Yes] See Section 6.
[Table]
If you used crowdsourcing or conducted research with human subjects...
[Table] (a) Did you include the full text of instructions given to participants and screenshots, if [Table]
applicable?
[N/A]
[Table]
(b) Did you describe any potential participant risks, with links to Institutional Review
[Table]
Board (IRB) approvals, if applicable?
[N/A]
[Table]
(c) Did you include the estimated hourly wage paid to participants and the total amount
[Table]
spent on participant compensation?
[N/A]
[Table]