Contrastive Representation Learning for Gaze Estimation
Abstract
Self-supervised learning (SSL) has become prevalent for learning representations in computer vision.
Notably, SSL exploits contrastive learning to encourage visual representations to be invariant under various image transformations.
The task of gaze estimation, on the other hand, demands not just invariance to various appearances but also equivariance to the geometric transformations.
In this work, we propose a simple contrastive representation learning framework for gaze estimation, named Gaze Contrastive Learning (GazeCLR) .
GazeCLR exploits multi-view data to promote equivariance and relies on selected data augmentation techniques that do not alter gaze directions for invariance learning.
Our experiments demonstrate the effectiveness of GazeCLR for several settings of the gaze estimation task.
Particularly, our results show that GazeCLR improves the performance of cross-domain gaze estimation and yields as high as 17 . 2% relative improvement.
Moreover, the GazeCLR framework is competitive with state-of-the-art representation learning methods for few-shot evaluation.
The code and pre-trained models are available at https://github.com/jswati31/gazeclr .
Introduction
Gaze represents the focus of human attention and serves as an essential cue for non-verbal communication.
While specialized gaze trackers can accurately measure a user’s gaze direction, there is substantial interest in gaze estimation using regular cameras.
Although, learning gaze estimation models from images is challenging and needs to transcend multiple “nuisance” characteristics such as facial features or head orientation to estimate gaze accurately.
In recent years, deep learning [1, 2, 3] has shown promising results for gaze estimation.
In part, this success stems from the availability of large-scale annotated datasets.
As a result, valuable datasets must contain a wide range of gaze directions, appearances and head poses, which is a laborious and time-consuming procedure.
Also, gaze annotations are difficult to obtain, which makes the creation of large, representative datasets challenging [4].
Therefore, methods that facilitate training with limited gaze annotations are highly desirable.
Self-supervised learning (SSL) has gained tremendous success over the past few years and emerged as a powerful tool for reducing over-reliance on human annotations [5, 6, 7].
Following a generally accepted paradigm, we consider a pre-training stage that requires no labels, followed by a fine-tuning stage using a relatively small number of labeled samples.
SSL is an effective approach for pre-training, where semantically meaningful representations are learned that can be seamlessly adapted during fine-tuning stage [8, 9, 10].
Specifically, a good pre-training would ensure that the embeddings for images associated with the same gaze direction are neighbors in the feature space, regardless of other non-relevant factors such as appearance.
Arguably, this could accelerate the job of fine-tuning, possibly reducing the number of required labeled samples.
InvariantProjectionHead Encoder
EquivariantProjectionHead
Encoder
Encoder Encoder
Pre-trained
Regressor
I. Gaze Contrastive Learning Framework
II.
Learning For Gaze Estimation
Supervision
Equivariance
Invariance (a) Two-stages of our framework
V i e w
Time (
[Figure]
Formation [Figure] positive pairs
In this work, for SSL pre-training, we focus on contrastive representation learning (CRL), which aims to map “positive" pair samples to embeddings that are close to each other, while mapping “negative" pairs apart from each other [11].
A popular approach is to generate pairs by applying two different transformations (or augmentations) to an input image forming a positive pair, and different images forming negative pairs.
This method encourages invariance in representations w.r.t.
similar types of transformations, where these transformations are assumed to model “nuisance” effects.
However, obtaining the necessary and sufficient set of “positive” and “negative” pairs remains a nontrivial and unanswered challenge for a given task.
This work attempts to answer this question for gaze estimation.
Recent CRL-based methods encourage the representations to be invariant to any image transformation, many of which are not suitable for gaze estimation.
For example, geometry-based image transformations (such as rotation) will change the gaze direction.
In contrast, it is beneficial to have invariance to appearance, e.g., a person’s identity, background, etc.
In this paper, we propose Gaze Contrastive Learning (or GazeCLR ) framework – a simple CRL-based unsupervised pre-training approach for gaze estimation, i.e., a pre-training method requiring no gaze label data.
In detail, our approach relies on invariance to image transforms (e.g., color jitter) that do not alter gaze direction, and equivariance to camera viewpoint, which requires additional information of multi-view geometry, i.e., images of the same person should be obtained at the same time by two or more cameras from different locations.
For learning equivariance , we leverage the fact that in a common reference system , two or more synchronous images of the same person from different camera viewpoints are associated with the same gaze direction.
The knowledge of the relative pose of each camera to the common reference system provides the relation of gaze directions defined in the respective camera space.
In other words, gaze direction has an equivariant relationship to camera viewpoints.
We claim that the requirement of using multiple cameras may be less onerous than obtaining gaze annotations for each image.
We use an existing multi-view gaze dataset (EVE [12]) which provides video sequences captured from four calibrated and synchronized cameras and contains gaze annotations, which are obtained using a gaze tracking device [13].
We neglect labels during pre-training and use them only for fine-tuning and evaluation.
Observe that, the relative camera pose information available with the EVE dataset is used only during pre-training stage.
Figure 1 presents an overview of the proposed idea.
To evaluate the GazeCLR , we perform self-supervised pre-training using the EVE dataset and transfer the learned representations for the gaze estimation task in various evaluation settings.
We demonstrate the effectiveness of representations by showing that the proposed method achieves superior performance on both within-dataset and cross-dataset (MPIIGaze [2] and Columbia [14]) evaluations by using only a small number of labeled samples for fine-tuning.
Our major contributions are summarized as follows:
We propose a simple contrastive learning method for gaze estimation that relies on the observation that gaze direction is invariant under selected appearance transformations and equivariant to any two camera viewpoints.
We also argue to learn equivariant representations by taking advantage of the multi-view data that can be seamlessly collected using multiple cameras.
Our empirical evaluations show that GazeCLR yields improvements for various settings of gaze estimation and is competitive with existing supervised [15] and unsupervised state-ofthe-art gaze representation learning methods [16, 17].
Proposed Method 2.1 Stage-I: Gaze Contrastive Learning ( GazeCLR ) Framework
GazeCLR is a framework to train an encoder that learns embeddings to induce desired set of invariance and equivariance for the gaze estimation task.
As stated earlier, the key intuition of GazeCLR is that we want to enforce invariance using selected appearance transformations (e.g., color jitter) and equivariance using synchronous images of the same person captured from multiple camera viewpoints.
Similar to previous SSL approaches [6, 18], we rely on the normalized temperature-scaled crossentropy loss (NT-Xent)[6] to encourage invariance or equivariance by maximizing the agreement between positive pairs and disagreement between the negative pairs.
In particular, we devise two variants of NT-Xent loss, namely, L I for invariance and L E for equivariance.
The GazeCLR framework has three sub-modules: a CNN-based encoder and two projection heads based on MLP layers, as illustrated in Figure 2.
The output of the encoder branches out into different projection heads depending on the type of input positive pair.
To abide by the invariance for gaze direction, we consider augmentations based on only appearance transformations denoted as A .
Let { I v i ,t } | V | i =1 be the synchronous frames for timestamp t coming from different camera views (i.e., { v i } | V | i =1 ), then we create following positive pairs:
Single-view positive pairs: We apply two randomly sampled augmentations from A to create a single-view positive pair.
Specifically, for any image I v i ,t , at a given timestamp t and view v i , we sample two augmentations a and a ′ from A and then ( a ( I v i ,t ) , a ′ ( I v i ,t ))
forms a positive pair to learn invariance.
The left branch of Figure 2 shows one such positive pair for view v 1 . 2.
Multi-view positive pairs: We consider all unique pairs of camera viewpoints from the same timestamp t and apply random augmentations from A , i.e., { ( a i ( I v i ,t ) , a j ( I v j ,t ))
| i, j ∈ { 1 , . . .
, | V |} | i ̸ = j } .
The corresponding outputs from the encoder are passed through projection head p 2 and multiplied by an appropriate rotation matrix to learn equivariance.
Next, to construct negative pairs, we do not sample them explicitly but use all other samples in the mini-batch as negative examples, similar to Chen et al.
The exact formulation of both loss functions L I and L E is described below.
For brevity, we omit t from I v i ,t and augmentation a in the following subsections.
Single-View Learning.
Recall, the goal of single-view learning is to induce invariance amongst representations under various appearance transformations.
Let v i ∈ V be any view and b ∈ [1 , . . .
, B ] be the batch index.
Given a batch size of B , we apply two augmentations to each sample in the batch yielding 2 B augmented images, and for each sample, we have one positive pair and (2 B − 1) negative pairs stemming from remaining samples in the batch.
Our encoder E extracts representations for all 2 B augmented images, which are further mapped by projection head p 1 ( · ) yielding embeddings ( { z b v i , z ′ b v i } B b =1 ).
With above notations, for any view v i , the proposed invariance loss function L I associated with a positive pair ( z bv i , z ′ bv i ) can be given as follows:
[Equation]
Encoder Encoder
Encoder Encoder
Encoder Encoder
Projector Projector
Projector Projector
Projector Projector
Multi-View Learning =
augmentation = rotation
where, z bv i = p 1 ( E ( I bv i )) , z ′ bv
i = p 1 ( E ( I ′ bv i )) , sim ( r, s ) =
exp (cid:18)τ r T s || r || · || s || (cid:19) , 1 [ l ̸
= b ] is an indicator function and τ is the temperature coefficient parameter.
It is worth noting that to minimize the loss in Eq. 1, it must hold that z
bv i and z ′
bv
i needs to be closer, which aligns with our goal of learning invariance to appearance transformations.
One challenge, however, is the risk of collapse (e.g., the network could simply learn each person’s identity).
To avoid this, we create mini-batches such that all samples in a batch are taken from a single participant.
Multi-View Learning.
We encourage equivariance in the gaze representations to different camera viewpoints through multi-view learning.
To do so, we transform embeddings to a common reference system, chosen as the screen reference system used during the EVE data collection.
Let { R SC vi } be the rotation matrix relating the camera viewpoint v i with the screen reference system.
For each sample I bv i in a batch of size B, the positive pair is given as ( I bv i , I bv j ) for two distinct camera viewpoints ( v i , v j ) i ̸
=
j .
All images for viewpoints v i and v j are first augmented then passed through encoder E and the projector head p 2 ( · ) which gives embeddings ˆ z
bv
i , ˆ z bv j ∈ R 3 × d ′ .
These embeddings are further multiplied by corresponding rotation matrices R SC vi to project embeddings in the common (screen) reference system.
We denote embeddings after rotation as { ¯ z bv i , ¯ z bv j } Bb [-=
 suchthat-]=1 such that ¯ z b v i
= R S C
v i ˆ z b v i .
Therefore, for a batch of size B, our equivariant loss L E associated with the positive pair (¯ z bv i , ¯ z bv j ) is as follows:
[Equation]
Overall loss function.
Given | V | camera viewpoints, we apply both L I and L E loss functions to each view.
Thus, our overall objective function for a batch size of B becomes:
[Equation]
2.2 Stage-II: Learning For Gaze Estimation
After pre-training, the encoder learned by GazeCLR framework is used for the task of gaze estimation and fine-tuned on a small labeled dataset.
To this end, we remove both projection heads p 1 and p 2 , and replace them with MLP regressor layers to predict 3D gaze direction.
For training MLP regressor, we use the supervised loss function given as
[Equation]
where ggg and ˆ g ˆ g ˆ g are the ground-truth and predicted gaze directions, respectively.
Experiments
We start by detailing the experimental setup (Sec. 3.1) followed by a brief explanation of considered baselines (Sec. 3.2).
Next, we evaluate the performance of our pre-trained encoder and show that representations from GazeCLR can help train an accurate gaze estimation model even with a relatively lesser amount of annotations.
For this task, we consider the within-dataset setting (Sec. 3.3).
We assess the transferable capability of our representations by evaluating them on different domains in linear layer training (frozen encoder) setting, where we considered only a few calibration samples from the test subject, as detailed in Sec. 3.4.
Thereafter, we compare GazeCLR with existing supervised [15] and unsupervised [16, 17] pre-training methods in Sec. 3.5.
Lastly, we probe the semantics of learned GazeCLR representations using a well-known t-SNE visualization technique (in
Sec. 3.6).
3.1 Setup
We train our GazeCLR framework on the EVE [12] dataset, which has videos collected in a constrained indoor setting with four different synchronized and calibrated camera views.
It has approximately 12 million frames collected from 54 participants with natural eye movements.
Following the splits considered by Park et al.
[12], there are 40 subjects in training and 6 subjects in the validation set.
We discard the data of test subjects due to the non-availability of labels.
We use training subjects for the pre-training stage, without using any gaze annotations.
For the gaze estimation stage, we evaluate on the data of validation subjects to report the performance.
We use all four camera views (i.e., | V | = 4 ) as well as the information about the relative pose between camera and screen ( R SC ) provided with the
EVE dataset.
Note that our framework can be extended to more number of camera views ( | V | > 4 ) using ETH-XGaze [19] dataset. In this paper, we consider pre-training only on EVE dataset as more views add on increased computational demand.
All experiments use ResNet-18 [20] as the encoder network and take the output from the average pooling layer.
The encoder is trained from scratch.
Following Chen et al.
[6], both projection heads p 1 ( · ) and p 2 ( · ) are two-layer MLP networks with ReLU non-linearity.
The output dimensions for the first and second layers are 512 and 180 , respectively.
The input image size is 128 × 128 .
Training details. GazeCLR is trained using SGD optimizer with initial learning rate = 0 . 03 , momentum = 0 . 9 , and cosine annealing [21] for the learning rate decay. We use a single
GeForce GTX GPU for training, with a batch size of 128, and train for 50K iterations.
Our mini-batch is made up of samples from a single participant. The temperature coefficient τ is set to 0 . For the augmentation transformations A , we apply random spatial cropping and resizing, blur, color perturbation ( p = 0 . 8 ) on brightness, contrast, saturation and hue, grayscale conversion ( p and auto-contrast
Data pre-processing.
We use face images available in the EVE dataset, obtained after applying a data-normalization procedure [22, 23].
The normalization pipeline transforms the gaze annotation to a normalized camera space through a rotation matrix M .
Note that we post-multiply R SC with M −as R SC is defined w.r.t.
the original camera reference frame, i.e., ¯ z v =
R SC v ( M ) − 1 ˆ z v .
We train GazeCLR framework in two different settings: (i) GazeCLR (Equiv) : where we only consider equivariance through the loss function L E and (ii) GazeCLR (Inv+Equiv) : where we considered both invariance and equivariance with equal weights using the overall objective L O .
We present the performance of both training setups in all the considered experimental settings.
Observe that,
GazeCLR (Inv) trained with only L I loss function is equivalent to SimCLR [6] baseline method.
3.2 Baselines
We compare our approach with six following baselines: (i) w/o Pre-training , i.e., an encoder is initialized using random weights, (ii) the vanilla Autoencoder , which has an encoder network that consists of same encoder layers as GazeCLR and five DenseNet [24] deconvolution blocks as decoder, and is trained with L2 loss, (iii) Novel View Synthesis [25] framework is trained on our dataset using same architecture as the auto-encoder, (iv) BYOL [7], (v) SimCLR [6] and (vi) Fully-Supervised is a
ResNet-18 model trained on the whole EVE training data and represents possibly an upper bound for the performance of GazeCLR .
For SimCLR and BYOL, we use the same augmentation set as in our proposed method.
3.3 Within-dataset Evaluation
For within-dataset evaluation, we perform pre-training on the training split of the EVE dataset without using labels.
Then we adapt the pre-trained encoder for the gaze estimation on a small subset of labeled data.
Precisely, we took five training subjects out of 40 (which form around 10% samples out of the whole EVE dataset) for the supervised gaze estimation stage and called it “MiniEVE”.
We validate on a fixed subject data chosen from training subjects and report the final performance for validation subjects.
Table 1 shows the mean angular errors (in degrees) obtained for different pre-training baselines and the proposed GazeCLR method.
To this end, we freeze the pre-trained encoder and simply train an MLP regressor using the “MiniEVE” dataset.
Note that, for two baselines, Autoencoder and
BYOL, we fine-tune the whole end-to-end framework along with the encoder as otherwise, they fail to converge when only their representations are used.
We indicate this behavior in Table 1, using the
Frozen column as ✓ if encoder is frozen otherwise as ✗ .
We observe that our method GazeCLR outperforms other pre-training baseline methods by only training an MLP regressor on the small amount of labeled data (“MiniEVE” is ∼ 10% of whole data).
Specifically, it can be seen that the performance achieved from GazeCLR helps in closing the gap with the fully-supervised baseline.
Our method GazeCLR (Inv+Equiv) shows a relative improvement of 25 .
1% compared to popular contrastive learning method SimCLR [6].
Additionally, GazeCLR (Equiv) shows a boost of 26 .
4% relative improvement over the SimCLR approach, suggesting that equivariant representations are very effective for the gaze estimation task.
We hypothesize that since we utilize similar augmentation strategies for creating both single-view and multi-view positive pairs,
GazeCLR (Equiv) performs almost comparable to GazeCLR (Inv+Equiv) .
3.4 Transfer Learning/Cross-dataset Evaluation
We perform a cross-dataset evaluation using a few-shot personalized gaze estimation to further demonstrate the cross-data generalization capabilities of the learned representations.
We evaluate
GazeCLR representations on two domain datasets different from pre-training data: MPIIGaze [1] and
Columbia [14].
MPIIGaze is a challenging dataset that has higher inter-subject variations.
We use the standard evaluation subset MPIIFaceGaze [2] containing around 37667 images captured fromsubjects.
The Columbia dataset consists of 5880 images collected from 56 subjects and is known to have high head pose variations.
To measure the quality of learned representations, we use Linear Layer Training (LLT) protocol, in which we freeze the trained encoder and learn a linear regressor on the target dataset.
For this [Table] experiment, we investigate under a few-shot setting where we sample a few calibration samples from the test subject for adaptation [Table]
and evaluate on the remaining samples of the same test subject.
Figure 3 shows the mean angular errors for LLT protocol on 20-shot, 50-shot, and 64-shot gaze estimation.
We first extract the gaze representations of a few calibration samples for each subject and learn a linear model on top of these representations.
We evaluate the trained model on the remaining samples of the subject.
We repeat above 10 times for each subject on both datasets and report mean angular errors for the same in Figure 3.
Observe that both proposed GazeCLR variants outperform all other baselines in all few-shot settings for both datasets.
Moreover, GazeCLR(Equiv) gives an relative improvement of around 17 .
2% over
SimCLR with only 20 calibration samples for Columbia.
We hypothesize that this behavior is due to high head-pose variations within Columbia and it suggests that: a) learning equivariance over multi-views is beneficial for the GazeCLR framework in improving performance, and b) GazeCLR representations are relatively more generalizable for cross-domain datasets than other baselines.
3.5 Comparison with state-of-the-art gaze representation learning
We further compare GazeCLR with existing state-of-the-art unsupervised [16, 17] and supervised [15] gaze representation learning methods.
For a fair comparison, we adopt the same evaluation protocols as used by these baseline methods and compare the GazeCLR performance against their performance.
GazeCLR vs Unsupervised Pre-training [16, 17].
We follow the same evaluation protocol as [16]. 5-fold and leave-one-out (15-fold) evaluations are used for the Columbia and MPIIGaze datasets, respectively.
In each fold, we freeze GazeCLR encoder and extract representations for randomly selected 50 samples with annotations and learn a simple MLP-based gaze estimator on top of that.
We repeat performance evaluation for 10 times and report mean angular errors in Table 2.
Note that [Table] previous methods [16, 17] exploit left and right eye patches to get SSL signal, whereas our approach relies on face patches obtained from multiple camera viewpoints.
In Table 2, we compare against the best-performing models of Yu et al.
[16] and Sun et al.
[17], for the 50-shot gaze estimation.
Notice that our method outperforms baselines with absolute improvements of 2 ◦ and 0 .
9 ◦ on MPIIGaze and Columbia, respectively.
It is worth emphasizing that our method is pre-trained on a different dataset than both evaluation datasets, unlike baseline approaches.
Again, it illustrates the strength of our approach in learning semantically meaningful representations for generalizable to other domains.
Moreover, note that Yu et al.
[16] use additional head-pose information, unlike our method.
GazeCLR vs Supervised Pre-training [15].
We evaluate the effectiveness of GazeCLR representations using the MAML framework [26], similar to FAZE [15].
For both GazeCLR and FAZE, we train a MAML-based gaze estimator on the representations for subjects from the GazeCapture [3] dataset.
Then, we adapt the gaze estimator model to each test subject of Columbia with k calibration samples and test on the remaining samples.
Figure 4 depicts the performance comparison of GazeCLR with
FAZE [15] for four different values of k .
It can be seen that our method consistently outperforms supervised pre-training baseline FAZE, for all values of k .
Notably, our framework uses zero labeled information to obtain gaze representations, unlike FAZE, which is pre-trained using ∼ 2 M labeled samples from the GazeCapture dataset.
3.6 Visualization of Gaze Representations
To further investigate the quality of learned representations, we project the gaze representations into 2-dimensions using t-SNE [27] algorithm as shown in Figure 5.
In Fig 5(a), we compute 2D visualization of equivariant representations obtained after applying rotation matrices, i.e., ¯ z .
[Figure] Projections in Fig 5(a) clearly demonstrate that gaze direction is invariant to the viewpoint, as images at the same timestamp from different views are mapped closer (shown with the same color border).
In Fig 5(b), we apply t-SNE algorithm on gaze representations obtained at output of encoder network, i.e., z = E ( · ) , for images from single camera viewpoint.
Projections corresponding to roughly similar gaze directions are naturally clustered and highlighted with different background colors.
Also, we observe clear patterns in the learned feature space where images within close vicinity are invariant to the subject’s identity, showing invariance towards appearances.
Related Work
Gaze Estimation.
Gaze estimation methods are built using large-scale datasets either having 2D target labels [3, 28] or 3D gaze directions [29, 30, 1].
Broadly, gaze estimation methods can be divided into two categories: appearance methods [31], which directly map image pixels to 3D gaze direction, and model methods which rely on eye-geometry [32].
Appearance methods perform better than traditional model methods in real-world settings [32, 1].
Recent progress in appearance methods relies heavily on deep learning to map eye/face images to gaze directions [1, 2].
Furthermore, a few gaze methods are hybrid.
For instance, Park et al.
[33] extracted the relevant eye landmarks from the images and then used these features to train model gaze estimators.
Other than eye images, several works [3, 34, 35, 36] exploit both eyes and face images in computing gaze direction.
Nevertheless, both appearance methods and hybrid extensions require a huge amount of labeled data to achieve their potential in terms of accuracy.
As a result of huge label dependence by appearance methods, efforts have been made in the direction of few-shot gaze estimation.
In particular, Liu et al.
[37] exploit a two-branch network to predict differential gaze between two images and use few calibration samples during inference.
Furthermore, [15, 38] disentangle gaze from other nuisance factors via training an encoder-decoder architecture [39] to learn gaze specific representations.
Recent approaches [40, 41, 42] leverage labeled source domain and unsupervised domain adaptation for improving the performance of gaze estimation task on target domain.
In a similar spirit, our work attempts to reduce the amount of required label information via learning gaze representations without relying on gaze labels and utilizing multi-view data [12, 19].
Self-Supervised Learning.
The goal of self-supervised representation learning is to learn good visual representations from a large collection of unlabeled images.
Earlier works in SSL [43, 44, 45, 46] used pretext tasks to learn generalizable semantic representations.
Some of the recent works [47, 5, 6, 48, 49, 50, 7] have shown a great success on several vision tasks, e.g, image classification [8, 51], object detection [9], semantic segmentation [10], and pose estimation [25].
The recent work by Spurr et al.
[18] extends SSL to hand pose estimation through geometric equivariance representations.
Tian et al.
[52] propose to use more than two views to learn invariant representations through contrastive learning.
Recently, a few unsupervised methods have been proposed to learn gaze-specific representations.
Specifically, Yu et al.
[16] exploit the gaze redirection task to train a gaze estimation model using paired eye images of the same subject.
Similarly, Sun et al.
[17] proposed a cross-encoder method to utilize patches of left and right eye images of the same subject as the self-supervised signal.
Gideon et al.
[53] is an extended version of [17] utilizing multi-view images and learning features representing head pose and relative gaze to improve in-domain few-shot gaze estimation performance.
However, unlike our work, these methods employ an encoder-decoder framework and thus require a relatively large number of parameters.
Also, contrastive SSL approaches are computationally efficient compared to generative SSL approaches [54].
Conclusion
We presented GazeCLR , a contrastive learning framework for gaze representations using multi-view camera images.
Our framework induces invariance and equivariance properties simultaneously in the learned representations and is effective for gaze estimation task in various settings.
Furthermore, we showed that GazeCLR representations have the potential to be effective across different domain datasets using only a few calibration samples.
GazeCLR is a general framework for equivariant representation learning, thus can be explored in the future for other geometry-based applications such as head pose estimation.
Acknowledgments and Disclosure of Funding
This research was supported by the National Eye Institute of the National Institutes of Health (NIH) under award number R01EY030952-01A1.
The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
We thank Mohit
[-Given a specific timestamp in a video,let two samples from different camera viewpoints with gaze directions be g v 
-]
[-We use a single 
-]
Yadav for helpful discussions.