Original: CLIP-F LOW : C ONTRASTIVE L EARNING BY SEMISUPERVISED I TERATIVE P SEUDO LABELING FOR O PTICAL F LOWE STIMATION
Revised: CLIP-F LOW : C ONTRASTIVE L EARNING WITH I TERATIVE P SEUDO LABELING FOR O PTICAL F LOW

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Synthetic datasets are often used to pretrain end-to-end optical flow networks, due to the lack of a large amount of labeled, real scene data.
Revised: Synthetic datasets are often used to pretrain end-to-end optical flow networks, due to the lack of a large amount of labeled real scene data.

Original: But major drops in accuracy occur when moving from synthetic to real scenes.
Revised: But major drops in accuracy occur when moving from synthetic to real scenes.

Original: How do we better transfer the knowledge learned from synthetic to real domains?
Revised: How do we better transfer the knowledge learned from synthetic to real domains?

Original: To this end, we propose
Revised: To this end, we propose

Original: CLIP-Flow, a semi-supervised iterative pseudo labeling framework to transfer the pretraining knowledge to the target real domain.
Revised: CLIP-Flow, a semi-supervised iterative pseudo labeling framework to transfer the pretraining knowledge to the target real domain.

Original: We leverage large-scale, unlabeled real data to facilitate transfer learning with the supervision of iteratively updated pseudo ground truth labels, bridging the domain gap between the synthetic and the real.
Revised: We leverage large-scale, unlabeled real data to facilitate transfer learning with the supervision of iteratively updated pseudo ground truth labels, bridging the domain gap between the synthetic and the real.

Original: In addition, we propose a contrastive flow loss on reference features and the warped features by pseudo ground truth flows, to further boost the accurate matching and dampen the mismatching due to motion, occlusion, or noisy pseudo labels.
Revised: In addition, we propose a contrastive flow loss on reference features and the warped features by pseudo ground truth flows, to further boost the accurate matching and dampen the mismatching due to motion, occlusion, or noisy pseudo labels.

Original: We adopt RAFT as backbone and obtain an F1-all error of 4.11%, i.e. a 19% error reduction from RAFT (5.10%) and ranking 2 nd place at submission on KITTI 2015 benchmark.
Revised: We adopt RAFT as backbone and obtain an F1-all error of 4.11%, i.e. a 19% error reduction from RAFT (5.10%) and ranking 2 nd place at submission on KITTI 2015 benchmark.

Original: Our framework can also be extended to other models, e.g. CRAFT, reducing the F1-all error from 4.79% to 4.66% on
Revised: Our framework can also be extended to other models, e.g. CRAFT, reducing the F1-all error from 4.79% to 4.66% on

Original: KITTI 2015 benchmark.
Revised: KITTI 2015 benchmark.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Optical flow is critical in many high level vision problems, such as action recognition (Simonyan &
Revised: Optical flow is critical in many high level vision problems, such as action recognition (Simonyan &

Original: Zisserman, 2014; Sevilla-Lara et al., 2018; Sun et al., 2018b), video segmentation (Yang et al., 2021;
Revised: Zisserman, 2014; Sevilla-Lara et al., 2018; Sun et al., 2018b), video segmentation (Yang et al., 2021;

Original: Yang & Ramanan, 2021) and editing (Bonneel et al., 2015), autonomous driving (Janai et al., 2020) and so on.
Revised: Yang & Ramanan, 2021) and editing (Bonneel et al., 2015), autonomous driving (Janai et al., 2020) and so on.

Original: Traditional methods (Horn & Schunck, 1981; Menze et al., 2015; Ranftl et al., 2014;
Revised: Traditional methods (Horn & Schunck, 1981; Menze et al., 2015; Ranftl et al., 2014;

Original: Zach et al., 2007) mainly focus on formulating flow estimation as solving optimization problems using hand-crafted features.
Revised: Zach et al., 2007) mainly focus on formulating flow estimation as solving optimization problems using hand-crafted features.

Original: The optimization is searched over the space of dense displacement fields between a pair of input images, which is often time-consuming.
Revised: The optimization is searched over the space of dense displacement fields between a pair of input images, which is often time-consuming.

Original: Recently, data driven deep learning methods Dosovitskiy et al.
Revised: Recently, data driven deep learning methods Dosovitskiy et al.

Original: ; Ilg et al.
Revised: ; Ilg et al.

Original: Teed & Deng (2020) have been proved successful in estimating the optical flow thanks to the availability of all kinds of high quality synthetic datasets (Butler et al., 2012b; Dosovitskiy et al., 2015; Mayer et al., 2016; Krispin et al., 2016).
Revised: Teed & Deng (2020) have been proved successful in estimating the optical flow thanks to the availability of all kinds of high quality synthetic datasets (Butler et al., 2012b; Dosovitskiy et al., 2015; Mayer et al., 2016; Krispin et al., 2016).

Original: However, most of the recent works (Dosovitskiy et al., 2015; Ilg et al., 2017; Teed & Deng, 2020; Jeong et al., 2022) mainly train on the synthetic datasets given that there is no sufficient real labeled optical flow datasets to be used to train a deep learning model.
Revised: Most of the recent works (Dosovitskiy et al., 2015; Ilg et al., 2017; Teed & Deng, 2020; Jeong et al., 2022) mainly train on the synthetic datasets given that there is no sufficient real labeled optical flow datasets to be used to train a deep learning model.

Original: State-of-the-art (SOTA) models always get more accurate results on the synthetic dataset like Sintel (Butler et al., 2012a) than the real scene dataset like KITTI 2015 (Menze & Geiger, 2015).
Revised: State-of-the-art (SOTA) models always get more accurate results on the synthetic dataset like Sintel (Butler et al., 2012a) than the real scene dataset like KITTI 2015 (Menze & Geiger, 2015).

Original: This is mainly because that the model tends to overfit the small training data, which echos in Tab. 1, i.e. , there is a big gap between the training F1-all error and test F1-all error when train and test on the KITTI dataset in all of the previous SOTA methods.
Revised: This is mainly because that the model tends to overfit the small training data, which echos in Tab. 1, i.e. , there is a big gap between the training F1-all error and test F1-all error when train and test on the KITTI dataset in all of the previous SOTA methods.

Original: Therefore, we argue that this gap in performance is because of dearth of real training data, and a big distribution gap between the synthetic data and real scene data.
Revised: Therefore, we argue that this gap in performance is because of dearth of real training data, and a big distribution gap between the synthetic data and real scene data.

Original: Although the model can perfectly explain all kinds of synthetic data, however, when dealing with real data, it performs rather unsatisfactorily.
Revised: Although the model can perfectly explain all kinds of synthetic data, however, when dealing with real data, it performs rather unsatisfactorily.

Original: Our proposed work focuses on bridging the glaring performance gap between the synthetic data and the real scene data.
Revised: Our proposed work focuses on bridging the glaring performance gap between the synthetic data and the real scene data.

Original: As in previous data driven approaches, smarter and longer training strategies prove to be beneficial and helps in obtaining better optical flow results.
Revised: As in previous data driven approaches, smarter and longer training strategies prove to be beneficial and helps in obtaining better optical flow results.

Original: Through our work we also try to find answers of the following two questions: (i) How to take advantage of the current SOTA optical flow models to further consolidate gain on real datasets?
Revised: Through our work we also try to find answers of the following two questions: (i) How to take advantage of the current SOTA optical flow models to further consolidate gain on real datasets?

Original: and (ii) How can we use semi-supervised learning along with contrastive feature representation learning strategies to effectively utilize the huge amount of unlabeled real data at our disposal?
Revised: and (ii) How can we use semi-supervised learning along with contrastive feature representation learning strategies to effectively utilize the huge amount of unlabeled real data at our disposal?

Original: Unsupervised visual representation learning (He et al., 2020; Chen et al., 2020b) has proved successful in boosting most of major vision related tasks like image classification, object detection and semantic segmentation to name a few.
Revised: Unsupervised visual representation learning (He et al., 2020; Chen et al., 2020b) has proved successful in boosting most of major vision related tasks like image classification, object detection and semantic segmentation to name a few.

Original: Work such as (He et al., 2020; Chen et al., 2020b) also emphasizes the importance of the contrastive loss, when dealing with huge dense dataset.
Revised: Work such as (He et al., 2020; Chen et al., 2020b) also emphasizes the importance of the contrastive loss, when dealing with huge dense dataset.

Original: Given that optical flow tasks generally lack real ground truth labels, we ask if leveraging the unsupervised visual representation learning boosts optical flow performance?
Revised: Given that optical flow tasks generally lack real ground truth labels, we ask if leveraging the unsupervised visual representation learning boosts optical flow performance?

Original: In order to answer this question, we examine the impact of contrastive learning and pseudo labeling during training under a semisupervised setting.
Revised: In order to answer this question, we examine the impact of contrastive learning and pseudo labeling during training under a semisupervised setting.

Original: We particularly conduct exhaustive experiments using KITTI-Raw Geiger et al.
Revised: We particularly conduct exhaustive experiments using KITTI-Raw Geiger et al.

Original: (2013) and KITTI 2015 (Menze & Geiger, 2015)
Revised: (2013) and KITTI 2015 (Menze & Geiger, 2015)

Original: datasets to evaluate its performance gain, and show encouraging results.
Revised: datasets to evaluate its performance gain, and show encouraging results.

Original: We believe that gain seen in model’s performance is reflective of the fact that employing representation learning techniques such as contrasting learning helps in achieving a much more refined 4D cost correlation volume.
Revised: We believe that gain seen in model’s performance is reflective of the fact that employing representation learning techniques such as contrasting learning helps in achieving a much more refined 4D cost correlation volume.

Original: To constrain the flow per pixel, we employ a simple positional encoding of 2D cartesian coordinates on the input frames as suggested in (Liu et al., 2018), which further consolidates the gain achieved by contrastive learning.
Revised: To constrain the flow per pixel, we employ a simple positional encoding of 2D cartesian coordinates on the input frames as suggested in (Liu et al., 2018), which further consolidates the gain achieved by contrastive learning.

Original: We follow this up with an iterative-flow-refinement training using pseudo labeling which further consolidates on the previous gains to give us SOTA results.
Revised: We follow this up with an iterative-flow-refinement training using pseudo labeling which further consolidates on the previous gains to give us SOTA results.

Original: At this point we would also like to highlight that we follow a specific and well calibrated training strategy to fully exploit the gains of our method.
Revised: At this point we would also like to highlight that we follow a specific and well calibrated training strategy to fully exploit the gains of our method.

Original: Without loss of generality, we use RAFT (Teed & Deng, 2020) as the backbone network for our experiments.
Revised: Without loss of generality, we use RAFT (Teed & Deng, 2020) as the backbone network for our experiments.

Original: To fairly compare our method with existing SOTA methods, we tested our proposed method on the
Revised: To fairly compare our method with existing SOTA methods, we tested our proposed method on the

Original: KITTI 2015 test dataset, and we achieve the best F1-all error score among all the published methods by a significant margin.
Revised: KITTI 2015 test dataset, and we achieve the best F1-all error score among all the published methods by a significant margin.

Original: To summarize our main contributions: 1) We provide a detailed training strategy, which uses SSL methods on top of the well known RAFT model to improve SOTA performance for optical flow estimation.
Revised: To summarize our main contributions: 1) We provide a detailed training strategy, which uses SSL methods on top of the well known RAFT model to improve SOTA performance for optical flow estimation.

Original: We present the ways to employ contrastive learning and pseudo labeling effectively and intelligently, such that both jointly help in improving upon existing benchmarks.
Revised: We present the ways to employ contrastive learning and pseudo labeling effectively and intelligently, such that both jointly help in improving upon existing benchmarks.

Original: We discuss the positive impact of a simple 2D positional encoding, which benefits flow training both for Sintel and KITTI 2015 datasets.
Revised: We discuss the positive impact of a simple 2D positional encoding, which benefits flow training both for Sintel and KITTI 2015 datasets.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Optical flow estimation.
Revised: Optical flow estimation.

Original: Maximizing visual similarity between neighboring frames by formulating the problem as an energy minimization (Black & Anandan, 1993; Bruhn et al., 2005; Sun et al., 2014) has been the primary approach for optical flow estimation.
Revised: Maximizing visual similarity between neighboring frames by formulating the problem as an energy minimization (Black & Anandan, 1993; Bruhn et al., 2005; Sun et al., 2014) has been the primary approach for optical flow estimation.

Original: Previous works such as (Dosovitskiy et al., 2015;
Revised: Previous works such as (Dosovitskiy et al., 2015;

Original: Ilg et al., 2017; Ranjan & Black, 2017; Sun et al., 2018a; 2019;
Revised: Ilg et al., 2017; Ranjan & Black, 2017; Sun et al., 2018a; 2019;

Original: Hui et al., 2018; 2020; Zou et al., 2018) have successfully established efficacy of deep neural networks in estimating optical flow both under supervised and self-supervised settings.
Revised: Hui et al., 2018; 2020; Zou et al., 2018) have successfully established efficacy of deep neural networks in estimating optical flow both under supervised and self-supervised settings.

Original: Iterative improvement in model architecture and better regularization terms has been primarily responsible for achieving better results.
Revised: Iterative improvement in model architecture and better regularization terms has been primarily responsible for achieving better results.

Original: But, most of these works fail to better handle occlusion, small fast-moving objects, capture global motion and rectify and recover from early mistakes.
Revised: But, most of these works fail to better handle occlusion, small fast-moving objects, capture global motion and rectify and recover from early mistakes.

Original: To overcome these limitations, Teed & Deng (2020) proposed RAFT, which adopts a learning-tooptimize strategy using a recurrent GRU-based decoder to iteratively update a flow field f which is initialized at zero.
Revised: To overcome these limitations, Teed & Deng (2020) proposed RAFT, which adopts a learning-tooptimize strategy using a recurrent GRU-based decoder to iteratively update a flow field f which is initialized at zero.

Original: Inspired by the success of RAFT, there has been a number of variants such as CRAFT (Sui et al., 2022), GMA (Jiang et al., 2021), Sparse volume RAFT (Jiang et al., 2021) and FlowFormer (Huang et al., 2022), all of which benefits from all-pair correlation volume way of estimating optical flow.
Revised: Inspired by the success of RAFT, there has been a number of variants such as CRAFT (Sui et al., 2022), GMA (Jiang et al., 2021), Sparse volume RAFT (Jiang et al., 2021) and FlowFormer (Huang et al., 2022), all of which benefits from all-pair correlation volume way of estimating optical flow.

Original: The current state-of-the-art work RAFT-OCTC (Jeong et al., 2022) also uses RAFT based architecture, and it imposes consistency based on various proxy tasks to improve flow estimation.
Revised: The current state-of-the-art work RAFT-OCTC (Jeong et al., 2022) also uses RAFT based architecture, and it imposes consistency based on various proxy tasks to improve flow estimation.

Original: Considering RAFT’s effectiveness, generalizability and relatively smaller model size, we adopt RAFT (Teed & Deng, 2020) as our base architecture and employ semi-supervised iterative pseudo labeling together with the contrastive flow loss, to achieve a state-of-the-arts result on KITTI 2015 (Menze & Geiger, 2015) benchmark.
Revised: Considering RAFT’s effectiveness, generalizability and relatively smaller model size, we adopt RAFT (Teed & Deng, 2020) as our base architecture and employ semi-supervised iterative pseudo labeling together with the contrastive flow loss, to achieve a state-of-the-arts result on KITTI 2015 (Menze & Geiger, 2015) benchmark.

Original: Semi-Supervised and Representation Learning.
Revised: Semi-Supervised and Representation Learning

Original: Semi-supervised learning (SSL) and representation learning have shown success for a range of computer vision tasks, both during the pretext task training and during specific downstream tasks.
Revised: Semi-supervised learning (SSL) and representation learning have shown success for a range of computer vision tasks, both during the pretext task training and during specific downstream tasks.

Original: Most of these methods leverage contrastive learning (Chen et al., 2020a;b; He et al., 2020), clustering (Caron et al., 2020) and pseudo-labeling (Caron et al., 2021;
Revised: Most of these methods leverage contrastive learning (Chen et al., 2020a;b; He et al., 2020), clustering (Caron et al., 2020) and pseudo-labeling (Caron et al., 2021;

Original: Chen & He, 2021; Grill et al., 2020; Hoyer et al., 2021) as an enforcing mechanism.
Revised: Chen & He, 2021; Grill et al., 2020; Hoyer et al., 2021) as an enforcing mechanism.

Original: Recent works such as (Caron et al., 2020; 2021;
Revised: Recent works such as (Caron et al., 2020; 2021;

Original: Chen et al., 2020a;b; 2021; Grill et al., 2020;
Revised: Chen et al., 2020a;b; 2021; Grill et al., 2020;

Original: He et al., 2020; Xie et al., 2021b; Yun et al., 2022) have empirically shown benefits of SSL for downstream tasks such as image classification, object detection, instance and semantic segmentation.
Revised: He et al., 2020; Xie et al., 2021b; Yun et al., 2022) have empirically shown benefits of SSL for downstream tasks such as image classification, object detection, instance and semantic segmentation.

Original: These works leverage contrastive loss in different shapes and forms to facilitate better representation learning.
Revised: These works leverage contrastive loss in different shapes and forms to facilitate better representation learning.

Original: For example, (He et al., 2020; Chen et al., 2020b) advocate for finding positive and negative keys with respect to a given encoded query to enforce a contrastive loss.
Revised: For example, (He et al., 2020; Chen et al., 2020b) advocate for finding positive and negative keys with respect to a given encoded query to enforce a contrastive loss.

Original: On similar lines, for dense prediction tasks, studies such as (O Pinheiro et al., 2020; Xiao et al., 2021; Xie et al., 2021a) enforce matching overlapping regions between two augmented images, where as (Yun et al., 2022) looks to form a positive/negative pair between adjacent patches.
Revised: On similar lines, for dense prediction tasks, studies such as (O Pinheiro et al., 2020; Xiao et al., 2021; Xie et al., 2021a) enforce matching overlapping regions between two augmented images, where as (Yun et al., 2022) looks to form a positive/negative pair between adjacent patches.

Original: As part of our approach we use contrastive flow loss between features of the neighboring frames, where we draw a one-to-one positive pair relations between the reference features and the warped features using (pseudo) ground truth flow or flow estimates, as shown in Fig.
Revised: As part of our approach we use contrastive flow loss between features of the neighboring frames, where we draw a one-to-one positive pair relations between the reference features and the warped features using (pseudo) ground truth flow or flow estimates, as shown in Fig.

Original: Pseudo labeling is another important approach in SSL training paradigm.
Revised: Pseudo labeling is another important approach in SSL training paradigm.

Original: Some studies leverage pseudo labeling to generate training labels as part of consistency training (Yun et al., 2019; Olsson et al., 2021; Hoyer et al., 2021), while other works propose to use pseudo labeling to improve training pretext task training as in (Caron et al., 2021; Chen & He, 2021;
Revised: Some studies leverage pseudo labeling to generate training labels as part of consistency training (Yun et al., 2019; Olsson et al., 2021; Hoyer et al., 2021), while other works propose to use pseudo labeling to improve training pretext task training as in (Caron et al., 2021; Chen & He, 2021;

Original: Grill et al., 2020).
Revised: Grill et al., 2020).

Original: Rather, we use pseudo labeling for an iterative refinement mechanism, through which we effectively distill the correct flow estimate for the KITTI-Raw dataset (Geiger et al., 2013).
Revised: Rather, we use pseudo labeling for an iterative refinement mechanism, through which we effectively distill the correct flow estimate for the KITTI-Raw dataset (Geiger et al., 2013).

Original: To the best of our knowledge, ours is the first work which leverages both contrastive learning and pseudo labeling for estimating optical flow in an SSL fashion.
Revised: To the best of our knowledge, ours is the first work which leverages both contrastive learning and pseudo labeling for estimating optical flow in an SSL fashion.

Original:  A PPROACH
Revised:  A PPROACH

Original: In this section, we describe our method CLIP-Flow, a semi-supervised framework for optical flow estimation by iterative pseudo labeling and contrastive flow loss.
Revised: In this section, we describe our method CLIP-Flow, a semi-supervised framework for optical flow estimation by iterative pseudo labeling and contrastive flow loss.

Original: Based on two SOTA optical flow networks RAFT (Teed & Deng, 2020) and CRAFT (Sui et al., 2022) as backbone ( c.f. Sec.3.1), we obtain non-trivial improvement by leveraging our iterative pseudo labeling (PL) ( c.f. Sec. 3.2) and the proposed contrastive flow loss ( c.f. Sec. 3.3).
Revised: Based on two SOTA optical flow networks RAFT (Teed & Deng, 2020) and CRAFT (Sui et al., 2022) as backbone ( c.f. Sec.3.1), we obtain non-trivial improvement by leveraging our iterative pseudo labeling (PL) ( c.f. Sec. 3.2) and the proposed contrastive flow loss ( c.f. Sec. 3.3).

Original: It should be noted that our CLIP-Flow can be easily extended to other optical flow networks, e.g. FlowNet (Dosovitskiy et al., 2015; Ilg et al., 2017), SpyNet (Ranjan & Black, 2017) and PWC-Net (Sun et al., 2018a), with little modification.
Revised: It should be noted that our CLIP-Flow can be easily extended to other optical flow networks, e.g. FlowNet (Dosovitskiy et al., 2015; Ilg et al., 2017), SpyNet (Ranjan & Black, 2017) and PWC-Net (Sun et al., 2018a), with little modification.

Original: 3. P RELIMINARIES
Revised: 3. P RELIMINARIES

Original: Given two consecutive RGB images I 1 , I 2 ∈ R H
Revised: Given two consecutive RGB images I 1 , I 2 ∈ R H

Original: × W × 3 , the optical flow f ∈ R
Revised: × W × 3 , the optical flow f ∈ R

Original: H
Revised: H

Original: × W × 2 is defined as a dense 2D motion field f
Revised: × W × 2 is defined as a dense 2D motion field f

Original: =
Revised: =

Original: ( f u , f v ) , which maps each pixel ( u, v ) in I 1 to its counterpart ( u ′ , v ′ )
Revised: ( f u , f v ) , which maps each pixel ( u, v ) in I 1 to its counterpart ( u ′ , v ′ )

Original: in I 2 , with u ′ =
Revised: in I 2 , with u ′ =

Original: u + f u
Revised: u + f u

Original: and v ′ =
Revised: and v ′ =

Original: v + f v .
Revised: v + f v .

Original: RAFT.
Revised: RAFT.

Original: Among end-to-end optical flow methods (Dosovitskiy et al., 2015;
Revised: Among end-to-end optical flow methods (Dosovitskiy et al., 2015;

Original: Ilg et al., 2017; Ranjan & Black, 2017; Sun et al., 2018a;
Revised: Ilg et al., 2017; Ranjan & Black, 2017; Sun et al., 2018a;

Original: Sui et al., 2022; Jeong et al., 2022; Teed & Deng, 2020), RAFT (Teed & Deng, 2020) features a learning-to-optimize strategy using a recurrent GRU-based decoder to iteratively update a flow field f which is initialized at zero.
Revised: Sui et al., 2022; Jeong et al., 2022; Teed & Deng, 2020), RAFT (Teed & Deng, 2020) features a learning-to-optimize strategy using a recurrent GRU-based decoder to iteratively update a flow field f which is initialized at zero.

Original: Specifically, it extracts features using a convolutional encoder g θ from the input images I 

Revised: Specifically, it extracts features using a convolutional encoder g θ from the input images I

Original: and I 2 , and outputs features at 1/8 resolution,
Revised: and I 2 , and outputs features at 1/8 resolution,

Original: i.e. , g θ
Revised: i.e. , g θ

Original: ( I 1 ) ∈
Revised: ( I 1 ) ∈

Original: R H ′
Revised: R H ′

Original: × W ′ ×
Revised: × W ′ ×

Original: C and g θ ( I 2 ) ∈
Revised: C and g θ ( I 2 ) ∈

Original: R H ′
Revised: R H ′

Original: × W ′ ×
Revised: × W ′ ×

Original: C , where H ′ =
Revised: C , where H ′ =

Original: H/ 8 and W ′
Revised: H/ 8 and W ′

Original: =
Revised: =

Original: W/ 8 for spatial dimension and C =256 for feature dimension.
Revised: W/ 8 for spatial dimension and C =256 for feature dimension.

Original: Also, a context network h θ is applied to the first input image I 1 .
Revised: Also, a context network h θ is applied to the first input image I 1 .

Original: Then all-pair visual similarity is computed by constructing a 4D correlation volume V ∈
Revised: Then all-pair visual similarity is computed by constructing a 4D correlation volume V ∈

Original: R H ′
Revised: R H ′

Original: × W ′ × H ′ × W ′ between features g θ
Revised: × W ′ × H ′ × W ′ between features g θ

Original: I 1 ) and g θ
Revised: I 1 ) and g θ

Original: ( I 2 ) .
Revised: ( I 2 ) .

Original: It can be computed via matrix multiplication as V =
Revised: It can be computed via matrix multiplication as V =

Original: g θ
Revised: g θ

Original: ( I 1 ) · g Tθ ( I 2 ) , i.e. , (cid:16) R ( H ′ · W ′ )
Revised: ( I 1 ) · g Tθ ( I 2 ) , i.e. , (cid:16) R ( H ′ · W ′ )

Original: × C , R C × ( H ′ · W ′ )
Revised: × C , R C × ( H ′ · W ′ )

Original: (cid:17) (cid:55)→ R ( H ′ · W ′ )
Revised: (cid:17) (cid:55)→ R ( H ′ · W ′ )

Original: × ( H ′ · W ′ ) , which is further reshaped to V ∈ R H ′
Revised: × ( H ′ · W ′ ) , which is further reshaped to V ∈ R H ′

Original: × W ′ × H ′ × W ′ .
Revised: × W ′ × H ′ × W ′ .

Original: Then RAFT builds a 4-layer coorelation pyramid { V s } 4 s =1 by pooling the last two dimensions of V with kernel sizes 2 s − 1 , respectively.
Revised: Then RAFT builds a 4-layer coorelation pyramid { V s } 4 s =1 by pooling the last two dimensions of V with kernel sizes 2 s − 1 , respectively.

Original: The GRU-based decoder estimates a sequence of flow estimates { f 1 , . . .
Revised: The GRU-based decoder estimates a sequence of flow estimates { f 1 , . . .

Original: , f T } ( T =12 or 24) from a zero initialized f 0 = 0 .
Revised: , f T } ( T =12 or 24) from a zero initialized f 0 = 0 .

Original: RAFT attains high accuracy, strong generalization as well as high efficiency.
Revised: RAFT attains high accuracy, strong generalization as well as high efficiency.

Original: We take RAFT as the backbone and achieve boosted performance, i.e. F1-all errors of 4.11 (ours) vs 5.10 (raft) on
Revised: We take RAFT as the backbone and achieve boosted performance, i.e. F1-all errors of 4.11 (ours) vs 5.10 (raft) on

Original: KITTI-2015 (Menze & Geiger, 2015)
Revised: KITTI-2015 (Menze & Geiger, 2015)

Original: ( c.f. Tab. 1).
Revised: ( c.f. Tab. 1).

Original: CRAFT.
Revised: CRAFT.

Original: To overcome the challenges of large displacements with motion blur and the limited field of view due to locality of convolutional features in RAFT, CRAFT (Sui et al., 2022) proposes to leverage transformer layers to learn global features by considering long-range dependence, and hence revitalize the 4D correlation volume V computation as in RAFT.
Revised: To overcome the challenges of large displacements with motion blur and the limited field of view due to locality of convolutional features in RAFT, CRAFT (Sui et al., 2022) proposes to leverage transformer layers to learn global features by considering long-range dependence, and hence revitalize the 4D correlation volume V computation as in RAFT.

Original: We also use CRAFT as the backbone and attain improvement, i.e. F1-all errors of 4.66 (ours) vs 4.79 (craft) on KITTI- (Menze & Geiger, 2015)
Revised: We also use CRAFT as the backbone and attain improvement, i.e. F1-all errors of 4.66 (ours) vs 4.79 (craft) on KITTI- (Menze & Geiger, 2015)

Original: ( c.f. Tab. 1).
Revised: ( c.f. Tab. 1).

Original: 3. I TERATIVE P SEUDO L ABELING
Revised: 3. I TERATIVE P SEUDO L ABELING

Original: Deep learning based optical flow methods are usually pretrained on synthetic data 1 and finetuned on small real data.
Revised: Deep learning based optical flow methods are usually pretrained on synthetic data 1 and fine-tuned on small real data.

Original: This begs an important question: How to effectively transfer the knowledge learned from synthetic domain to real world scenarios and bridge the big gap between them?
Revised: This begs an important question: How to effectively transfer the knowledge learned from synthetic domain to real world scenarios and bridge the big gap between them?

Original: Our semisupervised framework is proposed to improve the performance on real datasets D R , by iteratively transferring the knowledge learned from synthetic data D S and/or a few of available real datasets
Revised: Our semisupervised framework is proposed to improve the performance on real datasets D R , by iteratively transferring the knowledge learned from synthetic data D S and/or a few of available real datasets

Original: D trR (with sparse or dense ground truth optical flow labels).
Revised: D trR (with sparse or dense ground truth optical flow labels).

Original: Without loss of generality, we assume that the real data D R consists of i) a small amount of training data D trR ( e.g. KITTI 2015 (Menze &
Revised: Without loss of generality, we assume that the real data D R consists of i) a small amount of training data D trR ( e.g. KITTI 2015 (Menze &

Original: Geiger, 2015) training set with 200 image pairs) due to the expensive and tedious labeling by human, ii) a number of testing data D teR ( e.g. KITTI 2015 test set with 200 pairs), and iii) a large amount of unlabeled data D uR ( e.g. KITTI raw dataset (Geiger et al., 2013) having 84,642 images pairs) which is quite similar to the test domain.
Revised: Geiger, 2015) training set with 200 image pairs) due to the expensive and tedious labeling by human, ii) a number of testing data D teR ( e.g. KITTI 2015 test set with 200 pairs), and iii) a large amount of unlabeled data D uR ( e.g. KITTI raw dataset (Geiger et al., 2013) having 84,642 images pairs) which is quite similar to the test domain.

Original: Therefore, we propose to use the unlabeled, real KITTI
Revised: Therefore, we propose to use the unlabeled, real KITTI

Original: Raw data by generating pseudo ground truth labels using a master (or teacher) model to transfer the knowledge from pretraining on synthetic data or small real data to real data KITTI 2015 test set.
Revised: Raw data by generating pseudo ground truth labels using a master (or teacher) model to transfer the knowledge from pretraining on synthetic data or small real data to real data KITTI 2015 test set.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 2, our semi-supervised iterative pseudo labeling training strategy includessteps: 1) Training on a large amount of unlabeled data ( D uR ) supervised by a master (or teacher) model, which is chosen at the beginning as a model pretrained on large-scale synthetic and small real datasets, 2) Conducting k -fold cross validation on the labeled real dataset ( D trR ) to find best hyper-parameters, e.g. training steps for finetuing S ft , and 3) finetuning our model on the labeled dataset ( D trR ) using the best hyper-parameters selected above, and updating the finetuned model as a new version of the master (or teacher) model to repeat those steps for next iteration, until the predefined iteration steps N is reached or the gain of evaluation accuracy on test set ( D teR ) is marginal.
Revised: 2, our semi-supervised iterative pseudo labeling training strategy includessteps: 1) Training on a large amount of unlabeled data ( D uR ) supervised by a master (or teacher) model, which is chosen at the beginning as a model pretrained on large-scale synthetic and small real datasets, 2) Conducting k -fold cross validation on the labeled real dataset ( D trR ) to find best hyper-parameters, e.g. training steps for finetuing S ft , and 3) finetuning our model on the labeled dataset ( D trR ) using the best hyper-parameters selected above, and updating the finetuned model as a new version of the master (or teacher) model to repeat those steps for next iteration, until the predefined iteration steps N is reached or the gain of evaluation accuracy on test set ( D teR ) is marginal.

Original: The detailed algorithm is illustrated in Alg. 1.
Revised: The detailed algorithm is illustrated in Alg. 1.

Original: Semi-supervised learning on unlabeled real dataset.
Revised: Semi-supervised learning on unlabeled real dataset.

Original: Our proposed iterative pseudo labeling method aims at dealing with real imagery that usually lacks ground truth labels and is difficult to be accurately modeled by simulators due to reflective surfaces, sensor noise and illumination conditions (Cai et al., 2020).
Revised: Our proposed iterative pseudo labeling method aims at dealing with real imagery that usually lacks ground truth labels and is difficult to be accurately modeled by simulators due to reflective surfaces, sensor noise and illumination conditions (Cai et al., 2020).

Original: As shown in Alg. 1 and Fig.
Revised: As shown in Alg. 1 and Fig.

Original: 2-A, a pretrained baseline model ϕ bs , e.g. RAFT, and our pretrained model ϕ 0 our , e.g. RAFT-CF, a variant of the baseline RAFT by adding our contrastive flow loss ( c.f.
Revised: 2-A, a pretrained baseline model ϕ bs , e.g. RAFT, and our pretrained model ϕ 0 our , e.g. RAFT-CF, a variant of the baseline RAFT by adding our contrastive flow loss ( c.f.

Original: Sec 3.3) are provided and they are pretrained using the same input datasets of synthetic images ( D S , including C+T+S for FlyingChairs, FlyingThings3D, Sintel, respectively) and a small Input synthetic datasets a large amount of unlabeled data D uR )
Revised: Sec 3.3) are provided and they are pretrained using the same input datasets of synthetic images ( D S , including C+T+S for FlyingChairs, FlyingThings3D, Sintel, respectively) and a small number of real ones ( D trR , including K+H for KITTI 2015 and HD1K).

Original: Input ft do 13: ϕ  our ← L (cid:16) ϕ our(cid:0) D trR (cid:1) , ˆ f (cid:17) ▷ finetuning on  real train data D trR by loss L 14: s ← s +15: end while 16:
Revised: Then based on the pretrained model ϕ 0 our ( c.f. line 6 in Alg. 1), we train our model ϕ our on the large unlabeled real dataset ( D uR )

Original: i ← i +▷ increment index i for next pseudo labeling iteration 17: end while number of real ones ( D trR , including K+H for KITTI 2015 and HD1K).
Revised: [Table]

Original: Then based on the pretrained model ϕ 0 our ( c.f. line 6 in Alg. 1), we train our model ϕ our on the large unlabeled real dataset ( D uR ) with the supervision of the pseudo ground truth labels ˆ f ( c.f. line 5 in Alg. 1).
Revised: with the supervision of the pseudo ground truth labels ˆ f ( c.f. line 5 in Alg. 1).

Original: The pseudo labels ˆ f are generated by a master model ( c.f. Fig.
Revised: The pseudo labels ˆ f are generated by a master model ( c.f. Fig.

Original: 2-A), and the master model is initialized as the pretrained baseline ϕ bs at the first step, and will be updated in the following iteration with our new model ϕ our .
Revised: 2-A), and the master model is initialized as the pretrained baseline ϕ bs at the first step, and will be updated in the following iteration with our new model ϕ our .

Original: k-Fold Cross validation on labeled small real dataset Gathering annotated data for real world applications may be too expensive or infeasible.
Revised: applications may be too expensive or infeasible.

Original: In order to take most of the labeled dataset, we use k -fold cross validation to find the best training hyper-parameters, especially, the best training steps S ft for finetuing on the whole real dataset ( D trR , e.g. KITTI 2015 training set).
Revised: In order to take most of the labeled dataset, we use k -fold cross validation to find the best training hyper-parameters, especially, the best training steps S ft for finetuing on the whole real dataset ( D trR , e.g. KITTI 2015 training set).

Original: As shown in the Fig.
Revised: As shown in the Fig.

Original: 2-B, we divide the validation dataset into k folds ( e.g. k =5), and we evaluate the trained model ϕ our on the corresponding part of the validation set.
Revised: 2-B, we divide the validation dataset into k folds ( e.g. k =5), and we evaluate the trained model ϕ our on the corresponding part of the validation set.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 3, we pick the best hyper-parameter according to the lowest average validation error.
Revised: 3, we pick the best hyper-parameter according to the lowest average validation error.

Original: Finetuning the model.
Revised: Finetuning the model.

Original: After getting the best hyper-parameterw via the k -fold cross validation, we use the obtained hyper parameters to finetune our model ϕ our on the whole real labeled data ( D trR ( c.f. line 13 in Alg. 1).
Revised: After getting the best hyper-parameterw via the k -fold cross validation, we use the obtained hyper parameters to finetune our model ϕ our on the whole real labeled data ( D trR ( c.f. line 13 in Alg. 1).

Original: We use this finefuned model ϕ our to update the master (or teacher) model to produce second round pseudo ground truth optical flow labels.
Revised: We use this finefuned model ϕ our to update the master (or teacher) model to produce second round pseudo ground truth optical flow labels.

Original: These steps will be repeated until the predefined iteration N is reached or the gain of accuracy on real dataset ( D R is marginal.
Revised: These steps will be repeated until the predefined iteration N is reached or the gain of accuracy on real dataset ( D R is marginal).

Original:  C ONTRASTIVE F LOW
Revised: 3. C ONTRASTIVE F LOW

Original: Self-supervised Contrastive Learning.
Revised: Self -supervised Contrastive Learning.

Original: Contrastive learning first introduced by (Hadsell et al., 2006) has become increasingly successful for self-supervised representation learning (Wu et al., 2018;
Revised: Contrastive learning first introduced by (Hadsell et al., 2006) has become increasingly successful for self-supervised representation learning (Wu et al., 2018;

Original: Oord et al., 2018; Hjelm et al., 2019; Bachman et al., 2019; Chen et al., 2020a;
Revised: Oord et al., 2018; Hjelm et al., 2019; Bachman et al., 2019; Chen et al., 2020a;

Original: He et al., 2020; Chen et al., 2021) in computer vision.
Revised: He et al., 2020; Chen et al., 2021) in computer vision.

Original: It learns visual representations such that two similar (positive) points have a small distance and two dissimilar (negative) points have a large distance.
Revised: It learns visual representations such that two similar (positive) points have a small distance and two dissimilar (negative) points have a large distance.

Original: This can be formulated as a dictionary look-up problem given an encoded query q and a set of encoded samples { k 0 , k 1 , k 2 , . . .
Revised: This can be formulated as a dictionary look-up problem given an encoded query q and a set of encoded samples { k 0 , k 1 , k 2 , . . .

Original: as the keys of a dictionary (He et al., 2020; Chen et al., 2021).
Revised: as the keys of a dictionary (He et al., 2020; Chen et al., 2021).

Original: A general formula of a contrastive loss function, called InfoNCE (Oord et al., 2018), is defined as:
Revised: A general formula of a contrastive loss function, called InfoNCE (Oord et al., 2018), is defined as:

Original: [Equation]
Revised: [Equation]

Original: where k + is the positive sample of the query q , and the set { k − } consists of negative samples of q , τ is a temperature hyper-parameter, and the operator “ · ” is dot product.
Revised: where k + is the positive sample of the query q , and the set { k − } consists of negative samples of q , τ is a temperature hyper-parameter, and the operator “ · ” is dot product.

Original: In general, the query representation is q = g q ( x q ) where g q is an encoder network and x q is an input query image, and similarly, k = g k ( x k ) for keys.
Revised: In general, the query representation is q = g q ( x q ) where g q is an encoder network and x q is an input query image, and similarly, k = g k ( x k ) for keys.

Original: Semi-supervised Contrastive Flow.
Revised: Semi-supervised Contrastive Flow.

Original: To improve the representations for optical flow, we propose a contrastive flow loss to explicitly supervise the network training for learning better features and hence an informative correlation volume in RAFT (Teed & Deng, 2020).
Revised: To improve the representations for optical flow, we propose a contrastive flow loss to explicitly supervise the network training for learning better features and hence an informative correlation volume in RAFT (Teed & Deng, 2020).

Original: Given the input images Iand I 2 , the extracted features g θ ( I 1 ) and g θ ( I 2 ) , and the optical flow f 2 , a pixel index i in I 1 with the feature g 1 i ∈
Revised: Given the input images Iand I 2 , the extracted features g θ ( I 1 ) and g θ ( I 2 ) , and the optical flow f 2 , a pixel index i in I 1 with the feature g 1 i ∈

Original: g θ
Revised: g θ

Original: ( I 1 ) is warped to a corresponding pixel index j in I 2 , with j = i + f i in I 2 and the corresponding feature g 2 j ∈ g θ
Revised: ( I 1 ) is warped to a corresponding pixel index j in I 2 , with j = i + f i in I 2 and the corresponding feature g 2 j ∈ g θ

Original: ( I 2 ) is sampled from feature map g θ
Revised: ( I 2 ) is sampled from feature map g θ

Original: ( I 2 ) via bilinear interpolation.
Revised: ( I 2 ) via bilinear interpolation.

Original: We consider the corresponding pair ( i, j ) as a positive pair and other samples ( i, k ) for k ̸
Revised: We consider the corresponding pair ( i, j ) as a positive pair and other samples ( i, k ) for k ̸

Original: =
Revised: =

Original: j as negative pairs.
Revised: j as negative pairs.

Original: Therefore, we define the contrastive flow loss for index i as
Revised: Therefore, we define the contrastive flow loss for index i as

Original: [Equation]
Revised: [Equation]

Original: And the contrastive flow loss over all the valid pixels which have (pseudo) ground truth in image Iis defined as
Revised: And the contrastive flow loss over all the valid pixels which have (pseudo) ground truth in image Iis defined as

Original: [Equation]
Revised: [Equation]

Original: where N v is the number of valid pixels and recall j = i + f i .
Revised: where N v is the number of valid pixels and recall j = i + f i .

Original: The loss L CT in Eq.
Revised: The loss L CT in Eq.

Original: 3 can be efficiently computed as matrix multiplication.
Revised: 3 can be efficiently computed as matrix multiplication.

Original: Coordinate Encoding.
Revised: Coordinate Encoding.

Original: Optical flow is the task of estimating per-pixel motion between video frames.
Revised: Optical flow is the task of estimating per-pixel motion between video frames.

Original: Therefore, the coordinate of each pixel should be an important cue when predicting the optical flow for each pixel pair in I 1 and I 2 .
Revised: Therefore, the coordinate of each pixel should be an important cue when predicting the optical flow for each pixel pair in I 1 and I 2 .

Original: We proposed to concatenate 2D-coordinate map of each pixel as two additional channels to the original input frames.
Revised: We proposed to concatenate 2D-coordinate map of each pixel as two additional channels to the original input frames.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 1, the new input pair becomes I ′ 1 , I ′
Revised: 1, the new input pair becomes I ′ 1 , I ′

Original: 2 ∈ R
Revised: 2 ∈ R

Original: H
Revised: H

Original: × W × 5 .
Revised: × W × 5 .

Original: In the ablation study (Sec. 4.3), we demonstrate that incorporating the coordinate encoding consistently improves the optical flow estimates.
Revised: In the ablation study (Sec. 4.3), we demonstrate that incorporating the coordinate encoding consistently improves the optical flow estimates.

Original:  E XPERIMENTS 4. E XPERIMENTAL S ETTING AND D ATASET
Revised:  E XPERIMENTS 4. E XPERIMENTAL S ETTING AND D ATASET

Original: In our experiments, we have mainly focused on two recent and well-known base models, which are
Revised: In our experiments, we have mainly focused on two recent and well-known base models, which are

Original: RAFT(Teed & Deng, 2020) and CRAFT(Sui et al., 2022), to verify the efficacy of our proposed training strategy.
Revised: RAFT(Teed & Deng, 2020) and CRAFT(Sui et al., 2022), to verify the efficacy of our proposed training strategy.

Original: Our pretrained model based on RAFT and CRAFT are achieved by pretraining on dataset such as Sintel(Butler et al., 2012a),
Revised: Our pretrained model based on RAFT and CRAFT are achieved by pretraining on dataset such as Sintel(Butler et al., 2012a),

Original: HD1K (Krispin et al., 2016), FlyingChair(Dosovitskiy et al., 2015), and FlyingThing(Mayer et al., 2016).
Revised: HD1K (Krispin et al., 2016), FlyingChair(Dosovitskiy et al., 2015), and FlyingThing(Mayer et al., 2016).

Original: We use similar training and hyperparamter setting to achieve these models, as described in the original works.
Revised: We use similar training and hyperparamter setting to achieve these models, as described in the original works.

Original: We refer readers to (Teed &
Revised: We refer readers to (Teed &

Original: Deng, 2020; Sui et al., 2022) for more details.
Revised: Deng, 2020; Sui et al., 2022) for more details.

Original: Training during iterative refinement using pseudo labelling, is done on KITTI flow dataset(Menze & Geiger, 2015; Menze et al., 2018), where we use KITTI-Raw dataset, which consists of about 84642 images pairs but without any ground-truth labels for optical flow.
Revised: Training during iterative refinement using pseudo labelling, is done on KITTI flow dataset(Menze & Geiger, 2015; Menze et al., 2018), where we use KITTI-Raw dataset, which consists of about 84642 images pairs but without any ground-truth labels for optical flow.

Original: We call the publicly available 200 KITTI flow dataset with manually labeled optical flow groundtruth as KITTI-Flow-Val, and the other 200 reserved KITTI flow testing data as
Revised: We call the publicly available 200 KITTI flow dataset with manually labeled optical flow groundtruth as KITTI-Flow-Val, and the other 200 reserved KITTI flow testing data as

Original: KITTI-Flow-test.
Revised: KITTI-Flow-test.

Original: For a fair analysis, we conduct exhaustive ablation experiments and use the standard KITTI-Flowtest dataset, for evaluating our models.
Revised: For a fair analysis, we conduct exhaustive ablation experiments and use the standard KITTI-Flowtest dataset, for evaluating our models.

Original: In the next few subsections we discuss in detail the results achieved on KITTI dataset using our training strategy.
Revised: In the next few subsections we discuss in detail the results achieved on KITTI dataset using our training strategy.

Original: We go on to show that our proposed model (RAFT-CF) along with iterative pseudo labelling helps us achieve state-of-art result for KITTI dataset and shows significant improvement for synthetic dataset as well.
Revised: We go on to show that our proposed model (RAFT-CF) along with iterative pseudo labelling helps us achieve state-of-art result for KITTI dataset and shows significant improvement for synthetic dataset as well.

Original: 4. R ESULTS ON KITTI
Revised: 4. R ESULTS ON KITTI

Original: RAFT Model Training on KITTI dataset could be broadly subdivided in to three stages as shown in the 2 and is discussed in section 3.2.
Revised: RAFT Model Training on KITTI dataset could be broadly subdivided in to three stages as shown in the 2 and is discussed in section 3.2.

Original: All 3 stages defined in 2(A, B, and
Revised: All 3 stages defined in 2(A, B, and

Original: C), forms one iteration of pseudo-label enabled training.
Revised: C), forms one iteration of pseudo-label enabled training.

Original: In the first stage (Fig.2-A), we train our proposed RAFT-CF-CE (RAFT with Contrastive flow with Coordinate embedding) on
Revised: In the first stage (Fig.2-A), we train our proposed RAFT-CF-CE (RAFT with Contrastive flow with Coordinate embedding) on

Original: KITTI-Raw dataset using the pseudo flow labels, generated using original pretrained RAFT model as our Master RAFT model for 1.2m steps.
Revised: KITTI-Raw dataset using the pseudo flow labels, generated using original pretrained RAFT model as our Master RAFT model for 1.2m steps.

Original: In this stage our candidate RAFT-CFCE model will be updated by the supervision of the pseudo flows.
Revised: In this stage our candidate RAFT-CFCE model will be updated by the supervision of the pseudo flows.

Original: In the second stage (Fig.2-B), we take the updated RAFT-CF-CE model from part A and perform 5-fold cross validation on the KITTI-Flow-Val, by dividing the 200 image pairs into 5 parts, where each part includes 40 images pair.
Revised: In the second stage (Fig.2-B), we take the updated RAFT-CF-CE model from part A and perform 5-fold cross validation on the KITTI-Flow-Val, by dividing the 200 image pairs into 5 parts, where each part includes 40 images pair.

Original: We conduct a total of 5 experiments on the updated RAFT-CF-CE model, and for each group of cross-validation, we evaluate the
Revised: We conduct a total of 5 experiments on the updated RAFT-CF-CE model, and for each group of cross-validation, we evaluate the

Original: F1-all error score at every 200 steps.
Revised: F1-all error score at every 200 steps.

Original: We then calculate the average F1-all error score for each step through all the 5 groups of training, and pick the one that gives the lowest F1-all error score as the candidate best suited for fine-tuning our updated RAFT-CF-CE model.
Revised: We then calculate the average F1-all error score for each step through all the 5 groups of training, and pick the one that gives the lowest F1-all error score as the candidate best suited for fine-tuning our updated RAFT-CF-CE model.

Original: As seen in 3, we find that, we achieve an inflection point around 1.6k training steps, which gives us the best average accuracy on the validation dataset.
Revised: As seen in 3, we find that, we achieve an inflection point around 1.6k training steps, which gives us the best average accuracy on the validation dataset.

Original: Through this cross validation step, we efficiently narrow down the optimum number of training steps required to further fine-tune the model on KITTI-Flow-Val dataset.
Revised: Through this cross validation step, we efficiently narrow down the optimum number of training steps required to further fine-tune the model on KITTI-Flow-Val dataset.

Original: It also brings out the fact that, the model tends to grossly over-fit the small training data even after 2k steps.
Revised: It also brings out the fact that, the model tends to grossly over-fit the small training data even after 2k steps.

Original: Finally, In the third stage (Fig. 2-C), we finetune the updated RAFT-CF-CE-PL model from part A with the whole KITTI-Flow-Val for the optimum steps, which we get from Part B. After finetune, we get our final updated RAFT-CF-CE model for the current iteration.
Revised: Finally, In the third stage (Fig. 2-C), we finetune the updated RAFT-CF-CE-PL model from part A with the whole KITTI-Flow-Val for the optimum steps, which we get from Part B. After finetune, we get our final updated RAFT-CF-CE model for the current iteration.

Original: We repeat this process for iterative refinement for few more iterations till the model converges.
Revised: We repeat this process for iterative refinement for few more iterations till the model converges.

Original: The comparative results on KITTI-Flow-Val and KITTI-Flow-test among all the published SOTA methods is presented in Tab. 1.
Revised: The comparative results on KITTI-Flow-Val and KITTI-Flow-test among all the published SOTA methods is presented in Tab. 1.

Original: Results seen in Tab. 1, further validates robustness of our approach, as even-though all of our model have higher training error, it performs much better on test dataset.
Revised: Results seen in Tab. 1, further validates robustness of our approach, as even-though all of our model have higher training error, it performs much better on test dataset.

Original: Unlike other SOTA methods, which get lower training error but higher testing error, clearly reflecting towards an over-fitted model.
Revised: Unlike other SOTA methods, which get lower training error but higher testing error, clearly reflecting towards an over-fitted model.

Original: This is also because of the fact that we follow a semi-supervised approach leveraging contrastive and pseudo labels during our training.
Revised: This is also because of the fact that we follow a semi-supervised approach leveraging contrastive and pseudo labels during our training.

Original: Therefore, we claim that with the proposed training strategy, our model can be trained for more iterations and even more[Table] steps without any data-overfitting issues.
Revised: Therefore, we claim that with the proposed training strategy, our model can be trained for more iterations and even more steps without any data-overfitting issues.

Original: In our case, we conducted 3 iterations of iterative training and the last iteration has 5 million steps, giving us a F1-all error score of 4.11 on the KITTI-Flow-test with 2.78 training error on KITTI-Flow-Val.
Revised: In our case, we conducted 3 iterations of iterative training and the last iteration has 5 million steps, giving us a F1-all error score of 4.11 on the KITTI-Flow-test with 2.78 training error on KITTI-Flow-Val.

Original: Qualitative improvement using our model is presented  in Fig.
Revised: Qualitative improvement using our model is presented [Table] in Fig.

Original: 5 on KITTI -Flow-test 2015 (Menze & Geiger, 
Revised: 5 on KITTI -Flow-test 2015 (Menze & Geiger, 2015).

Revised: The right column is our predicted optical flow results, the middle column is the results from the baseline model RAFT.

Revised: Our model, as seen from the results, is make finer flow predictions and also better handles occluded pixels.

Original: CRAFT
Revised: CRAFT

Original: In order to demonstrate generalizability of the proposed contrastive loss and iterative pseudo labeling training strategy, we also trained the most recently released state-of-art model
Revised: In order to demonstrate generalizability of the proposed contrastive loss and iterative pseudo labeling training strategy, we also trained the most recently released state-of-art model

Original: CRAFT(Sui et al., 2022) with our training strategy.
Revised: CRAFT(Sui et al., 2022) with our training strategy.

Original: We follow a similar 3 stage approach as described in RAFT Model paragraph of 4.2 for the CRAFT model.
Revised: We follow a similar 3 stage approach as described in RAFT Model paragraph of 4.2 for the CRAFT model.

Original: During our experiments, we found that training CRAFT based model takes unexpectedly longer time compared to one based on RAFT.
Revised: During our experiments, we found that training CRAFT based model takes unexpectedly longer time compared to one based on RAFT.

Original: Due to this constrain, we ran the CRAFT based CRAFT-CE-PL and CRAFT-CF-CE-PL model for 1 iteration with 620k and 420k steps respectively.
Revised: Due to this constrain, we ran the CRAFT based CRAFT-CE-PL and CRAFT-CF-CE-PL model for 1 iteration with 620k and 420k steps respectively.

Original: We then tested our final model on KITTI-Flowtest and we get F1-all error score of 4.66 and 4.68 , which outperforms original CRAFT model(4.
Revised: We then tested our final model on KITTI-Flowtest and we get F1-all error score of 4.66 and 4.68 , which outperforms original CRAFT model(4.

Original: F1-all error) by a good margin, as shown in table 1.
Revised: F1-all error) by a good margin, as shown in table 1.

Original: 4. A BLATIONS
Revised: 4. A BLATIONS

Original: [Table] [Table]
Revised: [Table] [Table]

Original: As outlined in the earlier sections, our method can be broadly divided in to three independent subparts, which are: 1) Employing iterative training with P seudo L abeling (PL), 2)
Revised: As outlined in the earlier sections, our method can be broadly divided in to three independent subparts, which are: 1) Employing iterative training with P seudo L abeling (PL), 2)

Original: Adding C ontrastive
Revised: Adding C ontrastive

Original: F low loss to the baseline encoder features (CF), and 3)
Revised: F low loss to the baseline encoder features (CF), and 3)

Original: Applying C oordinate E ncoding (CE) to the raw input frames.
Revised: Applying C oordinate E ncoding (CE) to the raw input frames.

Original: Here we will analyze how each of them contribute the improved performance.
Revised: Here we will analyze how each of them contribute the improved performance.

Original: 4-(b) shows the F1-all error score on KITTI-Flow-Val.
Revised: 4-(b) shows the F1-all error score on KITTI-Flow-Val.

Original: The top red curve is the RAFT-PL model on trained on KITTI-Raw.
Revised: The top red curve is the RAFT-PL model on trained on KITTI-Raw.

Original: The middle green curve is model RAFT-CE-PL, and the bottom blue is the RAFT-CF-CE-PL model.
Revised: The middle green curve is model RAFT-CE-PL, and the bottom blue is the RAFT-CF-CE-PL model.

Original: Through the experiments it shows that the proposed iterative pseudo[Figure] labeling training strategy itself will highly increase the predicted optical flow accuracy.
Revised: Through the experiments it shows that the proposed iterative pseudo labeling training strategy itself will highly increase the predicted optical flow accuracy.

Original: Adding coordinate embedding and contrastive flow loss will further increase the optical flow accuracy.
Revised: Adding coordinate embedding and contrastive flow loss will further increase the optical flow accuracy.

Original: The quantitative results shown in Table 3.
Revised: The quantitative results shown in Table 3.

Revised: [Figure]

Original:  C ONCLUSIONS
Revised:  C ONCLUSIONS

Original: In this paper we have proposed a novel and effective semi-supervised learning strategy, with iterative pseudo label refinement and contrastive flow loss.
Revised: In this paper we have proposed a novel and effective semi-supervised learning strategy, with iterative pseudo label refinement and contrastive flow loss.

Original: Our framework aims at transferring the knowledge pretrained on the synthetic data to the target real domain.
Revised: Our framework aims at transferring the knowledge pretrained on the synthetic data to the target real domain.

Original: Through the iterative pseudo label refinement, we can leverage the ubiquitous, unlabeled real data to facilitate dense optical flow training and bridge the domain gap between the synthetic and the real.
Revised: Through the iterative pseudo label refinement, we can leverage the ubiquitous, unlabeled real data to facilitate dense optical flow training and bridge the domain gap between the synthetic and the real.

Original: The contrastive flow loss is applied on a pair of corresponding features (one is warped to another via the pseudo ground truth flow), to boost accurate matching due to reliable pseudo labels and to dampen mismatching due to noisy pseudo labels, occlusion or global motion.
Revised: The contrastive flow loss is applied on a pair of corresponding features (one is warped to another via the pseudo ground truth flow), to boost accurate matching due to reliable pseudo labels and to dampen mismatching due to noisy pseudo labels, occlusion or global motion.

Original: Experiments results on KITTI 2015 and Sintel using two backbones RAFT and CRAFT, demonstrate the effectiveness of our proposed semi-supervised learning framework.
Revised: Experiments results on KITTI 2015 and Sintel using two backbones RAFT and CRAFT, demonstrate the effectiveness of our proposed semi-supervised learning framework.

Original: We obtain the second best result (F1-all error of 4.11%) on KITTI 2015 benchmark among all the non-stereo methods ( c.f.
Revised: We obtain the second best result (F1-all error of 4.11%) on KITTI 2015 benchmark among all the non-stereo methods ( c.f.

Original: Sec.
Revised: Sec.

Original: A for detailed rankings).
Revised: A for detailed rankings).

Original: A A PPENDIX
Revised: A A PPENDIX

Original: We show additional results in this appendix.
Revised: We show additional results in this appendix.

Original: A. S CREENSHOTS OF KITTI 2015 O PTICAL F LOW B ENCHMARK
Revised: A. S CREENSHOTS OF KITTI 2015 O PTICAL F LOW B ENCHMARK

Original: Fig.
Revised: Fig.

Original: 6 shows the evaluation results on the KITTI 2015 optical flow benchmark 3 .
Revised: 6 shows the evaluation results on the KITTI 2015 optical flow benchmark 3 .

Original: Based on the baseline RAFT (Teed & Deng, 2020), our model RAFT-CF-PL3 (with CF for contrastive flow, and
Revised: Based on the baseline RAFT (Teed & Deng, 2020), our model RAFT-CF-PL3 (with CF for contrastive flow, and

Original: PL3 for iterative pseudo labeling at the 3rd iteration) obtains an F1-all error of 4.11%, i e. a 19% error reduction with respect to RAFT (5.10%).
Revised: PL3 for iterative pseudo labeling at the 3rd iteration) obtains an F1-all error of 4.11%, i e. a 19% error reduction with respect to RAFT (5.10%).

Original: Our model outperforms most of the evaluated nonstereo methods 4 as the time of submission, except for RAFT-OCTC
Revised: Our model outperforms most of the evaluated nonstereo methods 4 as the time of submission, except for RAFT-OCTC

Original: (Jeong et al., 2022).
Revised: (Jeong et al., 2022).

Original: The not-trivial improvement compared with RAFT, demonstrates the effectiveness of i) our proposed semi-supervised iterative pseudo labeling training strategy and, ii) the proposed contrastive flow loss, which facilitates the semi-supervised training by dampening the mismatching or displacement due to motion blur or occlusion when leveraging the pseudo labeling.
Revised: The not-trivial improvement compared with RAFT, demonstrates the effectiveness of i) our proposed semi-supervised iterative pseudo labeling training strategy and, ii) the proposed contrastive flow loss, which facilitates the semi-supervised training by dampening the mismatching or displacement due to motion blur or occlusion when leveraging the pseudo labeling.

Original: A. S CREENSHOTS OF S INTEL B ENCHMARK
Revised: A. S CREENSHOTS OF S INTEL B ENCHMARK

Original: Fig.
Revised: Fig.

Original: 7 and Fig.
Revised: 7 and Fig.

Original: 8 show the evaluation results on the MPI Sintel benchmark 5 on clean pass and final pass, respectively.
Revised: 8 show the evaluation results on the MPI Sintel benchmark 5 on clean pass and final pass, respectively.

Original: Upon the baseline RAFT (Teed & Deng, 2020), our model RAFT-CF (with CF for contrastive flow, and no iterative pseudo labeling due to Sintel having ground truth labels) obtains an epe of 1.519 (clean pass), i.e. a
Revised: Upon the baseline RAFT (Teed & Deng, 2020), our model RAFT-CF (with CF for contrastive flow, and no iterative pseudo labeling due to Sintel having ground truth labels) obtains an epe of 1.519 (clean pass), i.e. a

Original: % error reduction from RAFT with 1.609; and an epe of 2.645 (final pass
Revised: % error reduction from RAFT with 1.609; and an epe of 2.645 (final pass

Original: , a 7% error reduction from RAFT with 2.855.
Revised: , a 7% error reduction from RAFT with 2.855.

Original: The improved results show the effectiveness of our proposed contrastive flow loss.
Revised: The improved results show the effectiveness of our proposed contrastive flow loss.

Original: It helps the supervised training on the synthetic Sintel dataset (Butler et al., 2012a) by mitigating the mismatching due to occlusion, small fast-moving objects, and global motion, even though having the ground truth optical flow.
Revised: It helps the supervised training on the synthetic Sintel dataset (Butler et al., 2012a) by mitigating the mismatching due to occlusion, small fast-moving objects, and global motion, even though having the ground truth optical flow.