File size: 1,658 Bytes
93f2dde
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
f0aa576
 
 
 
 
93f2dde
f0aa576
93f2dde
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
---
license: mit
---
# TAC RGB encoder

<!-- Provide a quick summary of what the model is/does. -->

This model is used for encoding RGB image into a dense feature.

**Caution,** the model does not contain the last FC layer.
So, the output features are not aligned with depth.

## Model Details

### Model Description

<!-- Provide a longer summary of what this model is. -->

The model is pre-trained with RGB-D contrastive objectives, named TAC. 
Different from InfoNCE-based loss fuctions, TAC leverages the similarity between videos frames and estimate a similarity matrix as soft labels.
The backbone of this version is ViT-B/32.
The pre-training is conducted on a new unified RGB-D database, UniRGBD.
The main purpose of this work is depth representation.
So, the RGB encoder is just a side model.

### Model Sources

<!-- Provide the basic links for the model. -->

- **Repository:** [TAC](https://github.com/RavenKiller/TAC)
- **Paper:** [Learning Depth Representation from RGB-D Videos by Time-Aware Contrastive Pre-training](https://ieeexplore.ieee.org/document/10288539)


## Citation

<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->

```
@ARTICLE{10288539,
  author={He, Zongtao and Wang, Liuyi and Dang, Ronghao and Li, Shu and Yan, Qingqing and Liu, Chengju and Chen, Qijun},
  journal={IEEE Transactions on Circuits and Systems for Video Technology}, 
  title={Learning Depth Representation From RGB-D Videos by Time-Aware Contrastive Pre-Training}, 
  year={2024},
  volume={34},
  number={6},
  pages={4143-4158},
  doi={10.1109/TCSVT.2023.3326373}}

```