bconsolvo commited on
Commit
ce734de
1 Parent(s): 755bcbc

Update README.md (#2)

Browse files

- Update README.md (c188d745fbb431654ec4007bed424fbc78351a0b)

Files changed (1) hide show
  1. README.md +85 -11
README.md CHANGED
@@ -10,24 +10,98 @@ widget:
10
  example_title: Teapot
11
  - src: https://huggingface.co/datasets/mishig/sample_images/resolve/main/palace.jpg
12
  example_title: Palace
13
- ---
14
-
15
- # DPT (large-sized model)
16
-
17
- Dense Prediction Transformer (DPT) model trained on 1.4 million images for monocular depth estimation. It was introduced in the paper [Vision Transformers for Dense Prediction](https://arxiv.org/abs/2103.13413) by Ranftl et al. and first released in [this repository](https://github.com/isl-org/DPT).
18
 
19
- Disclaimer: The team releasing DPT did not write a model card for this model so this model card has been written by the Hugging Face team.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
20
 
21
- ## Model description
22
 
 
 
23
  DPT uses the Vision Transformer (ViT) as backbone and adds a neck + head on top for monocular depth estimation.
24
-
25
  ![model image](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/dpt_architecture.jpg)
26
 
27
- ## Intended uses & limitations
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
28
 
29
- You can use the raw model for zero-shot monocular depth estimation. See the [model hub](https://huggingface.co/models?search=dpt) to look for
30
- fine-tuned versions on a task that interests you.
31
 
32
  ### How to use
33
 
 
10
  example_title: Teapot
11
  - src: https://huggingface.co/datasets/mishig/sample_images/resolve/main/palace.jpg
12
  example_title: Palace
 
 
 
 
 
13
 
14
+ model-index:
15
+ - name: dpt-large
16
+ results:
17
+ - task:
18
+ type: monocular-depth-estimation
19
+ name: Monocular Depth Estimation
20
+ dataset:
21
+ type: MIX 6
22
+ name: MIX 6
23
+ metrics:
24
+ - type: Zero-shot transfer
25
+ value: 10.82
26
+ name: Zero-shot transfer
27
+ config: Zero-shot transfer
28
+ verified: false
29
+ ---
30
 
31
+ ## Model Details: DPT (large-sized model)
32
 
33
+ Dense Prediction Transformer (DPT) model trained on 1.4 million images for monocular depth estimation.
34
+ It was introduced in the paper [Vision Transformers for Dense Prediction](https://arxiv.org/abs/2103.13413) by Ranftl et al. (2021) and first released in [this repository](https://github.com/isl-org/DPT).
35
  DPT uses the Vision Transformer (ViT) as backbone and adds a neck + head on top for monocular depth estimation.
 
36
  ![model image](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/dpt_architecture.jpg)
37
 
38
+ The model card has been written in combination by the Hugging Face team and Intel.
39
+
40
+ | Model Detail | Description |
41
+ | ----------- | ----------- |
42
+ | Model Authors - Company | Intel |
43
+ | Date | March 22, 2022 |
44
+ | Version | 1 |
45
+ | Type | Computer Vision - Monocular Depth Estimation |
46
+ | Paper or Other Resources | [Vision Transformers for Dense Prediction](https://arxiv.org/abs/2103.13413) and [GitHub Repo](https://github.com/isl-org/DPT) |
47
+ | License | [MIT](https://github.com/isl-org/DPT/blob/main/LICENSE) |
48
+ | Questions or Comments | [Community Tab](https://huggingface.co/Intel/dpt-large/discussions) and [Intel Developers Discord](https://discord.gg/rv2Gp55UJQ)|
49
+
50
+ | Intended Use | Description |
51
+ | ----------- | ----------- |
52
+ | Primary intended uses | You can use the raw model for zero-shot monocular depth estimation. See the [model hub](https://huggingface.co/models?search=dpt) to look for
53
+ fine-tuned versions on a task that interests you. |
54
+ | Primary intended users | Anyone doing monocular depth estimation |
55
+ | Out-of-scope uses | This model in most cases will need to be fine-tuned for your particular task. |
56
+
57
+ | Factors | Description |
58
+ | ----------- | ----------- |
59
+ | Groups | Multiple datasets compiled together |
60
+ | Instrumentation | - |
61
+ | Environment | Inference completed on Intel Xeon Platinum 8280 CPU @ 2.70GHz with 8 physical cores and an NVIDIA RTX 2080 GPU. |
62
+ | Card Prompts | Model deployment on alternate hardware and software will change model performance |
63
+
64
+ | Metrics | Description |
65
+ | ----------- | ----------- |
66
+ | Model performance measures | Zero-shot Transfer |
67
+ | Decision thresholds | - |
68
+ | Approaches to uncertainty and variability | - |
69
+
70
+ | Training and Evaluation Data | Description |
71
+ | ----------- | ----------- |
72
+ | Datasets | The dataset is called MIX 6, and contains around 1.4M images. The model was initialized with ImageNet-pretrained weights.|
73
+ | Motivation | To build a robust monocular depth prediction network |
74
+ | Preprocessing | "We resize the image such that the longer side is 384 pixels and train on random square crops of size 384. ... We perform random horizontal flips for data augmentation." See [Ranftl et al. (2021)](https://arxiv.org/abs/2103.13413) for more details. |
75
+
76
+ ## Quantitative Analyses
77
+ | Model | Training set | DIW WHDR | ETH3D AbsRel | Sintel AbsRel | KITTI δ>1.25 | NYU δ>1.25 | TUM δ>1.25 |
78
+ | --- | --- | --- | --- | --- | --- | --- | --- |
79
+ | DPT - Large | MIX 6 | 10.82 (-13.2%) | 0.089 (-31.2%) | 0.270 (-17.5%) | 8.46 (-64.6%) | 8.32 (-12.9%) | 9.97 (-30.3%) |
80
+ | DPT - Hybrid | MIX 6 | 11.06 (-11.2%) | 0.093 (-27.6%) | 0.274 (-16.2%) | 11.56 (-51.6%) | 8.69 (-9.0%) | 10.89 (-23.2%) |
81
+ | MiDaS | MIX 6 | 12.95 (+3.9%) | 0.116 (-10.5%) | 0.329 (+0.5%) | 16.08 (-32.7%) | 8.71 (-8.8%) | 12.51 (-12.5%)
82
+ | MiDaS [30] | MIX 5 | 12.46 | 0.129 | 0.327 | 23.90 | 9.55 | 14.29 |
83
+ | Li [22] | MD [22] | 23.15 | 0.181 | 0.385 | 36.29 | 27.52 | 29.54 |
84
+ | Li [21] | MC [21] | 26.52 | 0.183 | 0.405 | 47.94 | 18.57 | 17.71 |
85
+ | Wang [40] | WS [40] | 19.09 | 0.205 | 0.390 | 31.92 | 29.57 | 20.18 |
86
+ | Xian [45] | RW [45] | 14.59 | 0.186 | 0.422 | 34.08 | 27.00 | 25.02 |
87
+ | Casser [5] | CS [8] | 32.80 | 0.235 | 0.422 | 21.15 | 39.58 | 37.18 |
88
+
89
+ Table 1. Comparison to the state of the art on monocular depth estimation. We evaluate zero-shot cross-dataset transfer according to the
90
+ protocol defined in [30]. Relative performance is computed with respect to the original MiDaS model [30]. Lower is better for all metrics. ([Ranftl et al., 2021](https://arxiv.org/abs/2103.13413))
91
+
92
+
93
+ | Ethical Considerations | Description |
94
+ | ----------- | ----------- |
95
+ | Data | The training data come from multiple image datasets compiled together. |
96
+ | Human life | The model is not intended to inform decisions central to human life or flourishing. It is an aggregated set of monocular depth image datasets. |
97
+ | Mitigations | No additional risk mitigation strategies were considered during model development. |
98
+ | Risks and harms | The extent of the risks involved by using the model remain unknown.|
99
+ | Use cases | - |
100
+
101
+ | Caveats and Recommendations |
102
+ | ----------- |
103
+ | There are no additional caveats or recommendations for this model. |
104
 
 
 
105
 
106
  ### How to use
107