Add ZoeDepth
Author: NielsRogge
Pull request: yes
Labels: name: New model

name: Vision

name: single-model-run-slow

name: run-slow

# What does this PR do?

This PR adds ZoeDepth as introduced in [ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth](https://arxiv.org/abs/2302.12288).

To do:

- [x] double check image processor (not sure we can support the [same resize](https://github.com/isl-org/ZoeDepth/blob/edb6daf45458569e24f50250ef1ed08c015f17a7/zoedepth/models/base_models/midas.py#L173)). Update image processor accordingly
- [x] remove testing scripts
- [x] verify relative position bias table/index when loading a beit model from the hub
- [x] add slow integration test
- [x] add image processor tests
- [x] should we add `backbone_hidden_size`?
- [x] make doc tests pass

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30136). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> Added tests. cc @ydshieh would it be possible to update make fixup to complain if an image processor is added but no tests are present (similar to models, tokenizers)?

(https://github.com/huggingface/transformers/pull/30136#discussion_r1573721245)

@NielsRogge I can take a look on this!

Thanks for your review. I've addressed all comments, the issue with BEiT is fixed as I now know that one can provide a [regex pattern for unexpected keys to be ignored](https://github.com/huggingface/transformers/blob/73014b561d5f88d728e46a57d346f516fefe3f2d/src/transformers/modeling_utils.py#L1264-L1267) (didn't know this :D ). 

cc @ydshieh there seems to be an issue regarding the new slow CI tests for a new model. Could you help me in looking into this?

 I have to check why such error in  

> Run all tests for the new model (models/zoedepth, single-gpu) (pull_request) 

@amyeroberts feel free to approve the PR, all checkpoints are on the hub now: https://huggingface.co/models?other=zoedepth

Hi @NielsRogge  You might want to pull the latest main and rebase. It looks like there is an issue of the PR slow CI (couldn't find a file)

cc @ydshieh looks like there's an issue with the CI runner

Hi @NielsRogge .

Thanks for flagging. At least, all the pytest are passing, it's just artifact upload issue. Will check.

Ah, I see the problem - the same model runs the tests twice!

Fixed #30634

Hi @amyeroberts addressed all comments, CI is green

Ok,

Pushed an empty commit to run the slow tests which confirmed all slow test are passing, the doc tests are tested by the `build_pr_documentation` check right?

> the doc tests are tested by the build_pr_documentation check right?

No, this checks the documentation can be built i.e. the webpage. 

See: https://huggingface.co/docs/transformers/main/en/testing#run-documentation-tests

Sorry I meant the `tests_pr_documentation_tests` check (which is passing)

Ah, yes, that should be fine as long as the docs and model examples are tested. The only thing is from the recent CI workflow, it doesn't look like the tests have [properly run (?)](https://app.circleci.com/pipelines/github/huggingface/transformers/94086/workflows/243f65ca-62c3-46f3-8dba-8af883b7d0dd/jobs/1235117/steps). At least, it's not possible to see which tests have run at all there 

@amyeroberts feel free to approve the PR as all comments have been addressed

@NielsRogge Have you tested for the issues related to weight loading / DPT? 

Yes, can confirm:
```python
>>> from transformers import ZoeDepthForDepthEstimation
>>> model = ZoeDepthForDepthEstimation.from_pretrained("Intel/zoedepth-nyu-kitti")
config.json: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2.23k/2.23k [00:00<00:00, 1.41MB/s]
model.safetensors: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.38G/1.38G [00:29<00:00, 47.2MB/s]
>>> model = ZoeDepthForDepthEstimation.from_pretrained("Intel/zoedepth-nyu")
config.json: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2.13k/2.13k [00:00<00:00, 3.27MB/s]
model.safetensors: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.38G/1.38G [00:29<00:00, 46.4MB/s]
>>> model = ZoeDepthForDepthEstimation.from_pretrained("Intel/zoedepth-kitti")
config.json: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2.12k/2.12k [00:00<00:00, 1.52MB/s]
model.safetensors: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.38G/1.38G [00:29<00:00, 46.0MB/s]
>>> 
```

@amyeroberts addressed your comment, failing CI is unrelated.

Slow tests passing locally, CI failing tests are unrelated, merging.

@NielsRogge Slow tests have to pass on the CI runs, not locally before merging. Please do not forcibly merge like this again. There are numerical differences which are introduced from the running environment and hardware which mean integration values may not match. You'll need to follow this up and make sure the slow tests pass on our CI images.