OneFormerImageProcessor does not support passing local config file, always tries to download from repo
Author: rbavery
Pull request: no

### System Info

- `transformers` version: 4.29.0.dev0
- Platform: Linux-5.19.0-41-generic-x86_64-with-glibc2.35
- Python version: 3.10.10
- Huggingface_hub version: 0.14.1
- Safetensors version: 0.3.1
- PyTorch version (GPU?): 2.0.0+cu117 (True)
- Tensorflow version (GPU?): 2.11.1 (False)
- Flax version (CPU?/GPU?/TPU?): 0.5.3 (cpu)
- Jax version: 0.3.6
- JaxLib version: 0.3.5
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@amyeroberts

this forum post I put up seems like a bug: https://discuss.huggingface.co/t/how-to-load-local-config-json-for-oneformerimageprocessor-without-invoking-huggingfacehub-downloader/38372

The OneFormerImageProcessor should accept local config files without trying to download them from a repo_path

https://github.com/huggingface/transformers/blob/v4.28.1/src/transformers/models/oneformer/image_processing_oneformer.py#L323

### Information

- [X] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```
from transformers import OneFormerProcessor
config_path = "/local/config/path"
OneFormerProcessor.from_pretrained(config_path, ignore_mismatched_sizes=True)ignore_mismatched_sizes=True)
```

### Expected behavior

the processor gets initialized and doesn't error with 

```
            + f"Repository Not Found for url: {response.url}."
            + "\nPlease make sure you specified the correct `repo_id` and"
            " `repo_type`.\nIf you are trying to access a private or gated repo,"
            " make sure you are authenticated."
```

Comments:
@rbavery Thanks for raising this issue. 

I'm able to load a processor locally on the development branch without issue: 
```python
from transformers import OneFormerProcessor

processor = OneFormerProcessor.from_pretrained('shi-labs/oneformer_ade20k_swin_tiny')
processor.save_pretrained('foo')

new_processor = OneFormerProcessor.from_pretrained('foo')
```

Note, the processor combines two processing objects - the image processor and a tokenizer - and so configurations + additional files are necessary for to successfully load both to create the processor. Could you share the files in the folder you're trying to load from? In the `foo` folder created, I see the following files: 
```
merges.txt			
special_tokens_map.json		
tokenizer_config.json
preprocessor_config.json	
tokenizer.json			
vocab.json
```

As a small side note, in the example snippet, I believe there's a small typo in the code, and should be: 

```python
from transformers import OneFormerProcessor
config_path = "/local/config/path"
OneFormerProcessor.from_pretrained(config_path, ignore_mismatched_sizes=True)
```




Hi
I have a similar problem  , even when cloning the files locally still need to download ade20k_panoptic.json and it will not work without it

Hi @ammarali32, 

Ah OK, I understand now. This download is happening because of the [prepare_metadata method](https://github.com/huggingface/transformers/blob/17a55534f5e5df10ac4804d4270bf6b8cc24998d/src/transformers/models/oneformer/image_processing_oneformer.py#L323), which looks to download the file from the hub, and by default points to the `"shi-labs/oneformer_demo"` path. After being downloaded once, it should be possible to work in offline mode as it will be stored in the cache. However, I appreciate this isn't a complete solution. 

If there's another repo on the hub you wish to download the class info file from, replacing `repo_path` when instantiating the image processor class should be enough. To make the class look to either local files or on the hub, the image processing code would need to be reworked a bit. This is something that should happen in the future, however it's not a piece of work I have capacity to work on at the moment. If anyone from the community would like to take this I'm happy to review any PRs.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

> ### System Info
> * `transformers` version: 4.29.0.dev0
> * Platform: Linux-5.19.0-41-generic-x86_64-with-glibc2.35
> * Python version: 3.10.10
> * Huggingface_hub version: 0.14.1
> * Safetensors version: 0.3.1
> * PyTorch version (GPU?): 2.0.0+cu117 (True)
> * Tensorflow version (GPU?): 2.11.1 (False)
> * Flax version (CPU?/GPU?/TPU?): 0.5.3 (cpu)
> * Jax version: 0.3.6
> * JaxLib version: 0.3.5
> * Using GPU in script?:
> * Using distributed or parallel set-up in script?:
> 
> ### Who can help?
> @amyeroberts
> 
> this forum post I put up seems like a bug: https://discuss.huggingface.co/t/how-to-load-local-config-json-for-oneformerimageprocessor-without-invoking-huggingfacehub-downloader/38372
> 
> The OneFormerImageProcessor should accept local config files without trying to download them from a repo_path
> 
> https://github.com/huggingface/transformers/blob/v4.28.1/src/transformers/models/oneformer/image_processing_oneformer.py#L323
> 
> ### Information
> * [x]  The official example scripts
> * [x]  My own modified scripts
> 
> ### Tasks
> * [x]  An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
> * [x]  My own task or dataset (give details below)
> 
> ### Reproduction
> ```
> from transformers import OneFormerProcessor
> config_path = "/local/config/path"
> OneFormerProcessor.from_pretrained(config_path, ignore_mismatched_sizes=True)ignore_mismatched_sizes=True)
> ```
> 
> ### Expected behavior
> the processor gets initialized and doesn't error with
> 
> ```
>             + f"Repository Not Found for url: {response.url}."
>             + "\nPlease make sure you specified the correct `repo_id` and"
>             " `repo_type`.\nIf you are trying to access a private or gated repo,"
>             " make sure you are authenticated."
> ```

Hey, you can try to modify the prepare_metadata function in image_processing_oneformer.py like this:
  ```python
def prepare_metadata(repo_path, class_info_file):
    metadata = {}
    with open('xxx/preprocessor_config.json', "r") as f:
        class_info = json.load(f)
    metadata = class_info['metadata']
    return metadata
  ```


thanks @TreastBean