Transformers cannot recognise `config.json` even though it is in model directory
Author: constantinethegr8
Pull request: no


I ran this code:

```
import os
os.environ['TRANSFORMERS_CACHE'] = 'G:\.cache'

from transformers import AutoModelForCausalLM, AutoTokenizer, PretrainedConfig

#config = PretrainedConfig('G:\.cache\models--gptjchatbot_model\config.json') tried this but doesn't work well

model = AutoModelForCausalLM.from_pretrained("models--gptjchatbot_model")

tokenizer = AutoTokenizer.from_pretrained("EleutherAI/gpt-j-6B")

prompt = (
    "In a shocking finding, scientists discovered a herd of unicorns living in a remote, "
    "previously unexplored valley, in the Andes Mountains. Even more surprising to the "
    "researchers was the fact that the unicorns spoke perfect English."
)

input_ids = tokenizer(prompt, return_tensors="pt").input_ids

gen_tokens = model.generate(
    input_ids,
    do_sample=True,
    temperature=0.8,
    top_p=0.9,
    max_length=100,
)
gen_text = tokenizer.batch_decode(gen_tokens)[0]

```

for one of my finetuned custom models and I got this debug prompt (and I'm running a virtual python environment)
```
Traceback (most recent call last):
  File "G:\pytorch_model.bin_float32_(cpu)\conda-llm\lib\site-packages\transformers\configuration_utils.py", line 620, in _get_config_dict
    resolved_config_file = cached_file(
  File "G:\pytorch_model.bin_float32_(cpu)\conda-llm\lib\site-packages\transformers\utils\hub.py", line 409, in cached_file
    resolved_file = hf_hub_download(
  File "G:\pytorch_model.bin_float32_(cpu)\conda-llm\lib\site-packages\huggingface_hub\utils\_validators.py", line 112, in _inner_fn
    validate_repo_id(arg_value)
  File "G:\pytorch_model.bin_float32_(cpu)\conda-llm\lib\site-packages\huggingface_hub\utils\_validators.py", line 173, in validate_repo_id
    raise HFValidationError(f"Cannot have -- or .. in repo_id: '{repo_id}'.")
huggingface_hub.utils._validators.HFValidationError: Cannot have -- or .. in repo_id: 'models--gptjchatbot_model'.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "G:\pytorch_model.bin_float32_(cpu)\conda-llm\gptjchatbot.py", line 8, in <module>
    model = AutoModelForCausalLM.from_pretrained("models--gptjchatbot_model")
  File "G:\pytorch_model.bin_float32_(cpu)\conda-llm\lib\site-packages\transformers\models\auto\auto_factory.py", line 434, in from_pretrained
    config, kwargs = AutoConfig.from_pretrained(
  File "G:\pytorch_model.bin_float32_(cpu)\conda-llm\lib\site-packages\transformers\models\auto\configuration_auto.py", line 852, in from_pretrained
    config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "G:\pytorch_model.bin_float32_(cpu)\conda-llm\lib\site-packages\transformers\configuration_utils.py", line 565, in get_config_dict
    config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "G:\pytorch_model.bin_float32_(cpu)\conda-llm\lib\site-packages\transformers\configuration_utils.py", line 641, in _get_config_dict
    raise EnvironmentError(
OSError: Can't load the configuration of 'models--gptjchatbot_model'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'models--gptjchatbot_model' is the correct path to a directory containing a config.json file
```


I ran an Anaconda environment on a separate drive and went through the work of changing  the cache directory because I have no space on `C:`. Is there a way for config.json to be recognized. It is in the actual folder and I even tried making a subdirectory called `config`. Please help. Thanks.

Comments:
`model = AutoModelForCausalLM.from_pretrained("models--gptjchatbot_model")` cannot work, you need to specify the whole path to the folder on your local steup.

Modified line 8
`model = AutoModelForCausalLM.from_pretrained("G:\.cache\models--gptjchatbot_model\")`

got this error

```
  File "G:\pytorch_model.bin_float32_(cpu)\conda-llm\gptjchatbot.py", line 8
    model = AutoModelForCausalLM.from_pretrained("G:\.cache\models--gptjchatbot_model\")
                                                                                     ^
SyntaxError: unterminated string literal (detected at line 8)
```

@constantinethegr8 This is a python syntax error resulting from the `\` at the end of the path. The following should work: 
`model = AutoModelForCausalLM.from_pretrained("G:\.cache\models--gptjchatbot_model")`

I used this line:
```
model = AutoModelForCausalLM.from_pretrained("G:\.cache\models--gpt4chan_model")
```
like you said but got this new error about dependecies in the conda environment
```
Traceback (most recent call last):
  File "G:\pytorch_model.bin_float32_(cpu)\conda-llm\gptjchatbot.py.py", line 8, in <module>
    model = AutoModelForCausalLM.from_pretrained("G:\.cache\models--gptjchatbot_model")
  File "G:\pytorch_model.bin_float32_(cpu)\conda-llm\lib\site-packages\transformers\models\auto\auto_factory.py", line 434, in from_pretrained
    config, kwargs = AutoConfig.from_pretrained(
  File "G:\pytorch_model.bin_float32_(cpu)\conda-llm\lib\site-packages\transformers\models\auto\configuration_auto.py", line 874, in from_pretrained
    return config_class.from_dict(config_dict, **unused_kwargs)
  File "G:\pytorch_model.bin_float32_(cpu)\conda-llm\lib\site-packages\transformers\configuration_utils.py", line 688, in from_dict
    config = cls(**config_dict)
  File "G:\pytorch_model.bin_float32_(cpu)\conda-llm\lib\site-packages\transformers\models\gptj\configuration_gptj.py", line 139, in __init__
    super().__init__(
  File "G:\pytorch_model.bin_float32_(cpu)\conda-llm\lib\site-packages\transformers\configuration_utils.py", line 332, in __init__
    import torch
  File "G:\pytorch_model.bin_float32_(cpu)\conda-llm\lib\site-packages\torch\__init__.py", line 128, in <module>
    raise err
OSError: [WinError 182] The operating system cannot run %1. Error loading "G:\pytorch_model.bin_float32_(cpu)\conda-llm\lib\site-packages\torch\lib\shm.dll" or one of its dependencies.
```

@constantinethegr8  The new error being raised is showing that `torch` cannot be imported, which isn't a transformers issue. I suggest trying to reinstall torch in your environment. You can check whether torch is importable and its version by running `python -c "import torch; print(torch.__version__)"` in the terminal.  

so i should not `import torch as pytorch`?

and I got torch version 1.13.1

Hi @constantinethegr8, 

> so i should not `import torch as pytorch`?

`import x as y` is just a renaming of the module `x` to `y` in the scope. Although is generally advised against importing with non-canonical names, there's nothing stopping you and I doubt it is the cause of the issue here. 

> and I got torch version 1.13.1

OK, this means torch is install and in your path. As you mentioned, there's likely some misconfiguration in the conda environment between the dependencies. This isn't a transformers issue. I would recommend creating a new conda environment or searching online to see if there's other people who have encountered the same error message and the solutions they have found. 

Thank you. Should I avoid using a conda environment then as I have used regular python with transformers?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.