Model doc examples for GPT-j-6b are slightly misleading for the GPU context
Author: 6r1d
Pull request: no

# 🚀 Feature request

The default example can provide a bit more info for GPU / CUDA users.

First part I got stuck on yesterday was selecting a device:

```python
# Model init + CUDA device
device = torch.device("cuda")
model = GPTJForCausalLM.from_pretrained("EleutherAI/gpt-j-6B", revision="float16", low_cpu_mem_usage=True)
model.to(device)
```

There's also tokenizer that should use the same device, otherwise a CPU gets involved and RAM use gets excessive:

```python
# Tokenizer use
input_ids = tokenizer(prompt, return_tensors="pt").input_ids.to(device)
```

## Motivation

Current documentation can provide more info for CUDA users.

## Your contribution

I would like to add a PR to the doc later, if my idea here makes sense.

Comments:
Hi,

That's actually how any PyTorch model works, this is not really specific to GPT-J. If you want to run PyTorch models on GPU, you need to send both your model and data to the GPU. This is explained in the [PyTorch intro tutorial](https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html).

We do assume this knowledge, but if you think the documentation can be improved, feel free to open a PR.

> but if you think the documentation can be improved, feel free to open a PR.

Alright, please don't close the issue yet, I will do a PR when I have a bit of time.
I've already talked to several people who wanted to use GPT-j as an API part, locally or not, and error messages appearing aren't always pointing in the right direction.

I'd rarely ask for help - but I must have taken a wrong turn.
I enable lfs + clone 24gb / pytorch_model.bin
 git clone https://huggingface.co/EleutherAI/gpt-j-6B

I read instructions - 

just says - do this
```python
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("EleutherAI/gpt-j-6B")
model = AutoModelForCausalLM.from_pretrained("EleutherAI/gpt-j-6B")
```

I shifted directories around so that there's a relative folder / EleutherAI then the git gpt-j-6B repo. 

I try with / without the "./" before EleutherAI 
tokenizer = AutoTokenizer.from_pretrained("./EleutherAI/gpt-j-6B")
model = AutoModelForCausalLM.from_pretrained("./EleutherAI/gpt-j-6B")

but I keep keep hitting error

```shell
Traceback (most recent call last):
  File "test.py", line 3, in <module>
    tokenizer = AutoTokenizer.from_pretrained("./EleutherAI/gpt-j-6B")
  File "/home/jp/miniconda3/envs/torch/lib/python3.8/site-packages/transformers/tokenization_auto.py", line 206, in from_pretrained
    config = AutoConfig.from_pretrained(pretrained_model_name_or_path, **kwargs)
  File "/home/jp/miniconda3/envs/torch/lib/python3.8/site-packages/transformers/configuration_auto.py", line 206, in from_pretrained
    config_class = CONFIG_MAPPING[config_dict["model_type"]]
KeyError: 'gptj'
```

maybe the git didn't clone correctly? 





@johndpope you don't need to clone the model repository, the `EleutherAI/gpt-j-6B` identifier will directly fetch from the hub. Of course, cloning the repo and using a local path also works.

Your error indicates that your `transformers` version does not have the GPT-J model available - what is your version? Can you update to a more recent version and let us know if it fixes your issue? Thank you.

Thanks @LysandreJik - solved one problem - probably going to just need more RAM. 
pip uninstall transformers # v 3.0
pip install transformers   # 4.12



This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.