NameError: name 'torch' is not defined
Author: KaifAhmad1
Pull request: no

### System Info

#### Device Info: 
Device: `DESKTOP-0EE5HES`
Processor: `11th Gen Intel Core i5-1135G7 @ 2.40GHz`
Windows Edition: `Windows 11 Home Single Language`
System Type: `64-bit operating system, x64-based processor`
Version: `23H2 (Build 22631.2861)`

#### Package Info: 
transformers; `4.36.2`
Python: `3.10.10`
Huggingface_hub: `0.20.1`
Tensorflow: `2.15.0`
accelerate: `0.25.0`
torch: `2.1.2`



### Who can help?

@ArthurZucker
@younesbelkada

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

#### Packages: 

```
!pip install -qU \
      transformers \
      accelerate \
      einops \
      langchain \
       xformers \
       bitsandbytes \
        faiss-gpu \
        sentence_transformers \
        datasets

```
#### Loading the Llama-2-7B-Chat in notebook 
```
# Import necessary libraries
import torch 
from torch import cuda, bfloat16
import transformers
```

```
# Specify the pre-trained model ID
model_id = 'meta-llama/Llama-2-7b-chat-hf'
```

```
# Determine the device to be used (GPU if available, otherwise CPU)
device = f'cuda:{cuda.current_device()}' if cuda.is_available() else 'cpu'

```

```
# Configure quantization settings using the BitsAndBytesConfig
bnb_config = transformers.BitsAndBytesConfig(
    load_in_4bit=False,
    bnb_4bit_quant_type='nf4',
    bnb_4bit_use_double_quant=True,
    bnb_4bit_compute_dtype=bfloat16
)
```

```
# Initialize HF (Hugging Face) items with an access token
hf_auth = 'HF Token Here'
```

```
# Load model configuration using AutoConfig
model_config = transformers.AutoConfig.from_pretrained(
    model_id,
    use_auth_token=hf_auth
)
```

```
# Load the pre-trained model for causal language modeling using AutoModelForCausalLM
model = transformers.AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    config=model_config,
    quantization_config=bnb_config,
    device_map='auto',
    use_auth_token=hf_auth
)
```

##### Here is the Ouput of this code snippet: 

```
/usr/local/lib/python3.10/dist-packages/transformers/models/auto/auto_factory.py:472: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
  warnings.warn(
Loading checkpoint shards: 0%
0/2 [00:00<?, ?it/s]
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
[<ipython-input-8-55cdaac704cb>](https://localhost:8080/#) in <cell line: 2>()
      1 # Load the pre-trained model for causal language modeling using AutoModelForCausalLM
----> 2 model = transformers.AutoModelForCausalLM.from_pretrained(
      3     model_id,
      4     trust_remote_code=True,
      5     config=model_config,

4 frames
[/usr/local/lib/python3.10/dist-packages/transformers/integrations/bitsandbytes.py](https://localhost:8080/#) in set_module_quantized_tensor_to_device(module, tensor_name, device, value, fp16_statistics)
     56     old_value = getattr(module, tensor_name)
     57 
---> 58     if old_value.device == torch.device("meta") and device not in ["meta", torch.device("meta")] and value is None:
     59         raise ValueError(f"{tensor_name} is on the meta device, we need a `value` to put in on {device}.")
     60 

NameError: name 'torch' is not defined
```

### Expected behavior

```
from torch import cuda, bfloat16
import transformers

model_id = 'meta-llama/Llama-2-7b-chat-hf'

device = f'cuda:{cuda.current_device()}' if cuda.is_available() else 'cpu'

# set quantization configuration to load large model with less GPU memory
# this requires the `bitsandbytes` library
bnb_config = transformers.BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type='nf4',
    bnb_4bit_use_double_quant=True,
    bnb_4bit_compute_dtype=bfloat16
)

# begin initializing HF items, you need an access token
hf_auth = 'HF Token Here'
model_config = transformers.AutoConfig.from_pretrained(
    model_id,
    use_auth_token=hf_auth
)

model = transformers.AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    config=model_config,
    quantization_config=bnb_config,
    device_map='auto',
    use_auth_token=hf_auth
)

# enable evaluation mode to allow model inference
model.eval()

print(f"Model loaded on {device}")
 ```

```
/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration_auto.py:1067: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
  warnings.warn(

config.json:   0%|          | 0.00/614 [00:00<?, ?B/s]
/usr/local/lib/python3.10/dist-packages/transformers/models/auto/auto_factory.py:472: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
  warnings.warn(
model.safetensors.index.json:   0%|          | 0.00/26.8k [00:00<?, ?B/s]
Downloading shards:   0%|          | 0/2 [00:00<?, ?it/s]
model-00001-of-00002.safetensors:   0%|          | 0.00/9.98G [00:00<?, ?B/s]
model-00002-of-00002.safetensors:   0%|          | 0.00/3.50G [00:00<?, ?B/s]
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
generation_config.json:   0%|          | 0.00/188 [00:00<?, ?B/s]
Model loaded on cuda:0
```


Comments:
The error is misleading, it actually means that bitsandbytes is not available. That's because you're using Windows, and the library only works on Linux (see [here](https://github.com/TimDettmers/bitsandbytes#tldr)).

Note that you leaked your token twice, you should rotate it.

Would you like to open a PR for a fix? 🤗 

Yes, I'd be happy to submit a fix.

Actually I thought the issue was in `src/transformers/integrations/bitsandbytes.py`:
```python
if is_bitsandbytes_available():  
    import bitsandbytes as bnb  
    import torch  
    import torch.nn as nn
```
because it only imports torch if bitsandbytes is available. When torch is used later, it raises "NameError: name 'torch' is not defined".

But when PyTorch is properly installed, the function `transformers.utils.import_utils._get_module()` is called before the code above (I missed this when I answered) and it raises a RuntimeError that clearly indicates that the issue comes from bitsandbytes. So I don't think there's anything to fix.

Alright, closing as completed then! 

@KaifAhmad1 don't forget to [install PyTorch](https://pytorch.org/get-started/locally/), and you can follow the instructions [here](https://github.com/TimDettmers/bitsandbytes/issues/807) to install bitsandbytes on Windows.