Running Phi1.0 on a Mac M2
Author: erlebach
Pull request: no

### System Info

transformers-cli env

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

- `transformers` version: 4.33.2
- Platform: macOS-13.5.2-arm64-arm-64bit
- Python version: 3.10.10
- Huggingface_hub version: 0.17.1
- Safetensors version: 0.3.3
- Accelerate version: 0.23.0
- Accelerate config: not found
- PyTorch version (GPU?): 2.0.1 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: No.
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@ArthurZucker  @younesbelkada 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction


I am running the Phi1.0 model with the following Poetry setup on a mac (without CUDA): 
```
[tool.poetry]
name = "my_phi-1"
version = "0.1.0"
description = ""
authors = ["erlebach <gordon.erlebach@gmail.com>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.10"
torch = "^2.0.1"
numpy = "^1.25.2"
transformers = "^4.33.1"
datasets = "^2.14.5"
tiktoken = "^0.4.0"
wandb = "^0.15.10"
tqdm = "^4.66.1"
einops = "^0.6.1"


[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
```
and get a stacktrace when running with Python 10.x: 
```
Traceback (most recent call last):
  File "/Users/erlebach/src/2023/poetry/my_phi-1/phi-1/sample.py", line 12, in <module>
    outputs = model.generate(**inputs, max_length=200)
  File "/Users/erlebach/src/2023/poetry/my_phi-1/.venv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/Users/erlebach/src/2023/poetry/my_phi-1/.venv/lib/python3.10/site-packages/transformers/generation/utils.py", line 1602, in generate
    return self.greedy_search(
  File "/Users/erlebach/src/2023/poetry/my_phi-1/.venv/lib/python3.10/site-packages/transformers/generation/utils.py", line 2450, in greedy_search
    outputs = self(
  File "/Users/erlebach/src/2023/poetry/my_phi-1/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/Users/erlebach/.cache/huggingface/modules/transformers_modules/microsoft/phi-1/621f8448067757fafc89009d08bd7d71ad0239e1/modeling_mixformer_sequential.py", line 771, in forward
    hidden_layer = module(hidden_layer, past_cache=past_key_values)
  File "/Users/erlebach/src/2023/poetry/my_phi-1/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/Users/erlebach/.cache/huggingface/modules/transformers_modules/microsoft/phi-1/621f8448067757fafc89009d08bd7d71ad0239e1/modeling_mixformer_sequential.py", line 625, in forward
    hidden_states = self.ln(hidden_states)
  File "/Users/erlebach/src/2023/poetry/my_phi-1/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/Users/erlebach/src/2023/poetry/my_phi-1/.venv/lib/python3.10/site-packages/torch/nn/modules/normalization.py", line 190, in forward
    return F.layer_norm(
  File "/Users/erlebach/src/2023/poetry/my_phi-1/.venv/lib/python3.10/site-packages/torch/nn/functional.py", line 2512, in layer_norm
    return handle_torch_function(
  File "/Users/erlebach/src/2023/poetry/my_phi-1/.venv/lib/python3.10/site-packages/torch/overrides.py", line 1534, in handle_torch_function
    result = mode.__torch_function__(public_api, types, args, kwargs)
  File "/Users/erlebach/src/2023/poetry/my_phi-1/.venv/lib/python3.10/site-packages/torch/utils/_device.py", line 62, in __torch_function__
    return func(*args, **kwargs)
  File "/Users/erlebach/src/2023/poetry/my_phi-1/.venv/lib/python3.10/site-packages/torch/nn/functional.py", line 2515, in layer_norm
    return torch.layer_norm(input, normalized_shape, weight, bias, eps, torch.backends.cudnn.enabled)
RuntimeError: "LayerNormKernelImpl" not implemented for 'Half'
```
I am not using Cuda. I probably must load a different library? Thank you for any insight!

   Gordon



### Expected behavior

I would expect no stacktrace.

Comments:
Hey! The error suggest that you are running the model in half precision (torch.float16) but layer norm is not supported. You should either try in `bfloat16` or just float32`

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Thank you, Arthur. I'll check it out. Just saw your reply.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.