HF pipeline throws error
Author: sindhuvahinis
Pull request: no

### System Info

HF Pipeline actually trying to generate the outputs on CPU despite including the device_map=auto as configuration for GPT_NeoX 20B model.

Workaround is to use model.generate method by manually converting the input_ids to GPU.

```
[INFO ] PyProcess -     prediction = self.hf_pipeline(data, **parameters)
[INFO ] PyProcess -   File "/usr/local/lib/python3.8/dist-packages/transformers/pipelines/text_generation.py", line 187, in __call__
[INFO ] PyProcess -     return super().__call__(text_inputs, **kwargs)
[INFO ] PyProcess -   File "/usr/local/lib/python3.8/dist-packages/transformers/pipelines/base.py", line 1063, in __call__
[INFO ] PyProcess -     outputs = [output for output in final_iterator]
[INFO ] PyProcess -   File "/usr/local/lib/python3.8/dist-packages/transformers/pipelines/base.py", line 1063, in <listcomp>
[INFO ] PyProcess -     outputs = [output for output in final_iterator]
[INFO ] PyProcess -   File "/usr/local/lib/python3.8/dist-packages/transformers/pipelines/pt_utils.py", line 111, in __next__
[INFO ] PyProcess -     item = next(self.iterator)
[INFO ] PyProcess -   File "/usr/local/lib/python3.8/dist-packages/transformers/pipelines/pt_utils.py", line 112, in __next__
[INFO ] PyProcess -     processed = self.infer(item, **self.params)
[INFO ] PyProcess -   File "/usr/local/lib/python3.8/dist-packages/transformers/pipelines/base.py", line 990, in forward
[INFO ] PyProcess -     model_outputs = self._forward(model_inputs, **forward_params)
[INFO ] PyProcess -   File "/usr/local/lib/python3.8/dist-packages/transformers/pipelines/text_generation.py", line 229, in _forward
[INFO ] PyProcess -     generated_sequence = self.model.generate(input_ids=input_ids, attention_mask=attention_mask, **generate_kwargs)
[INFO ] PyProcess -   File "/usr/local/lib/python3.8/dist-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
[INFO ] PyProcess -     return func(*args, **kwargs)
[INFO ] PyProcess -   File "/usr/local/lib/python3.8/dist-packages/transformers/generation_utils.py", line 1422, in generate
[INFO ] PyProcess -     return self.sample(
[INFO ] PyProcess -   File "/usr/local/lib/python3.8/dist-packages/transformers/generation_utils.py", line 2049, in sample
[INFO ] PyProcess -     next_token_scores = logits_warper(input_ids, next_token_scores)
[INFO ] PyProcess -   File "/usr/local/lib/python3.8/dist-packages/transformers/generation_logits_process.py", line 92, in __call__
[INFO ] PyProcess -     scores = processor(input_ids, scores)
[INFO ] PyProcess -   File "/usr/local/lib/python3.8/dist-packages/transformers/generation_logits_process.py", line 233, in __call__
[INFO ] PyProcess -     indices_to_remove = scores < torch.topk(scores, top_k)[0][..., -1, None]
[INFO ] PyProcess - RuntimeError: "topk_cpu" not implemented for 'Half'
```


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

https://github.com/deepjavalibrary/djl-serving/blob/master/engines/python/setup/djl_python/huggingface.py#L129 - This is the code we used to test. 

### Expected behavior

This error is happening for only GPT Neox 20B https://huggingface.co/EleutherAI/gpt-neox-20b. It worked for Bloom 7B amd gptj models. 

Comments:
Hey! Would you mind providing a minimal reproducing script? 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.