

TITLE
Adding GPTNeoX (Tensorflow version)

COMMENTS
8

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 0
rocket: 0
eyes: 0

BODY
### Model description

Huggingface has GPTNeoX model by ElutherAI. It's a 20 billion parameter autoregressive language model trained on the Pile, whose weights will be made freely and openly available to the public through a permissive license.

However Huggingface currently has only PyTorch implementation of the model.
I would like to contribute it's corresponding TensorFlow implementation.

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

_No response_