OWG/gpt-j-6B

GPT-J 6B

Model Description

GPT-J 6B is a transformer model trained using Ben Wang's Mesh Transformer JAX. "GPT-J" refers to the class of model, while "6B" represents the number of trainable parameters.

Original implementation

Follow this link to see the original implementation.

How to use

Download the model by cloning the repository via git clone https://huggingface.co/OWG/gpt-j-6B.

Then you can use the model with the following code:

from onnxruntime import InferenceSession, SessionOptions, GraphOptimizationLevel
from transformers import AutoTokenizer


tokenizer = AutoTokenizer.from_pretrained("EleutherAI/gpt-j-6B")

options = SessionOptions()
options.graph_optimization_level = GraphOptimizationLevel.ORT_ENABLE_ALL

session = InferenceSession("path/to/model.onnx", sess_options=options)
session.disable_fallback()

TODO