Can it run on CPU?
#8
by
BastiNi
- opened
I tried to run the model on my local laptop, but I keep getting the following error on inference:
RuntimeError: "LayerNormKernelImpl" not implemented for 'Half'
I'm loading the model using
model = GPTJForCausalLM.from_pretrained("EleutherAI/gpt-j-6B", revision="float16", torch_dtype=torch.float16, low_cpu_mem_usage=True)
I tried several PyTorch and hugging face versions, including the ones from this repo (torch==1.9.1, transformers==4.12.3).
Maybe the model is not supporting CPU inference?
Try using the float32 version if you want to do CPU inference
stellaathena
changed discussion status to
closed