fix device

by ariG23498 HF staff - opened Oct 15

←

IBM Granite org Oct 15

•

The device="auto" does not work when we offload a torch tensor.

A better way to use is to map device to auto in the model, and later use model.device for offloading the input tensors!

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Ready to merge

This branch is ready to get merged automatically.

· Sign up or log in to comment