Spaces:

MedTiouti
/

SandHillRoadPodcast

Runtime error

Med Tiouti commited on Jan 13, 2024

Commit

bccedad

1 Parent(s): 69a46d6

Offloader fixed again

Files changed (1) hide show

app.py CHANGED Viewed

@@ -29,7 +29,7 @@ def get_model(model_name):
     model = AutoModelForCausalLM.from_pretrained(
         model_repo,
         device_map='auto',
-        offload="offload",
         torch_dtype=torch.float16,
         low_cpu_mem_usage=True,
         trust_remote_code=True

     model = AutoModelForCausalLM.from_pretrained(
         model_repo,
         device_map='auto',
+        offload_folder="offload",
         torch_dtype=torch.float16,
         low_cpu_mem_usage=True,
         trust_remote_code=True