Spaces:

hunterXdk
/

RagModels

Sleeping

hunterXdk commited on Nov 26, 2024

Commit

deb7ff3

verified ·

1 Parent(s): 71beb2b

removed device_map

Files changed (1) hide show

app.py CHANGED Viewed

@@ -14,7 +14,7 @@ import torch
 # Load a Hugging Face model (e.g., LLaMA or Falcon)
 model_name = "mixedbread-ai/mxbai-embed-2d-large-v1"  # Replace with your preferred model
 tokenizer = AutoTokenizer.from_pretrained(model_name)
-model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.float16, device_map="auto")

 # Load a Hugging Face model (e.g., LLaMA or Falcon)
 model_name = "mixedbread-ai/mxbai-embed-2d-large-v1"  # Replace with your preferred model
 tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.float16)