How to update context_window here (as by default its 512)

by ianuvrat - opened Nov 7, 2023

Nov 7, 2023

llm = AutoModelForCausalLM.from_pretrained("TheBloke/MistralLite-7B-GGUF",
model_file="mistrallite.Q4_K_M.gguf",
model_type="mistral",
gpu_layers=50)

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment