Spaces:
Sleeping
Sleeping
Commit History
Fix: when using http worker, only download if inference is on localhost
52fc709
Add retry logic upon schema fail for Mistral API calls
1e622b4
Fix import
cf4506f
Add log_to_jsonl function to data.py and remove duplicate function from utils.py
44bab49
Add JSONL disk file logging functionality to app.py and utils.py
abbebf8
Tidying: reordering code a bit
124003a
Add Mistral API support due to my RunPod serverless system reliability issues
8093276
Add TODO for Runpod timeout handling
3c6c618
Assert RunPod env vars are setup before trying to use them
00af17e
Revert expected serverless output metadata stripper
c013599
Luke Stanley
commited on
Avoid unneeded imports, make serverless output more sensible, removing some debugging and comments
469f650
Luke Stanley
commited on
Fix RUNPOD_ENDPOINT_ID environment variable
ce5ad5f
Luke Stanley
commited on
Introduces worker mode env var
56e785c
Luke Stanley
commited on
Ensure N_GPU_LAYERS is int
9475016
Luke Stanley
commited on
Update default GPU layer, temperature values
e327a9e
Add env vars to set GPU layer count and context size, make verbose
e01e28e
Add n_gpu_layers parameter to Llama initialization
88e6118
Fix: Move n_ctx parameter to model setup!
358cd20
Fix check for LLM_MODEL_PATH to avoid load error
ff938c3
Auto-downloads model if env var is not set
74d6e52
Luke Stanley
commited on
Make llm_stream_sans_network actually stream to stdout
a0f49a0
Luke Stanley
commited on
Default to in-memory LLM interface
ddb0d91
Luke Stanley
commited on