Instructions to use gqilabs/GQI-Eval-135M with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use gqilabs/GQI-Eval-135M with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="gqilabs/GQI-Eval-135M")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("gqilabs/GQI-Eval-135M") model = AutoModelForCausalLM.from_pretrained("gqilabs/GQI-Eval-135M", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use gqilabs/GQI-Eval-135M with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "gqilabs/GQI-Eval-135M" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "gqilabs/GQI-Eval-135M", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/gqilabs/GQI-Eval-135M
- SGLang
How to use gqilabs/GQI-Eval-135M with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "gqilabs/GQI-Eval-135M" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "gqilabs/GQI-Eval-135M", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "gqilabs/GQI-Eval-135M" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "gqilabs/GQI-Eval-135M", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use gqilabs/GQI-Eval-135M with Docker Model Runner:
docker model run hf.co/gqilabs/GQI-Eval-135M
GQI Evaluation Model Access
Access is limited to approved GQI pilot participants.
By requesting access, you confirm that you have read and agree to the GQI Evaluation Model License.
Log in or Sign Up to review the conditions and access this model content.
GQI Evaluation Model
This restricted checkpoint is supplied only for an approved GQI evaluation. Customer data remains in the evaluator's environment.
It is not the private production GQI model.
Access and license
Access requests are reviewed manually. Access is granted only to named pilot participants who accept the GQI Evaluation Model License, and may be revoked at any time.
The license permits limited internal testing and benchmarking. Production, customer-facing, hosted-service, redistribution, and other commercial use require a separate written agreement with GQI.
Use
Use the separately supplied private handoff package for .fit and .predict
instructions, the frozen example, and the registered evaluation report. This
repository contains no customer data.
Attribution
Required third-party attribution and license notices are included with the gated repository files.
- Downloads last month
- 8