Instructions to use thesantoshbist/fwu-llm with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use thesantoshbist/fwu-llm with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="thesantoshbist/fwu-llm")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("thesantoshbist/fwu-llm")
model = AutoModelForCausalLM.from_pretrained("thesantoshbist/fwu-llm", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use thesantoshbist/fwu-llm with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "thesantoshbist/fwu-llm"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "thesantoshbist/fwu-llm",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/thesantoshbist/fwu-llm

SGLang

How to use thesantoshbist/fwu-llm with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "thesantoshbist/fwu-llm" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "thesantoshbist/fwu-llm",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "thesantoshbist/fwu-llm" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "thesantoshbist/fwu-llm",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use thesantoshbist/fwu-llm with Docker Model Runner:
```
docker model run hf.co/thesantoshbist/fwu-llm
```

FWU Assistant Model

Model Description

This is a fine-tuned LLM model specialized for Far Western University (FWU) information and educational assistance. The model has been customized to provide accurate information about FWU programs, courses, admissions, faculty, and campus resources while maintaining general conversational abilities.

Training Data

This model was trained on:

Conversations with students and faculty at FWU
Academic information and educational resources
General knowledge with emphasis on educational contexts

Capabilities

Answers questions about Far Western University programs and policies
Provides assistance with academic inquiries
Helps with general knowledge questions
Maintains conversational context for natural interactions

Use Cases

Student information services
Academic guidance
Educational assistance
University information desk
Virtual campus guide
Conversational AI
Question answering
Text generation

Limitations

Limited knowledge of events after training cutoff
May occasionally provide incorrect information
Not a replacement for official university guidance
Doesn't have access to student records or private information

Ethical Considerations

This model is intended for educational and informational purposes only. It should not be used for making critical academic or administrative decisions without verification from official university sources.

Additional Information

Developed by Santosh Bist at Far Western University. For issues or feedback, please contact thesantoshbist@gmail.com.

Model Description

Custom LLM 1B Parameter mainly for FWU or Far Western University.

Author

Santosh Bist

Version

1.0.0

Downloads last month: 10

Safetensors

Model size

1B params

Tensor type

F16