Instructions to use jin05102518/Astral-7B-Instruct-v0.01 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use jin05102518/Astral-7B-Instruct-v0.01 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="jin05102518/Astral-7B-Instruct-v0.01")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForMultimodalLM

tokenizer = AutoTokenizer.from_pretrained("jin05102518/Astral-7B-Instruct-v0.01")
model = AutoModelForMultimodalLM.from_pretrained("jin05102518/Astral-7B-Instruct-v0.01")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use jin05102518/Astral-7B-Instruct-v0.01 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "jin05102518/Astral-7B-Instruct-v0.01"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jin05102518/Astral-7B-Instruct-v0.01",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/jin05102518/Astral-7B-Instruct-v0.01

SGLang

How to use jin05102518/Astral-7B-Instruct-v0.01 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "jin05102518/Astral-7B-Instruct-v0.01" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jin05102518/Astral-7B-Instruct-v0.01",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "jin05102518/Astral-7B-Instruct-v0.01" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jin05102518/Astral-7B-Instruct-v0.01",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use jin05102518/Astral-7B-Instruct-v0.01 with Docker Model Runner:
```
docker model run hf.co/jin05102518/Astral-7B-Instruct-v0.01
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

영어 Instruct 데이터셋을 사용했고 한국어 데이터셋은 오직 KoAlpaca-v1.1a 데이터셋만을 Full-Finetune으로 학습한 Mistral 모델입니다.

Prompt 예시

prompt = "<|system|>\n</s>\n<|user|>\n쿠버네티스에 대해 알려줘</s>"

구동 예시

<s> <|system|>
</s>    
<|user|>
쿠버네티스에 대해 설명해 줘</s>    
<|assistant|>
쿠버네티스는 컨테이너를 관리하는 오픈소스 시스템입니다. 쿠버네티스는 컨테이너를 실행하고, 컨테이너가 실행되는 환경을 제공합니다. 쿠버네티스는 여러 개의 호스트 컴퓨터를 관리하며, 이러한 호스트 컴퓨터를 통해 컨테이너를 실행할 수 있습니다. 쿠버네티스는 컨테이너를 배치하고 관리하는 기능을 제공합니다. 또한, 쿠버네티스는 서비스 메쉬, 스토리지, 배치 컨트롤 플러그인 등 다양한 기능을 제공합니다. 이러한 기능들은 모두 컨테이너를 관리하고 배치하는데 유용하며, 쿠버네티스는 이러한 기능들을 통해 컨테이너를 효과적으로 관리하고 배치할 수 있는 시스템으로 자리 잡고 있습니다.</s>   
<|user|>
위의 설명을 영어로 번역해 줘</s>   
<|assistant|>
Kubernetes is an open-source system for container management. Kubernetes manages the containers, and provides the environment for running the containers. Kubernetes manages multiple hosts, and runs the containers on these hosts. Kubernetes is a system for deploying and managing containers. Kubernetes also provides features such as service mesh, storage, and deployment control plugins. All of these features are useful for managing and deploying containers. Kubernetes is a system that effectively manages containers and deploys them efficiently.</s>

알립니다.

해당 모델은 시험적이며 기술적 시험을 위해 일시적으로 만들어졌습니다.

Downloads last month: 4

Model tree for jin05102518/Astral-7B-Instruct-v0.01

Quantizations

1 model

jin05102518
/

Astral-7B-Instruct-v0.01

Prompt 예시

구동 예시

알립니다.

Model tree for jin05102518/Astral-7B-Instruct-v0.01

Dataset used to train jin05102518/Astral-7B-Instruct-v0.01

Space using jin05102518/Astral-7B-Instruct-v0.01 1