AppleMind-1.0-Mini-GGUF

Instructions to use AppleMind-AI/AppleMind-1.0-Mini-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AppleMind-AI/AppleMind-1.0-Mini-GGUF with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="AppleMind-AI/AppleMind-1.0-Mini-GGUF")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("AppleMind-AI/AppleMind-1.0-Mini-GGUF", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use AppleMind-AI/AppleMind-1.0-Mini-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf AppleMind-AI/AppleMind-1.0-Mini-GGUF:BF16
# Run inference directly in the terminal:
llama cli -hf AppleMind-AI/AppleMind-1.0-Mini-GGUF:BF16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf AppleMind-AI/AppleMind-1.0-Mini-GGUF:BF16
# Run inference directly in the terminal:
llama cli -hf AppleMind-AI/AppleMind-1.0-Mini-GGUF:BF16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf AppleMind-AI/AppleMind-1.0-Mini-GGUF:BF16
# Run inference directly in the terminal:
./llama-cli -hf AppleMind-AI/AppleMind-1.0-Mini-GGUF:BF16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf AppleMind-AI/AppleMind-1.0-Mini-GGUF:BF16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf AppleMind-AI/AppleMind-1.0-Mini-GGUF:BF16

Use Docker

docker model run hf.co/AppleMind-AI/AppleMind-1.0-Mini-GGUF:BF16

LM Studio
Jan

vLLM

How to use AppleMind-AI/AppleMind-1.0-Mini-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AppleMind-AI/AppleMind-1.0-Mini-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AppleMind-AI/AppleMind-1.0-Mini-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/AppleMind-AI/AppleMind-1.0-Mini-GGUF:BF16

SGLang

How to use AppleMind-AI/AppleMind-1.0-Mini-GGUF with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "AppleMind-AI/AppleMind-1.0-Mini-GGUF" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AppleMind-AI/AppleMind-1.0-Mini-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "AppleMind-AI/AppleMind-1.0-Mini-GGUF" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AppleMind-AI/AppleMind-1.0-Mini-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use AppleMind-AI/AppleMind-1.0-Mini-GGUF with Ollama:
```
ollama run hf.co/AppleMind-AI/AppleMind-1.0-Mini-GGUF:BF16
```

Unsloth Studio

How to use AppleMind-AI/AppleMind-1.0-Mini-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for AppleMind-AI/AppleMind-1.0-Mini-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for AppleMind-AI/AppleMind-1.0-Mini-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for AppleMind-AI/AppleMind-1.0-Mini-GGUF to start chatting

Docker Model Runner
How to use AppleMind-AI/AppleMind-1.0-Mini-GGUF with Docker Model Runner:
```
docker model run hf.co/AppleMind-AI/AppleMind-1.0-Mini-GGUF:BF16
```

Lemonade

How to use AppleMind-AI/AppleMind-1.0-Mini-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull AppleMind-AI/AppleMind-1.0-Mini-GGUF:BF16

Run and chat with the model

lemonade run user.AppleMind-1.0-Mini-GGUF-BF16

List all available models

lemonade list

Atomic Chat

plsss

by AxionLab-official - opened about 13 hours ago

Discussion

AxionLab-official

about 13 hours ago

Please 1Bit quantization

GGUFGuy

AppleMind AI org about 13 hours ago

sadly impossible, the model is too small :(
logs:

llama_model_loader: - kv 9: general.dataset.1.name str = FineWeb HQ
llama_model_loader: - kv 10: general.dataset.1.organization str = Epfml
llama_model_loader: - kv 11: general.dataset.1.repo_url str = https://huggingface.co/epfml/FineWeb-HQ
llama_model_loader: - kv 12: general.dataset.2.name str = Smollm Corpus
llama_model_loader: - kv 13: general.dataset.2.organization str = HuggingFaceTB
llama_model_loader: - kv 14: general.dataset.2.repo_url str = https://huggingface.co/HuggingFaceTB/...
llama_model_loader: - kv 15: general.tags arr[str,21] = ["lm", "language-model", "causal-lm",...
llama_model_loader: - kv 16: general.languages arr[str,1] = ["en"]
llama_model_loader: - kv 17: gpt2.block_count u32 = 2
llama_model_loader: - kv 18: gpt2.context_length u32 = 256
llama_model_loader: - kv 19: gpt2.embedding_length u32 = 20
llama_model_loader: - kv 20: gpt2.feed_forward_length u32 = 80
llama_model_loader: - kv 21: gpt2.attention.head_count u32 = 2
llama_model_loader: - kv 22: gpt2.attention.layer_norm_epsilon f32 = 0.000010
llama_model_loader: - kv 23: general.file_type u32 = 1
llama_model_loader: - kv 24: general.quantization_version u32 = 2
llama_model_loader: - kv 25: tokenizer.ggml.model str = gpt2
llama_model_loader: - kv 26: tokenizer.ggml.pre str = gpt-2
llama_model_loader: - kv 27: tokenizer.ggml.tokens arr[str,50260] = ["!", """, "#", "$", "%", "&", "'", ...
llama_model_loader: - kv 28: tokenizer.ggml.token_type arr[i32,50260] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv 29: tokenizer.ggml.merges arr[str,50000] = ["Ġ t", "Ġ a", "h e", "i n", "r e",...
llama_model_loader: - kv 30: tokenizer.ggml.bos_token_id u32 = 50256
llama_model_loader: - kv 31: tokenizer.ggml.eos_token_id u32 = 50256
llama_model_loader: - kv 32: tokenizer.ggml.unknown_token_id u32 = 50256
llama_model_loader: - kv 33: tokenizer.ggml.padding_token_id u32 = 50256
llama_model_loader: - kv 34: tokenizer.chat_template str = {% for message in messages %}{{ '<|' ...
llama_model_loader: - type f32: 19 tensors
llama_model_loader: - type f16: 9 tensors
warning: token_embd.weight - ncols 20 not divisible by 32 (required for type q8_0) llama_model_quantize: failed to quantize: no tensor type fallback is defined for type q8_0
llama_quantize: failed to quantize model from '/home/japugalt/llama.cpp/quantize/AppleMind-1.0-Mini/AppleMind-1.0-Mini-F16.gguf'

Finished in 0.0 seconds.

✗ FAILED: Q5_K_M
Exit code: 1

################################################################################

Q6_K

################################################################################

================================================================================
RUNNING

/home/japugalt/llama.cpp/build/bin/llama-quantize /home/japugalt/llama.cpp/quantize/AppleMind-1.0-Mini/AppleMind-1.0-Mini-F16.gguf /home/japugalt/llama.cpp/quantize/AppleMind-1.0-Mini/AppleMind-1.0-Mini-Q6_K.gguf Q6_K

version: 0.1.0-dev (build 10398, commit 8e7f22b67)
built with GNU 16.1.1 for Linux x86_64
llama_quantize: quantizing '/home/japugalt/llama.cpp/quantize/AppleMind-1.0-Mini/AppleMind-1.0-Mini-F16.gguf' to '/home/japugalt/llama.cpp/quantize/AppleMind-1.0-Mini/AppleMind-1.0-Mini-Q6_K.gguf' as Q6_K
llama_model_loader: loaded meta data with 35 key-value pairs and 28 tensors from /home/japugalt/llama.cpp/quantize/AppleMind-1.0-Mini/AppleMind-1.0-Mini-F16.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = gpt2
llama_model_loader: - kv 1: general.type str = model
llama_model_loader: - kv 2: general.name str = Edd1E6E300Aca8Dc2E387Eaa780B944659De5C6C
llama_model_loader: - kv 3: general.size_label str = 1.0M
llama_model_loader: - kv 4: general.license str = apache-2.0
llama_model_loader: - kv 5: general.dataset.count u32 = 3
llama_model_loader: - kv 6: general.dataset.0.name str = Fineweb Edu
llama_model_loader: - kv 7: general.dataset.0.organization str = HuggingFaceFW
llama_model_loader: - kv 8: general.dataset.0.repo_url str = https://huggingface.co/HuggingFaceFW/...
llama_model_loader: - kv 9: general.dataset.1.name str = FineWeb HQ
llama_model_loader: - kv 10: general.dataset.1.organization str = Epfml
llama_model_loader: - kv 11: general.dataset.1.repo_url str = https://huggingface.co/epfml/FineWeb-HQ
llama_model_loader: - kv 12: general.dataset.2.name str = Smollm Corpus
llama_model_loader: - kv 13: general.dataset.2.organization str = HuggingFaceTB
llama_model_loader: - kv 14: general.dataset.2.repo_url str = https://huggingface.co/HuggingFaceTB/...
llama_model_loader: - kv 15: general.tags arr[str,21] = ["lm", "language-model", "causal-lm",...
llama_model_loader: - kv 16: general.languages arr[str,1] = ["en"]
llama_model_loader: - kv 17: gpt2.block_count u32 = 2
llama_model_loader: - kv 18: gpt2.context_length u32 = 256
llama_model_loader: - kv 19: gpt2.embedding_length u32 = 20
llama_model_loader: - kv 20: gpt2.feed_forward_length u32 = 80
llama_model_loader: - kv 21: gpt2.attention.head_count u32 = 2
llama_model_loader: - kv 22: gpt2.attention.layer_norm_epsilon f32 = 0.000010
llama_model_loader: - kv 23: general.file_type u32 = 1
llama_model_loader: - kv 24: general.quantization_version u32 = 2
llama_model_loader: - kv 25: tokenizer.ggml.model str = gpt2
llama_model_loader: - kv 26: tokenizer.ggml.pre str = gpt-2
llama_model_loader: - kv 27: tokenizer.ggml.tokens arr[str,50260] = ["!", """, "#", "$", "%", "&", "'", ...
llama_model_loader: - kv 28: tokenizer.ggml.token_type arr[i32,50260] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv 29: tokenizer.ggml.merges arr[str,50000] = ["Ġ t", "Ġ a", "h e", "i n", "r e",...
llama_model_loader: - kv 30: tokenizer.ggml.bos_token_id u32 = 50256
llama_model_loader: - kv 31: tokenizer.ggml.eos_token_id u32 = 50256
llama_model_loader: - kv 32: tokenizer.ggml.unknown_token_id u32 = 50256
llama_model_loader: - kv 33: tokenizer.ggml.padding_token_id u32 = 50256
llama_model_loader: - kv 34: tokenizer.chat_template str = {% for message in messages %}{{ '<|' ...
llama_model_loader: - type f32: 19 tensors
llama_model_loader: - type f16: 9 tensors
warning: token_embd.weight - ncols 20 not divisible by 32 (required for type q8_0) llama_model_quantize: failed to quantize: no tensor type fallback is defined for type q8_0
llama_quantize: failed to quantize model from '/home/japugalt/llama.cpp/quantize/AppleMind-1.0-Mini/AppleMind-1.0-Mini-F16.gguf'

Finished in 0.0 seconds.

✗ FAILED: Q6_K
Exit code: 1

################################################################################

Q8_0

################################################################################

================================================================================
RUNNING

version: 0.1.0-dev (build 10398, commit 8e7f22b67)
built with GNU 16.1.1 for Linux x86_64
llama_quantize: quantizing '/home/japugalt/llama.cpp/quantize/AppleMind-1.0-Mini/AppleMind-1.0-Mini-F16.gguf' to '/home/japugalt/llama.cpp/quantize/AppleMind-1.0-Mini/AppleMind-1.0-Mini-Q8_0.gguf' as Q8_0
llama_model_loader: loaded meta data with 35 key-value pairs and 28 tensors from /home/japugalt/llama.cpp/quantize/AppleMind-1.0-Mini/AppleMind-1.0-Mini-F16.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = gpt2
llama_model_loader: - kv 1: general.type str = model
llama_model_loader: - kv 2: general.name str = Edd1E6E300Aca8Dc2E387Eaa780B944659De5C6C
llama_model_loader: - kv 3: general.size_label str = 1.0M
llama_model_loader: - kv 4: general.license str = apache-2.0
llama_model_loader: - kv 5: general.dataset.count u32 = 3
llama_model_loader: - kv 6: general.dataset.0.name str = Fineweb Edu
llama_model_loader: - kv 7: general.dataset.0.organization str = HuggingFaceFW
llama_model_loader: - kv 8: general.dataset.0.repo_url str = https://huggingface.co/HuggingFaceFW/...
llama_model_loader: - kv 9: general.dataset.1.name str = FineWeb HQ
llama_model_loader: - kv 10: general.dataset.1.organization str = Epfml
llama_model_loader: - kv 11: general.dataset.1.repo_url str = https://huggingface.co/epfml/FineWeb-HQ
llama_model_loader: - kv 12: general.dataset.2.name str = Smollm Corpus
llama_model_loader: - kv 13: general.dataset.2.organization str = HuggingFaceTB
llama_model_loader: - kv 14: general.dataset.2.repo_url str = https://huggingface.co/HuggingFaceTB/...
llama_model_loader: - kv 15: general.tags arr[str,21] = ["lm", "language-model", "causal-lm",...
llama_model_loader: - kv 16: general.languages arr[str,1] = ["en"]
llama_model_loader: - kv 17: gpt2.block_count u32 = 2
llama_model_loader: - kv 18: gpt2.context_length u32 = 256
llama_model_loader: - kv 19: gpt2.embedding_length u32 = 20
llama_model_loader: - kv 20: gpt2.feed_forward_length u32 = 80
llama_model_loader: - kv 21: gpt2.attention.head_count u32 = 2
llama_model_loader: - kv 22: gpt2.attention.layer_norm_epsilon f32 = 0.000010
llama_model_loader: - kv 23: general.file_type u32 = 1
llama_model_loader: - kv 24: general.quantization_version u32 = 2
llama_model_loader: - kv 25: tokenizer.ggml.model str = gpt2
llama_model_loader: - kv 26: tokenizer.ggml.pre str = gpt-2
llama_model_loader: - kv 27: tokenizer.ggml.tokens arr[str,50260] = ["!", """, "#", "$", "%", "&", "'", ...
llama_model_loader: - kv 28: tokenizer.ggml.token_type arr[i32,50260] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv 29: tokenizer.ggml.merges arr[str,50000] = ["Ġ t", "Ġ a", "h e", "i n", "r e",...
llama_model_loader: - kv 30: tokenizer.ggml.bos_token_id u32 = 50256
llama_model_loader: - kv 31: tokenizer.ggml.eos_token_id u32 = 50256
llama_model_loader: - kv 32: tokenizer.ggml.unknown_token_id u32 = 50256
llama_model_loader: - kv 33: tokenizer.ggml.padding_token_id u32 = 50256
llama_model_loader: - kv 34: tokenizer.chat_template str = {% for message in messages %}{{ '<|' ...
llama_model_loader: - type f32: 19 tensors
llama_model_loader: - type f16: 9 tensors
warning: token_embd.weight - ncols 20 not divisible by 32 (required for type q8_0) llama_model_quantize: failed to quantize: no tensor type fallback is defined for type q8_0
llama_quantize: failed to quantize model from '/home/japugalt/llama.cpp/quantize/AppleMind-1.0-Mini/AppleMind-1.0-Mini-F16.gguf'

Finished in 0.0 seconds.

✗ FAILED: Q8_0
Exit code: 1

================================================================================
FINISHED

Successful: 0

Skipped: 0

Failed: 31
✗ Q1_0
✗ Q2_0
✗ Q4_0
✗ Q4_1
✗ Q5_0
✗ Q5_1
✗ IQ1_S
✗ IQ1_M
✗ IQ2_XXS
✗ IQ2_XS
✗ IQ2_S
✗ IQ2_M
✗ TQ1_0
✗ TQ2_0
✗ Q2_K
✗ Q2_K_S
✗ IQ3_XXS
✗ IQ3_XS
✗ IQ3_S
✗ IQ3_M
✗ Q3_K_S
✗ Q3_K_M
✗ Q3_K_L
✗ IQ4_NL
✗ IQ4_XS
✗ Q4_K_S
✗ Q4_K_M
✗ Q5_K_S
✗ Q5_K_M
✗ Q6_K
✗ Q8_0

================================================================================
GGUF FILES

AppleMind-1.0-Mini-BF16.gguf 3.65 MiB
AppleMind-1.0-Mini-F16.gguf 3.65 MiB
AppleMind-1.0-Mini-F32.gguf 5.58 MiB

Total storage: 12.87 MiB

GGUF directory:
/home/japugalt/llama.cpp/quantize/AppleMind-1.0-Mini

================================================================================
FAILED FORMATS

Q1_0
Q2_0
Q4_0
Q4_1
Q5_0
Q5_1
IQ1_S
IQ1_M
IQ2_XXS
IQ2_XS
IQ2_S
IQ2_M
TQ1_0
TQ2_0
Q2_K
Q2_K_S
IQ3_XXS
IQ3_XS
IQ3_S
IQ3_M
Q3_K_S
Q3_K_M
Q3_K_L
IQ4_NL
IQ4_XS
Q4_K_S
Q4_K_M
Q5_K_S
Q5_K_M
Q6_K
Q8_0

The other formats were still generated.
Download complete: : █████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7.45MB, 1.80MB/s
Reconstruction complete: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7.77MB / 7.77MB, 1.39MB/s

GGUFGuy

AppleMind AI org about 13 hours ago

not even q8 is possible

AxionLab-official

about 12 hours ago

Oh, ok :(

AxionLab-official changed discussion status to closed about 12 hours ago

GGUFGuy

AppleMind AI org about 12 hours ago

you will have to wait until applemind 1.0 medium releases (5M params)

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment

plsss

Q6_K

================================================================================RUNNING

Q8_0

================================================================================RUNNING

================================================================================FINISHED

================================================================================GGUF FILES

================================================================================FAILED FORMATS

================================================================================
RUNNING

================================================================================
RUNNING

================================================================================
FINISHED

================================================================================
GGUF FILES

================================================================================
FAILED FORMATS