FriskyFennec/Ling-3.0-tiny-base-30T-GGUF

Ling-3.0-tiny-base-30T

A GGUF variant of inclusionAI's Ling-3.0-tiny-base-30T model, suitable for base/foundational model experiments and text completion use cases.

The model was converted & quantized using llama.cpp's build 0.4.0-dev.

Examples

Running via llama.cpp

Model's tested with,

llama-server.exe --model Ling-3.0-tiny-base-30T-BF16.gguf --ctx-size 8192 --gpu-layers 256 --main-gpu 0 --flash-attn auto --mlock --timeout 600 --host 127.0.0.1 --port 8080

Note: The model was trained with 8192 context length (config.json, developer's clarification). For the GGUF variant of the model with full context length (256K), refer to FriskyFennec/Ling-3.0-tiny-base-midtrain-GGUF.

Output

Samplers:

• Temperature: 0.7
• Top P: 0.95
• Top K: 40
• Repetition Penalty: 1.1
• Repetition Penalty Range: 256

Input,

Once upon a time,

Output 1,

 in the world of music, there were two genres that seemed to have little in common: hip hop and rock. However, as the years went on, these two seemingly disparate worlds began to merge, creating a unique sound that captivated audiences worldwide.

Output 2,

 the internet was a place where people shared their photos and videos with friends and family. But then, someone came up with an idea that would change everything: sharing pictures of yourself on social media.
At first, it seemed like a harmless way to

Output 3,

 in the magical land of Minecraft, there lived a character known as The Wither. The Wither is a powerful and mysterious creature that can be found deep underground, often in dark caves. It looks like a giant skeleton with three eyes and is covered

License

The model inherits the original MIT license from inclusionAI/Ling-3.0-tiny-base-30T.