unsloth

/

Kimi-K3-GGUF

like 350

Follow

Unsloth AI 31k

could ya do more bits / quant showcases? would be intresting to see what quant is beter visualy

That is neat!

I wish Unsloth could partner with Artificial Analysis or Datacurve to run some benchmarks across various quants. Not just K3, but other models too. I find the typical KLD measurements to be an unsatisfying alternative, and it would be some great research to see how much quantization degrades models across a wide range of benchmarks.

@danielhanchen Can you paste the full prompt please for the aquarium want to compare 2 bit vs Fable 5

Can you even get tool calls to work?

Gemini, brilliant as ever.

1-bit Kimi K3 its already beyond 1 last year frontier closed weight.

Gemini, brilliant as ever.

You have to use the full prompt:
Create a large glass aquarium whose side panel develops a visible crack and then bursts.
The simulation must include:
Water escaping through the opening with flow strength based on water depth and decreasing as the tank drains
A curved water jet affected by gravity
A spreading puddle that collides with the room boundaries
Fish, rocks, plants, and a floating toy reacting differently according to density, buoyancy, drag, and current
Objects transitioning correctly from underwater motion to airborne motion and then to floor collisions
Fish attempting to swim against the current before being swept through the breach
Glass fragments with angular velocity, collisions, and water resistance
A visible waterline that lowers continuously rather than disappearing all at once
Let the user drag the crack vertically before triggering the failure. A lower crack should initially produce a stronger jet than a higher crack.

Is it 5.6 sol?

I love you Daniel. thank you Unsloth!

Kimi K3 via Moonshot API vs Kimi K3 from unsloth

(2-bit, Kimi-K3-GGUF/UD-Q2_K_XL. Deployed on EC2 instance with 8xB200)

Kimi K3 via Moonshot API vs Kimi K3 from unsloth

(2-bit, Kimi-K3-GGUF/UD-Q2_K_XL. Deployed on EC2 instance with 8xB200)

---

Click to expand full prompt

You are a senior real-time graphics engineer and 3D technical artist. Build and verify a polished Three.js/WebGL aquarium failure simulation in the current working directory.

OBJECTIVE

Create a glass aquarium whose visible side panel develops a branching crack, remains visibly fractured long enough to read clearly, and then bursts outward. Glass shards, escaping water, and the falling water level must form one coherent, deterministic sequence.

DELIVERABLES

• Create ./index.html as the sole application file.
• Optional Blender-generated runtime assets may be saved beside it as ./aquarium-*.glb.
• Do not create a package.json, build system, source directory, documentation file, or server script.
• Keep every runtime asset in the current working directory and reference it with a relative path.
• Use Blender MCP only if it materially improves the result. If unavailable or unnecessary, generate the entire scene procedurally with Three.js.
• A pinned CDN may be used only for Three.js, OrbitControls, and GLTFLoader. Do not use remote models, textures, HDRIs, or asset-generation services.

BEFORE IMPLEMENTING

1. Inspect the current directory and available tools.
2. If index.html already exists, inspect it and preserve unrelated work; do not overwrite it blindly.
3. Present a concise implementation plan covering scene construction, fracture geometry, animation states, water escape, and verification.
4. Then implement the complete experience. Do not stop after planning or merely provide code in chat.

TECHNICAL REQUIREMENTS

• Use Three.js with WebGLRenderer; do not fake the scene with Canvas 2D, a video, or CSS animation.
• Keep the implementation inside index.html using HTML, CSS, and JavaScript modules.
• Use physically convincing materials:
  • transparent glass with transmission, roughness, thickness, and IOR;
  • a separately modeled water volume;
  • a visible floor and environment that make transparency and reflections readable.
• Add restrained lighting, shadows, and a camera angle that clearly exposes the failing side panel.
• Include enough aquarium detail to establish scale, such as gravel, rocks, plants, bubbles, or fish, without distracting from the failure.
• Use responsive rendering, correct pixel ratio handling, resize support, and OrbitControls.
• Use a seeded pseudo-random generator so every replay produces the same fracture and burst.

FAILURE SEQUENCE

Implement an explicit state machine: intact → cracking → cracked → bursting → aftermath.

1. Intact:

   • The aquarium is stable and filled with water.
   • Hold briefly so the initial condition is readable.

2. Cracking:

   • A crack begins from a clear stress point on one visible side panel.
   • It propagates over time as a branching fracture network.
   • Render the fracture slightly offset from the glass to prevent z-fighting.
   • Give it enough contrast to remain visible over both water and background.
   • Do not substitute a single straight line, texture swap, or instant appearance.

3. Cracked:

   • Hold the completed fracture visibly for at least one second before failure.
   • Add subtle glass stress cues such as vibration, fine secondary branches, or creaking motion.

4. Bursting:

   • Break only the affected side panel; the rest of the aquarium must remain recognizable.
   • Replace that panel with at least 30 irregular triangular or polygonal shards derived from the same fracture region.
   • Shards must originate on the panel plane, move outward, rotate independently, fall under gravity, and interact approximately with the floor.
   • Water must escape through the failed side rather than expanding uniformly in every direction.
   • Represent the escaping water with a performant particle, point, or instanced-mesh system whose velocity, gravity, spread, and opacity create a convincing surge.
   • Lower or deform the remaining water volume during the spill so it does not appear infinitely full.
   • Add a restrained impact cue such as droplets, mist, a spreading puddle, or camera shake.

5. Aftermath:

   • Leave the broken tank, scattered shards, reduced water level, and spilled water visible.
   • Keep animation running for settling particles and small residual effects.

INTERACTION

• Start the sequence automatically after a short delay.
• Provide compact, accessible Replay and Reset controls.
• Spacebar triggers or replays the failure; R resets it.
• Display a small state label so the current phase can be verified.
• Replay and reset must restore every object, timer, particle, shard, and camera effect without reloading the page.

QUALITY AND PERFORMANCE

• Favor a coherent illusion over an expensive full fluid simulation.
• Use object pooling, Points, or InstancedMesh where appropriate.
• Avoid per-frame geometry creation and unbounded allocations.
• Cap frame delta after tab suspension.
• Ensure the default camera never starts inside the glass or water.
• Maintain a usable experience on both desktop and mobile-sized viewports.
• Add a WebGL-unavailable fallback message.
• Do not add unrelated UI, frameworks, analytics, authentication, or speculative features.

VERIFICATION

After implementation:

1. Serve the current directory with a temporary local static HTTP server.
2. Open the page in a browser and inspect the complete sequence.
3. Check the browser console for errors and missing assets.
4. Verify at approximately 1440×900 and 390×844.
5. Confirm that:
   • the crack visibly grows before the burst;
   • the cracked state persists for at least one second;
   • only the selected side panel fails;
   • shards originate from that panel and move outward;
   • water exits through the breach while the tank level drops;
   • Replay and Reset work repeatedly;
   • all referenced assets resolve from the current directory;
   • no uncaught console errors occur.
6. Fix any failed check, with at most two attempts per distinct error.
7. Stop the temporary server when verification is complete.

PERMITTED ACTIONS

• Create or edit ./index.html.
• Create optional ./aquarium-*.glb files through Blender MCP.
• Run non-destructive inspection and verification commands.
• Start and stop a temporary local static server.

FORBIDDEN ACTIONS

• Do not modify files outside the current working directory.
• Do not install dependencies, deploy, push to Git, or contact external services.
• Do not delete or overwrite unrelated files.
• Stop and ask before any destructive action, work outside the stated scope, or architectural change that would violate the single-page deliverable.

DONE WHEN

The simulation runs from index.html under a static server, visibly progresses through all five states, produces a coherent side-panel fracture and outward burst with water loss, resets reliably, and passes every verification item without console errors.

After each major stage, report ✅ followed by one concise completion note. Finish with:

• files created or changed;
• whether Blender MCP was used;
• verification commands and results;
• any remaining limitation in one sentence.

Note: Both versions did not use Blender during generation.

Not bad, interestingly the Q2 made the glass go away. Which Q2 version btw?

@danielhanchen Kimi-K3-GGUF/UD-Q2_K_XL; https://huggingface.co/unsloth/Kimi-K3-GGUF/tree/main/UD-Q2_K_XL

Gemini, brilliant as ever.

Why gemini 3.1 pro gives me this sxxt

Qwen3.6 27b in codex:

Inkling-Small-276B-12B, UD-Q2_K_XL:

@danielhanchen

Here is a 2D generation comparison:

@vstiff Qwen3.6-27B, BF16:

@vstiff Qwen3.6-27B, BF16:

Still the king 👑

Great demonstration of how powerful 1 bit can be! BUT...

Can you (AND EVERYONE ELSE) do multiple runs?

All you AI "reviewers" are doing the equivalent of saying "draw a cat" into image generators and then comparing results. I understand that there's time and cost involved, but without multiple generations, comparisons are not indicative of true quality.

@HenryT2 :

The 2 bit version is holding up great as my new daily driver, already put it to use. I wish I could use vLLM instead of lama.cpp, it only outputs at 17 t/s.

Fable 5 vs GPT 5 6 Luna vs GPT 5 6 Sol vs Kimi K3 vs Opus 5 rendering the same cyberpunk street in Blender. Side by side for 60 seconds.
Each model made different creative choices - lighting, composition, atmosphere.
Which AI render wins?
https://youtube.com/shorts/Vv59xTwpdMo?feature=share

That is neat!

I wish Unsloth could partner with Artificial Analysis or Datacurve to run some benchmarks across various quants. Not just K3, but other models too. I find the typical KLD measurements to be an unsatisfying alternative, and it would be some great research to see how much quantization degrades models across a wide range of benchmarks.

We have demonstrated 2 bits in DSV4 previously in our customer sft/RL models. If you master the key points extremly quantization, you will agree that extreme low bits work if it is quantized in the right way.

https://x.com/yiakwy2023/status/2065656112465273059?s=20

Congrats to @danielhanchen and Unloth team that they have mastered these techniques and shaped the model in the first place.

The TQ 2 quant didn't work for me unfortunately. It kept telling the the ggml type was wrong (type 64 and should be 0-43). I tried messing around merging various PRs from the Unsloth llama.cpp fork but no luck, too many conflicts. Not sure if I'm an idiot or that quant isn't properly supported yet.

The TQ 2 quant didn't work for me unfortunately. It kept telling the the ggml type was wrong (type 64 and should be 0-43). I tried messing around merging various PRs from the Unsloth llama.cpp fork but no luck, too many conflicts. Not sure if I'm an idiot or that quant isn't properly supported yet.

I've got the same false for Q1 quant for my 512gb workstation:

'''
0.07.097.661 I srv load_model: loading model 'F:.lmstudio\models\unsloth\Kimi-K3-GGUF\Kimi-K3-UD-Q1_0-00001-of-00011.gguf'
0.07.189.050 E gguf_init_from_reader: tensor 'blk.1.ffn_down_exps.weight' has invalid ggml type 66. should be in [0, 43)
0.07.189.064 E gguf_init_from_reader: failed to read tensor info
0.07.194.637 E llama_model_load: error loading model: llama_model_loader: failed to load GGUF split from F:.lmstudio\models\unsloth\Kimi-K3-GGUF\Kimi-K3-UD-Q1_0-00002-of-00011.gguf
0.07.194.648 E llama_model_load_from_file_impl: failed to load model
0.07.194.721 E common_fit_params: encountered an error while trying to fit params to free device memory: failed to load model
0.07.249.685 E gguf_init_from_reader: tensor 'blk.1.ffn_down_exps.weight' has invalid ggml type 66. should be in [0, 43)
0.07.249.696 E gguf_init_from_reader: failed to read tensor info
0.07.254.996 E llama_model_load: error loading model: llama_model_loader: failed to load GGUF split from F:.lmstudio\models\unsloth\Kimi-K3-GGUF\Kimi-K3-UD-Q1_0-00002-of-00011.gguf
0.07.255.008 E llama_model_load_from_file_impl: failed to load model
0.07.255.028 E cmn common_init_: failed to load model 'F:.lmstudio\models\unsloth\Kimi-K3-GGUF\Kimi-K3-UD-Q1_0-00001-of-00011.gguf'
0.07.255.032 E srv load_model: failed to load model, 'F:.lmstudio\models\unsloth\Kimi-K3-GGUF\Kimi-K3-UD-Q1_0-00001-of-00011.gguf'
0.07.255.044 I srv operator(): operator(): cleaning up before exit...
0.07.265.479 E srv llama_server: exiting due to model loading error
'''

It happened with unsloth's PR for llama.cpp