Kype720184/DorsetHeatwaveLLM2

A little about me

I am 13, am in a normal non-grammar school and I have SEND (Special Educational Needs and Disabilities).

So I decided that as it is a heatwave (albeit less intense), so it is DorsetHeatwaveLLM time!

My idea was simple: Make the sequel.

And I made this model on Sunday 5th July 2026.

And to the question "Also, have any Grammar students my time period beat me to making a similar sized model first?", nobody told me so, so I will assume the answer is no. Or at least no in Dorset from what I know.

The not so grand return of DorsetHeatwaveLLM

And as the UK Health Security Agency pushed a yellow heat warning (yes, I know, not as bad as the original DorsetHeatwaveLLM had it), as you can see from the image below... ...this gives us the (not so) grand return of DorsetHeatwaveLLM with DorsetHeatwaveLLM2!

NOTE: THIS IS A ~2B MODEL! SMALL MODELS LIKE THIS CAN HALLUCINATE, ROLEPLAY OR GET THINGS WRONG. DO NOT USE THIS FOR MISSION CRITICAL APPLICATIONS!

Specifications

This is the result.

Hyperparams

• Batch size: 2
• Learning rate: 0.0002
• Optimizer: AdamW 8-bit
• Max steps: 450
• Context length: 2048
• Warmup steps: 5

LoRA

• Rank: 32
• Alpha: 32
• Dropout: 0
• Variant: lora

Other info

• Seed: 1326

I decided not to do tests.

So if you want them ...

...make your own tests!

I can be biased to show my model is better than it actually is so try it for yourself.

GGUF Versions

Find info on those here.

Some nice questions to ask, maybe nobody cares:

Note: "I" and "Me" in this secrtion refer to you, and "You" refers to me. This note only applies for these questions in the Q, not in the answers, otherwise known in a bullet point starting with A below the question starting with Q next to a number.

Q1: Is this software Free or Nonfree?

• A1: Same as DorsetHeatwaveLLM. It is Nonfree because the Free Software Foundation states that "All software included in a free ML application has to offer every user the four freedoms that define free software. This applies to both the software that processes training data, and the software that interprets model parameters as context for prompts to produce human-usable output. This is necessary but not sufficient.", and as this model is based on Qwen 3 1.7B, which has only Open Weights under Apache 2.0, this model is legitimately Nonfree.

Q2: I found somehing concerning with how you finetuned the model. Something went wrong.

• A2: Tell me then in the Community tab.

Q3: Why is Unsloth not in the national curicilum?

• A3: Probably no one, or barely anyone, at the DfE even knows local LLM inference (running models) let alone finetuning (training a model on a dataset)! And I guess we are lucky it is not taught.

Q3.1: Why is it lucky Unsloth is not taught?

• A3.1: Because probably the best computers with GPUs secondary schools have are ones for basic CAD and maybe 2 or 4GB RAM, and at best, maybe 6GB. Also, students will not understand and be very bad at this and say things such as "My model broke!", "This is ragebait!", "This is too slow", "It is too hot in here" (the heat from all thsoe GPUs for hours at near 100% utilisation and screaming fans will make the room very hot) amongst other complaints.

Q4: What was the highest temperature the GPU hotspot got to?

• A4: The hottest it got to with the original DorsetHeatwaveLLM is 94°C, wheras because DorsetHeatwaveLLM2 was on a cooler heatwave (litterally) and I put the room fan next to my PC, DorsetHeatwaveLLM2 had the peak temperature of the hotspot at 86°C.

Q5: What GPU application do you use? The NVIDIA control panel is dead and MSI afterburner is incompatible with Ubuntu!

• A5: Guess what! I use a better tool I use called LACT. And in my opinion, I do not care of MSI afterburner.

Q6: What is a Grammar school?

• A6: In the UK education system, a grammar school is a state-funded secondary school that chooses its pupils based on their "academic ability" shown in the 11+, which now has an entire industry around it. Find the original revision and latest revision from these hyperlinks, which is the legacy bloat from the 1940s that Grammar schools partially rely on.

Q6.1: So what does thse laws mean?

• A6.1: The exact phrase that states "abilities and aptitudes" became the statutory loophole and structural pillar for the 11-Plus exam. So to figure out a child’s "aptitude" and push them into either an "elite" grammar school or a practical secondary modern, LEAs turned to standardized intelligence testing. This single phrasing choice essentially birthed the modern multi-million-pound 11+ tutoring and preparation industry. Without that, none of this 11+ junk would exsist.

Q6.2: Isn't the 1944 Act itself is dead?

• A6.2: Turns out, yes, and that Grammar schools do not rely on a living 1940s statute to operate today. Instead, their survival relies on transitional savings and exemptions cooked into the modern 1998 legislation. When the Labour government banned the creation of new academic selection schools in 1998, they carved out a specific exemption for the remaining 163 grammar schools (scattered across areas like Kent, Buckinghamshire, and Dorset).

Q6.3: Some rant on how Grammar schools are better and should not be abolished

• A6.3: I do not care.

Q7: Why not use LoftQ?

• A7: If I were finetuning something like Gemma 4 E2B or some model with a total of up to 7B or 8B parameters on my RTX 3060 Ti, then yes, LoftQ is the best. But LoftQ despite using less VRAM, still is slow, and you cannot change that. And as I am finetuned a 1.7B model, not a 4B or over one, LoftQ would have been worse in this case.

A joke not related to AI

Joke: What is an emulator for Windows apps and a drinknot to be consumed by under 18s? Answer: Wine!

Credits (Kype720184 style)

Credits I would like to give:

• JuDDGES: For the en-appealcourt-coded-instruct_v02 dataset under an MIT License (No direct involvement or endorsement)
• Alibaba: For making the Qwen 3 1.7B Base model (No direct involvement or endorsement)
• HuggingFace: For making a brilliant platform for hosting LLMs (No direct involvement or endorsement)
• Unsloth: For repackaging the Qwen 3 1.7B Base model (No direct involvement or endorsement)
• Unsloth: For making the amazing finetuning tool that I used (No direct involvement or endorsement)
• Canonical: For making Ubuntu and Ubuntu Pro, which I use on my main PC as it is my favourite GNU/Linux distro (I know I am nt a GNU purist but it is GNU/Linux, but you can find out about the naming here*), except for Snaps (No direct involvement or endorsement)
• NVIDIA: For making the RTX 3060 Ti I have (No direct involvement or endorsement)
• MSI: For being the retailer of the 3060 Ti I have, and they have very nice TORX 3 fans that I have to admit, are exteremely good (No direct involvement or endorsement)
• LACT: For being the absolute best GPU control application, I forgot to credit you in the original (No direct involvement or endorsement)
• Me: For putting all this together into 1 huge model, handling the (now less) heat and the dedication (Direct involvement)

---

*Or you can call it GNU/X11/Apache/Linux/TeX/Perl/Python/FreeCiv if you really want to