mind doing phi 4 mini?
https://huggingface.co/unsloth/Phi-4-mini-instruct
has some fixes
AFAIK, phi4/mini is extremely censored, abliteration doesn't help a lot.
be our hero and make an uncensored version?
Uncensoring models is not without cost. Fine-tuning will damage model's capability somehow. For example, r1-1776 shows a significant IQ loss compared to r1 original model, though perplexity-ai pretends it isn't. Given that phi-4 series models are so censored, I'm afraid that uncensoring it will lead to completely destroying it.
But since it's just about 3B param, I will put it on my schedule and give it a shot ;)
hell yah!
- Good news: I uncensored it.
- Bad news: I fucked it up.
Small models like phi-4-mini (~3B param size) are delicate vase, any inappropriate modification will break it into pieces. Although it's only 3B param size, it consumes more VRAM than Qwen 7B in LoRA finetuning. I have tried several times, unable to find a good set of hyperparameters that won't fuck the model up.
interesting. well thank you anyway, i really appreciate the effort!