Very good
I don't have a test suite to run, but after some quick testing this did give me better results than goliath. Good job :)
Now it would be interesting to see what could be done with miqu using the same method.
I've tried making a LORA out of Miqu, but it came out completely dysfunctional. Miqu seems to be too different from base llama(rope, context length?) to be made into a LORA using LORD. If we get some better tool for extraction, then it would be possible. For now all we can do is extract LORAs from Miqu finetunes. I've tested only two of those and I was not very impressed by them. Undi95/Miqu-70B-Alpaca-DPO somehow came out more censored than the original(loss in BC) without any significant gain in "creativity", ShinojiResearch/Senku-70B-Full was drier(loss in S). If you know any good Miqu tunes feel free to suggest them.
Yeah, I tried some DARE merges with miqu that didn't really work, I'm guessing that's the same issue. Senku is the only one that worked reasonably well for me but I still liked base miqu better.