Merged... :P
Hi, thank you for this great model again!
Just to let you know... I couldn't resist and finally made merge of Midnight Rose with CodeLlama Python --> CodeRosa-70B-AB1 It is quite interesting mix!
Wow! That's really cool. Does it do well at coding?
It took me some time till I get it working the way it is. I may be biased after all testing different, half broken iterations durring latest days and comparing them with other models...
It has different aproach to coding tasks and like to give concrete answers and you have to prompt it futher for details. So you may get impresion it is worse. It has attitude to generalize more but I think it can explain things better, give more details.
I would say you can feel it's dual nature of writer and coder. It walks after you like a dog and can produce varied results between swipes and is able to write longish outputs of even 1.3K of tokens. I feel it could be improved so I will try different recipes to see if I can make it more stable, so this dual personality will became more uniform.
I am eager to try a mix with CodeLlama Instruct to see how it goes or see it it is possible to mix Midnight Rose with CodeLlama base version and get 16K context. That would be a thing! 8K is mostly enought for most tasks but having 16K would address like 99,5% of my use cases for now. I think uncensored model in mix with some codding model is the way to go for me. I think Llama models have best written language and all this censoring has as consequence of been very emotional model that can write very sensual texts, adding to that, capabilities in something usefull like programing knowladge and joining this base with model that will uncensor it, in my feelings, is the way to get great everyday companion model and I deffinitly would like to go futher in this direction.
So many ideas to try, I am a bit woried about my m.2 drives. :O