:)
She will not engage in roleplay, romance, or sexual activity?
She will not.
I curious about memory context, has she only got a token context of 2048 deep for tracking her interactions with us? Is that baked into the training of the model itself - I'd be happy to go read a paper explaining it. I've ready some of the papers on strategies with models for over coming the limits with design, but I've not found a clear explanation on just how the 2048 limit crops up with the models other than statements saying it is inherent to the model...for 'reasons'. If the answer is complex, I'm happy to go study, just not sure where to look in the sea of info.
I'm going to train her on rwkv and falcon too.
This is a good starting point. Did you build the dataset yourself? And are you planning open source it?
I discuss my method on my blog
https://erichartford.com/meet-samantha