cool idea! did it work?
#2
by
ehartford
- opened
cool idea! did it work?
yes it worked, but not very well.
The model did combine the features of both source models, but also became dumber in the process, some amount of the finetuning seems to have been lost.
So mixed-vocab sized merges are sub optimal at this time.