"max_length": 4096, "max_position_embeddings": 4096,
#1
by
pseudotensor
- opened
Why do these have no 'max_length' and have max_position_embeddings=2048 while meta-llama ones have "max_length": 4096,
"max_position_embeddings": 4096
Can these not be properly used for 4k context?
They can be used for 4K context. The difference in the config.json is because the original Meta config.json files didn't have max_length: 4096 and I've not yet gone back to fix my config.json files. But it shouldn't affect inference in most clients, which have their own parameter for specifying length.
Anyway, I will fix these config.json files today.