relatively short context length and I believe out of 28 tokens and then right at the end of training double that two times two up to 512 tokens for prediction and so they just train at 512 the whole way through it's the same model capacity it has the same runtime per sequence length but they just have you know they spend the pre-training compute to buy that and when you're thinking about deploying the system you know one of the important criteria to realize is especially when you're talking about a system that might get