Fix for use in LM Studio [Turn Flash Attention On]
#5
by
YorkieOH10
- opened
When you use this model in LM Studio - you need to use the included ChatML preset.
Then in Settings (Right hand side chat screen) Go to -> Model Initialization -> Flash Attention -> Turn it on