Remove hmp from gaudi_config.json and README
@jwieczorekhabana
Regression tests passed for this model :)
One last thing: in the model card, you added "This model is supported only in mixed precision training with bf16 type.". Do you mean here that mixed precision is supported only with bf16 (and not with fp16)? It sounds a bit ambiguous to me that it could run only in mixed precision and not in full fp32 precision. Maybe you can rephrase it? Or even remove it, not sure it is that important. WDYT?
@regisss
On our side we verify this model only with bf16 mixed precision training. We would like to inform the users that bf16 mixed precision training is assured and in case of any issues with training it should be treated as undesirable. Although training in full fp32 precision should be possible we do not guarantee that it won't change later on. Impact on models of changes in our software stack is monitored and we don't want to introduce new features that might affect them negatively.
We have capacity to ensure that bf16 mixed precision training is of good quality. In case of full precision it is possible that some alterations might be necessary improve quality. In case of any issues feedback about them is most welcome but performance and accuracy of bf16 mixed precision training are our highest priority
@jwieczorekhabana
Sounds good!
In that case, would it be possible to change "This model is supported only in mixed precision training with bf16 type." to something like "It is strongly recommended to train this model doing bf16 mixed-precision training for optimal performance and accuracy."?
@regisss
I've updated all relevant PRs with your suggestion.
Thanks!