I clicked on the Amazon Sagemaker deploy and followed the steps given. However, it throws an error. Should I change the "instance type" parameter?
The inference widget doesn't run, Sagemaker also doesn't work. So I do not know how to use this model. Help is massively appreciated.
I realised that there is no way to actually run these models on system without having access to A100 GPU's. In Amazon to request that instance, it costs approx. USD 32. Therefore, unless there are folks from big companies or academia, this model cannot be used.
It should be possible to run the model with Large Model Loading on Amazon SageMaker. But there is not yet a container with the supported
transformers version available meaning you would need to create a custom
requirements.txt to deploy to sagemaker. example
For instance type i am not sure which one is enough I would either try