aws-neuron

/

optimum-neuron-cache

like 13

Follow

AWS Inferentia and Trainium 72

Hi @gaussfer , as discussed with you on the optimum-neuron repo. We will need to add tensor parallel support first. Unfortunately, the tp support for t5 is not working for some tracing issue. AWS's side is working on solving it, optimum team will prioritize the t5 tp support as well once it's fixed.

Related: https://github.com/huggingface/optimum-neuron/issues/479

Thanks @Jingya .