Please add the following model to the neuron cache
Inference cache is only supported for causal lm models. cc @Jingya
Hi @k10 , marian type models are not yet supported by optimum-neuron. To add its cache, we will need to add the export and inference support for it first.
marian
optimum-neuron
I opened a ticket here, feel free to pick the task up if you want to contribute!
Β· Sign up or log in to comment