How do I export it to torchscript?
#2
by
elavneet
- opened
I've been trying to export the model to a torchscript - torch.jit.trace(mode, (args)). But, I'm unable to figure out the args.
I tried following code:
model = AutoModelForSeq2SeqLM.from_pretrained("ramsrigouthamg/t5-large-paraphraser-diverse-high-quality", torchscript = True)
tokenizer = AutoTokenizer.from_pretrained("ramsrigouthamg/t5-large-paraphraser-diverse-high-quality")
input_ids = tokenizer(article, return_tensors='pt').input_ids
attention_mask = input_ids.ne(model.config.pad_token_id).long()
decoder_input_ids = tokenizer(article, return_tensors='pt').input_ids
traced_model = torch.jit.trace(model, (input_ids, attention_mask, decoder_input_ids))
But it doesn't work for the arguments. Here's another git issue repo related to my post: https://github.com/huggingface/transformers/issues/5647
I appreciate any reply to this thread.
elavneet
changed discussion status to
closed
What was the solution to this?
Any solution?