Blenderbot export issue in ONNX and C++
Author: Zapotecatl
Pull request: no
Labels: name: Migration

# 📚 Migration



Hi everyone, my issue is regarding blenderbot: facebook/blenderbot-400M-distill. I did the exportation with: 

`python -m transformers.onnx --model=facebook/blenderbot-400M-distill onnx/`

And I get:

```
Some weights of the model checkpoint at facebook/blenderbot-400M-distill were not used when initializing BlenderbotModel: ['final_logits_bias', 'lm_head.weight']
- This IS expected if you are initializing BlenderbotModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing BlenderbotModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).   
Using framework PyTorch: 1.10.0+cu102
Overriding 1 configuration item(s)
        - use_cache -> False      
...\venv\lib\site-packages\transformers\models\blenderbot\modeling_blenderbot.py:219: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if attn_weights.size() != (bsz * self.num_heads, tgt_len, src_len):
...\venv\lib\site-packages\transformers\models\blenderbot\modeling_blenderbot.py:225: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if attention_mask.size() != (bsz, 1, tgt_len, src_len):
...\venv\lib\site-packages\transformers\models\blenderbot\modeling_blenderbot.py:256: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if attn_output.size() != (bsz * self.num_heads, tgt_len, self.head_dim):
...\venv\lib\site-packages\transformers\models\blenderbot\modeling_blenderbot.py:845: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if input_shape[-1] > 1:
Validating ONNX model...
        -[✓] ONNX model output names match reference model ({'last_hidden_state'})
        - Validating ONNX Model output "last_hidden_state":
                -[✓] (2, 8, 1280) matches (2, 8, 1280)     
                -[✓] all values close (atol: 1e-05)        
All good, model saved at: onnx/model.onnx
```

**The warning: "TracerWarning: Converting a tensor to a Python boolean...", I think it is the classic problem of control flow**. I have tried to export blenderbot with torch.jit.script(model). However, I get:

```
return any(hasattr(m, "gradient_checkpointing") and m.gradient_checkpointing for m in self.modules())
                   ~ <--- HERE
```

**I understand that TORCHSCRIPT dosen't support the function: any. Therefore, there isn't possibility of export blenderbot as a script and overcomes the problems of control flow**.

Anyway, I wrote the next onnx programm in C++:

```
#include <iostream>
#include <onnxruntime_cxx_api.h>

using namespace std;

void max(std::vector<float>& input, int64_t size_input_ids, int64_t size_embedding, std::vector<int>& output) {
	double bigger = 0.0;
	int index = 0;
	int k = 0;

	output.clear();
	for (size_t i = 0; i < size_input_ids; i++) {//128
		bigger = 0.0;
		index = 0;
		for (size_t j = 0; j < size_embedding; j++) {//1280 or 8008
            if (input[k] > bigger) {
				bigger = input[k];
				index = j;
			}
			k++;
		}
		output.push_back(index);
	}
}

int main()
{

	Ort::Env env;
	Ort::RunOptions runOptions;
	Ort::Session session(nullptr);

	auto modelPath = L"D:\\Nexus\\NexusV_1_0_0\\x64\\Release\\Conversational\\Model\\model.onnx";

	// create session
	session = Ort::Session(env, modelPath, Ort::SessionOptions{ nullptr });
	
	constexpr int64_t size_input = 128;
	constexpr int64_t size_output = 128;
	constexpr int64_t output_01 = 1280;//8008
	constexpr int64_t numOutputElements = size_output * output_01;
	constexpr int64_t output_02 = 1280;
	constexpr int64_t numOutputElements_02 = size_output * output_02;

	// define shape
	const array<int64_t, 2> inputShape = { 1, size_input};
	const array<int64_t, 3> outputShape = { 1, size_output, output_01 };
	const array<int64_t, 3> outputShape_02 = { 1, size_output, output_02 };

	std::vector<int64_t> input_ids(size_input);
	std::vector<int64_t> attention_mask(size_input);
	std::vector<int64_t> decoder_input_ids(size_input);
	std::vector<int64_t> decoder_attention_mask(size_input);

	std::vector<float> last_hidden_state(numOutputElements);
	std::vector<float> last_output_02(numOutputElements_02);

	for (size_t  i = 0; i < input_ids.size(); i++) {
		input_ids[i] = 0;
		attention_mask[i] = 0;
		decoder_input_ids[i] = 0;
		decoder_attention_mask[i] = 0;
	}

	//IDs: I want to order a Pizza
	input_ids[0] = 281;
	input_ids[1] = 538;
	input_ids[2] = 287;
	input_ids[3] = 1831;
	input_ids[4] = 265;
	input_ids[5] = 440;
	input_ids[6] = 4425;
	input_ids[7] = 2;

	attention_mask[0] = 1;
	attention_mask[1] = 1;
	attention_mask[2] = 1;
	attention_mask[3] = 1;
	attention_mask[4] = 1;
	attention_mask[5] = 1;
	attention_mask[6] = 1;
	attention_mask[7] = 1;
		
	for (size_t  i = 0; i < last_hidden_state.size(); i++)
		last_hidden_state[i] = 0;

	for (size_t i = 0; i < last_output_02.size(); i++)
		last_output_02[i] = 0;

	// define Tensor
	auto memory_info = Ort::MemoryInfo::CreateCpu(OrtDeviceAllocator, OrtMemTypeCPU);
				
	std::vector<Ort::Value> inputTensors;
	inputTensors.push_back(Ort::Value::CreateTensor<int64_t>(memory_info, input_ids.data(), input_ids.size(), inputShape.data(), inputShape.size()));
	inputTensors.push_back(Ort::Value::CreateTensor<int64_t>(memory_info, attention_mask.data(), attention_mask.size(), inputShape.data(), inputShape.size()));
	inputTensors.push_back(Ort::Value::CreateTensor<int64_t>(memory_info, decoder_input_ids.data(), decoder_input_ids.size(), inputShape.data(), inputShape.size()));
	inputTensors.push_back(Ort::Value::CreateTensor<int64_t>(memory_info, decoder_attention_mask.data(), decoder_attention_mask.size(), inputShape.data(), inputShape.size()));

	std::vector<Ort::Value> outputTensors;
	outputTensors.push_back(Ort::Value::CreateTensor<float>(memory_info, last_hidden_state.data(), last_hidden_state.size(), outputShape.data(), outputShape.size()));
	outputTensors.push_back(Ort::Value::CreateTensor<float>(memory_info, last_output_02.data(), last_output_02.size(), outputShape_02.data(), outputShape_02.size()));
		
	// define names
	Ort::AllocatorWithDefaultOptions allocator;
	std::vector < const char*> inputNames;
	inputNames.push_back(session.GetInputName(0, allocator));
	inputNames.push_back(session.GetInputName(1, allocator));
	inputNames.push_back(session.GetInputName(2, allocator));
	inputNames.push_back(session.GetInputName(3, allocator));
    
	std::vector < const char*> outputNames;
	outputNames.push_back(session.GetOutputName(0, allocator));
	outputNames.push_back(session.GetOutputName(1, allocator));
		
	// run inference
	try {
		session.Run(Ort::RunOptions{ nullptr }, inputNames.data(), inputTensors.data(), 4, outputNames.data(), outputTensors.data(), 2);
	}
	catch (Ort::Exception& e) {
		cout << e.what() << endl;
		return 1;
	}
	
	std::vector<int> output_ids;
	max(last_hidden_state, size_input, output_01, output_ids);

	for (size_t i = 0; i < output_ids.size(); i++) {
		cout << output_ids[i] << ", ";
	}
	cout << endl;
  
	return 0;
}
```

I get the output:

`652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652, 652,`

**I think the reason why the ids are the same is regarding the control flow problem** (or maybe there is something wrong in my program).

I understand that blenderbot is supported: [(https://huggingface.co/docs/transformers/main/en/serialization#torchscript)]

**Please, could you give me a suggestion to export correctly blenderbot or get the correct output ids?**

Note: I also use this tutorial ([https://www.kaggle.com/code/danieliusv/blenderbot1-0-test-and-export/notebook]), but I get the same results.

**Environment**

Windows 10
PyCharm 2021.3 (Community Edition)
Visual Studio 2019


Comments:
cc @lewtun, maybe also @michaelbenayoun and @mfuntowicz 

> Hi everyone, my issue is regarding blenderbot: facebook/blenderbot-400M-distill. I did the exportation with:
> 
> `python -m transformers.onnx --model=facebook/blenderbot-400M-distill onnx/`

Hi @Zapotecatl , could you try to add the right feature to your export command? I see that `facebook/blenderbot-400M-distill` is for `ConditionalGeneration` in the [model config file](https://huggingface.co/facebook/blenderbot-400M-distill/blob/main/config.json#L8) but you use the default feature while exporting.

Here are all available features for Blenderbot ONNX conversion: `default`, `default-with-past`, `causal-lm`, `causal-lm-with-past`, `seq2seq-lm` and `seq2seq-lm-with-past`.

It seems that the output of `BlenderbotForConditionalGeneration` is `Seq2SeqLMOutput` as you can see on [the source code](https://github.com/huggingface/transformers/blob/31ec2cb2badfbdd4c1ac9c6c9b8a74e974984206/src/transformers/models/blenderbot/modeling_blenderbot.py#L1342)

Something like this command could help:
```bash
python -m transformers.onnx --model=facebook/blenderbot-400M-distill --feature=seq2seq-lm onnx/
```


Hi @ChainYo, I appreciate your help. I executed the command. However, there is a issue regarding the "Outputs values". The ouput is this: 

```
python -m transformers.onnx --model=facebook/blenderbot-400M-distill --feature=seq2seq-lm onnx/
Using framework PyTorch: 1.10.0+cu102
Overriding 1 configuration item(s)
        - use_cache -> False
C:\Users\Jorge\PycharmProjects\inteBERT\venv\lib\site-packages\transformers\models\blenderbot\modeling_blenderbot.py:219: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if attn_weights.size() != (bsz * self.num_heads, tgt_len, src_len):
C:\Users\Jorge\PycharmProjects\inteBERT\venv\lib\site-packages\transformers\models\blenderbot\modeling_blenderbot.py:225: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if attention_mask.size() != (bsz, 1, tgt_len, src_len):
C:\Users\Jorge\PycharmProjects\inteBERT\venv\lib\site-packages\transformers\models\blenderbot\modeling_blenderbot.py:256: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if attn_output.size() != (bsz * self.num_heads, tgt_len, self.head_dim):
C:\Users\Jorge\PycharmProjects\inteBERT\venv\lib\site-packages\transformers\models\blenderbot\modeling_blenderbot.py:845: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if input_shape[-1] > 1:
Validating ONNX model...
        -[✓] ONNX model output names match reference model ({'logits'})
        - Validating ONNX Model output "logits":
                -[✓] (2, 8, 8008) matches (2, 8, 8008)
                -[x] values not close enough (atol: 1e-05)
Traceback (most recent call last):
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\Jorge\PycharmProjects\inteBERT\venv\lib\site-packages\transformers\onnx\__main__.py", line 99, in <module>
    main()
  File "C:\Users\Jorge\PycharmProjects\inteBERT\venv\lib\site-packages\transformers\onnx\__main__.py", line 92, in main
    validate_model_outputs(onnx_config, preprocessor, model, args.output, onnx_outputs, args.atol)
  File "C:\Users\Jorge\PycharmProjects\inteBERT\venv\lib\site-packages\transformers\onnx\convert.py", line 416, in validate_model_outputs
    "Outputs values doesn't match between reference model and ONNX exported model: "
ValueError: Outputs values doesn't match between reference model and ONNX exported model: Got max absolute difference of: 2.276897430419922e-05
```
I get the same results with: seq2seq-lm-with-past, default-with-past, causal-lm and causal-lm-with-past. Of course, I get the original result with the option default.







By default `atol` is set to 1e-05, you should try to set it to 1e-04 in your case.

Here is the new command:

```bash
$ python -m transformers.onnx --model=facebook/blenderbot-400M-distill --feature=seq2seq-lm --atol=1e-04 onnx/
```


Yes, thanks, now the model is exported correctly (I changed the value of my variable constexpr int64_t output_01 = 8008). 
However, the issue about the repetition continues. Let me show you:

```
python -m transformers.onnx --model=facebook/blenderbot-400M-distill --feature=seq2seq-lm --atol=1e-04 onnx/
Using framework PyTorch: 1.10.0+cu102
Overriding 1 configuration item(s)
        - use_cache -> False
C:\Users\Jorge\PycharmProjects\inteBERT\venv\lib\site-packages\transformers\models\blenderbot\modeling_blenderbot.py:219: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if attn_weights.size() != (bsz * self.num_heads, tgt_len, src_len):
C:\Users\Jorge\PycharmProjects\inteBERT\venv\lib\site-packages\transformers\models\blenderbot\modeling_blenderbot.py:225: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if attention_mask.size() != (bsz, 1, tgt_len, src_len):
C:\Users\Jorge\PycharmProjects\inteBERT\venv\lib\site-packages\transformers\models\blenderbot\modeling_blenderbot.py:256: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if attn_output.size() != (bsz * self.num_heads, tgt_len, self.head_dim):
C:\Users\Jorge\PycharmProjects\inteBERT\venv\lib\site-packages\transformers\models\blenderbot\modeling_blenderbot.py:845: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if input_shape[-1] > 1:
Validating ONNX model...
        -[✓] ONNX model output names match reference model ({'logits'})
        - Validating ONNX Model output "logits":      
                -[✓] (2, 8, 8008) matches (2, 8, 8008)
                -[✓] all values close (atol: 0.0001)
All good, model saved at: onnx/model.onnx
```
Now, I get in my C++ program the next output:

`2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 21, 21, 21, 2, 2, 2, 2, 2, 2, 2, 2, 21, 21, 21, 21, 2, 2, 2, 2, 2, 21, 21, 21, 2, 2, 2, 2, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 2, 2, 2, 21, 21, 21, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2`

The output that I expect is something like that:

1,  281,  913, 4955,    8,  714,  287,  413, 1252,  361,  304,  398, 38,  281,  398, 6180, 3870,   19,  383, 1230,   19,  298, 4014,  340, 1043,   21,    2. 
(Decode: I love pizza! What toppings do you like? I like vegetables, meats, and condiments.)















I never used onnx model with C++, but I think you should check the way you load the model, how you tokenize inputs and what is your InferenceSession code.

Thanks for your suggestions @ChainYo.

I checked my C++ code with care. I didn't find errors.

I did the equivalent program in phyton. Please, let me show you:

```
import torch.onnx
import onnx
import onnxruntime as ort
from transformers import BlenderbotTokenizer

model = onnx.load("onnx/model.onnx")
onnx.checker.check_model(model)
#print(onnx.helper.printable_graph(model.graph))

print('Blender check')

tokenizer = BlenderbotTokenizer.from_pretrained("facebook/blenderbot-400M-distill")
utterance = "I want to order a Pizza"
tokens_ids = tokenizer(utterance, return_tensors="pt")

input_ids = torch.cat((tokens_ids['input_ids'], torch.zeros(1, 120, dtype=torch.int64)), dim=1)
attention_mask = torch.cat((tokens_ids['attention_mask'], torch.zeros(1, 120, dtype=torch.int64)), dim=1)
decoder_input_ids = torch.zeros(1, 128, dtype=torch.int64)
decoder_attention_mask = torch.zeros(1, 128, dtype=torch.int64)

#print(input_ids)
#print(attention_mask)

ort_session = ort.InferenceSession('onnx/model.onnx')
args = {'input_ids': input_ids.cpu().detach().numpy(), 'attention_mask': attention_mask.cpu().detach().numpy(), 'decoder_input_ids': decoder_input_ids.cpu().detach().numpy(), 'decoder_attention_mask': decoder_attention_mask.cpu().detach().numpy()}
outputs = ort_session.run(None, args,)

#outputs: logits and 711; outputs[0]: logits; outputs[0][0]: output 128 X 8008
res = outputs[0][0].argmax(axis=1)

print(res)

# Decoding the model output
deco = tokenizer.decode(res)
print(deco)

print("End BlenderBot")
```
The output that I get is the same that in my C++ program:

```
Blender check
[ 2  2  2  2  2  2  2  2  2  2  2  2  2  2  2  2  2  2  2  2  2  2  2  2
  2  2  2  2  2  2  2  2  2  2  2  2  2  2  2  2  2  2 21 21 21  2  2  2
  2  2  2  2  2 21 21 21 21  2  2  2  2  2 21 21 21  2  2  2  2 21 21 21
 21 21 21 21 21 21 21 21 21 21 21 21 21 21 21  2  2  2 21 21 21  2  2  2
  2  2  2  2  2  2  2  2  2  2  2  2  2  2  2  2  2  2  2  2  2  2  2  2
  2  2  2  2  2  2  2  2]










































...







....




...



..................


...



































End BlenderBot
```
I really want to overcome this, any suggestion is welcome!


Do you get the same result with another Blenderbot model if you have one to test?

Your python code looks good, I must try it to see how it goes

Hi, I tried to export blenderbot-3B and blenderbot-1B-distill whit the command:

`python -m transformers.onnx --model=facebook/blenderbot-1B-distill --feature=seq2seq-lm --atol=1e-04 onnx/`

However, I can't export that models. I get the next:

```
feature=seq2seq-lm --atol=1e-04 onnx/
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Using framework PyTorch: 1.10.0+cu102
Overriding 1 configuration item(s)
        - use_cache -> False
C:\Users\Jorge\PycharmProjects\inteBERT\venv\lib\site-packages\transformers\models\blenderbot\modeling_blenderbot.py:219: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if attn_weights.size() != (bsz * self.num_heads, tgt_len, src_len):
C:\Users\Jorge\PycharmProjects\inteBERT\venv\lib\site-packages\transformers\models\blenderbot\modeling_blenderbot.py:225: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if attention_mask.size() != (bsz, 1, tgt_len, src_len):
C:\Users\Jorge\PycharmProjects\inteBERT\venv\lib\site-packages\transformers\models\blenderbot\modeling_blenderbot.py:256: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if attn_output.size() != (bsz * self.num_heads, tgt_len, self.head_dim):
C:\Users\Jorge\PycharmProjects\inteBERT\venv\lib\site-packages\transformers\models\blenderbot\modeling_blenderbot.py:845: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if input_shape[-1] > 1:
Validating ONNX model...
Traceback (most recent call last):
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\Jorge\PycharmProjects\inteBERT\venv\lib\site-packages\transformers\onnx\__main__.py", line 99, in <module>
    main()
  File "C:\Users\Jorge\PycharmProjects\inteBERT\venv\lib\site-packages\transformers\onnx\__main__.py", line 92, in main
    validate_model_outputs(onnx_config, preprocessor, model, args.output, onnx_outputs, args.atol)
  File "C:\Users\Jorge\PycharmProjects\inteBERT\venv\lib\site-packages\transformers\onnx\convert.py", line 350, in validate_model_outputs
    session = InferenceSession(onnx_model.as_posix(), options, providers=["CPUExecutionProvider"])
  File "C:\Users\Jorge\PycharmProjects\inteBERT\venv\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 335, in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
  File "C:\Users\Jorge\PycharmProjects\inteBERT\venv\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 381, in _create_inference_session
    sess.initialize_session(providers, provider_options, disabled_optimizers)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Deserialize tensor model.decoder.layers.18.self_attn.v_proj.bias failed.open file model.decoder.layers.18.self_attn.v_proj.bias fail, errcode = 2 - El sistema no puede encontrar el archivo especificado.
```









Excuse me @lewtun , I know you did the PR about the Blenderbot exportation, maybe you have some advice? I really appreciate any suggestion.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hi, I had also the same issue with blenderbot onnx, the problem was actually in control flow. Exactly in this place `if input_shape[-1] > 1`. You have to export it manually by passing seq_len = 2, I used [1, 1](two BOS tokens). At the beginning when you have only one input token for the decoder it expands the attention mask one way, further in a second way. I hope it helps