How to use BartEncoder and BartDecoder
Author: ryliu68
Pull request: no

### System Info

I'm a CVer, I want to use Bart as Auto-Encoder in CV task.

Here I have a question, how to use BartEncoder to encode **A** to **z**, and then how to decode **z** to "A"?
I need an example code; please help me, than you very much.


Maybe like this two?

one:
```
model = BartForConditionalGeneration.from_pretrained('facebook/bart-base')
tokenizer = BartTokenizer.from_pretrained('facebook/bart-base')

inputs = A

encoder = model.model.encoder
decoder = model.model.decoder

z= encoder(input_ids = inputs["input_ids"])

A= decoder(z)
```

it will meet following error: ValueError: You have to specify either decoder_input_ids or decoder_inputs_embeds.


two:
```
from transformers.models.bart.modeling_bart import BartEncoder
from transformers.models.bart.modeling_bart import BartDecoder

inputs = A

z= BartEncoder(input_ids = inputs["input_ids"])

A= BartDecoder(z)

So, what should I do? I didn't find a document for this; please help me with this, thank you again.

@ArthurZucker @gante @Narsil 


### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

as I provided code.

### Expected behavior

Write an example code or document

Comments:
@LRY0111 Even though BART is a Sequence to Sequence Transformer trained with a Denoising Autoencoder objective, it has been trained to reconstruct the original text. So I don't think you can use BART to represent "A" as "z" using the base model.
You can find more information in the model docs [here](https://huggingface.co/docs/transformers/model_doc/bart).

You can use the raw model for text infilling. However, the model is mostly meant to be fine-tuned on a supervised dataset. See the [model hub](https://huggingface.co/models?search=bart) to look for fine-tuned versions on a task that interests you.

@awinml  I appreciate your help. I mean that I want to get the latent representation z of a sentence A (by Encoder) and make some changes on z to formulate z' ; finally, reverse this process by Decoder to reconstruct the A'.

So I need to know how to do this A-->z-->A with Bart Encoder and Decoder; I want some example code here. Thank you very much.

cc @gante 

Hey @LRY0111 👋 

You were close to the correct usage, but there is a detail you missed in your solution :) The decoder must be used in an auto-regressive fashion, which we conveniently implemented in our `.generate()` method (see [this blog post](https://huggingface.co/blog/how-to-generate)). See the snippet below for an example.

```py
from transformers import BartForConditionalGeneration, BartTokenizer

model = BartForConditionalGeneration.from_pretrained('facebook/bart-base')
tokenizer = BartTokenizer.from_pretrained('facebook/bart-base')

inputs = tokenizer(["This is a test. Hello world"], return_tensors="pt")

encoder = model.model.encoder

# z.last_hidden_state has the encoded output. If you manipulate it, you may need to 
# rebuild the `BaseModelOutput` data class, which `.generate()` expects
z = encoder(input_ids=inputs["input_ids"])

A = model.generate(encoder_outputs=z, max_new_tokens=20)
print(tokenizer.decode(A[0], skip_special_tokens=True))
```

Well noted with thanks. I’ll try the solution you provided. Thank you again.

Best regards,

从 Windows 版邮件<https://go.microsoft.com/fwlink/?LinkId=550986>发送

发件人: Joao ***@***.***>
发送时间: 2023年5月3日 21:42
收件人: ***@***.***>
抄送: ***@***.***>; ***@***.***>
主题: Re: [huggingface/transformers] How to use BartEncoder and BartDecoder (Issue #23074)


Hey @LRY0111<https://github.com/LRY0111> 👋

You were close to the correct usage, but there is a detail you missed in your solution :) The decoder must be used in an auto-regressive fashion, which we conveniently implemented in our .generate() method (see this blog post<https://huggingface.co/blog/how-to-generate>). See the snippet below for an example.

from transformers import BartForConditionalGeneration, BartTokenizer



model = BartForConditionalGeneration.from_pretrained('facebook/bart-base')

tokenizer = BartTokenizer.from_pretrained('facebook/bart-base')



inputs = tokenizer(["This is a test. Hello world"], return_tensors="pt")



encoder = model.model.encoder

decoder = model.model.decoder



# z.last_hidden_state has the encoded output. If you manipulate it, you may need to

# rebuild the `BaseModelOutput` data class, which `.generate()` expects

z = encoder(input_ids=inputs["input_ids"])



A = model.generate(encoder_outputs=z, max_new_tokens=20)

print(tokenizer.decode(A[0], skip_special_tokens=True))

—
Reply to this email directly, view it on GitHub<https://github.com/huggingface/transformers/issues/23074#issuecomment-1533053566>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AFUZSAVRRMEA7V27JD4OVQTXEJODRANCNFSM6AAAAAAXQ3WM6M>.
You are receiving this because you were mentioned.Message ID: ***@***.***>



This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.