TAPAS for Question Generation
Author: saichandrapandraju
Pull request: no

Hi,

Is there a way to generate questions for a table with TAPAS ? or is it only for Question Answering ?

Comments:
Yes you can. TAPAS is an encoder-model, and can be used in an encoder-decoder set-up, like so:

```
from transformers import EncoderDecoderModel

model = EncoderDecoderModel.from_encoder_decoder_pretrained("google/tapas-base", "bert-base-cased")
```
You can specify any decoder you want, here I'm using BERT as a decoder, but you can also use GPT-2, etc (any model that supports the `is_decoder` logic).

For more information, see the [docs](https://huggingface.co/transformers/model_doc/encoderdecoder.html) of `EncoderDecoderModel`. 

Thanks @NielsRogge ,

I went through the concept of `EncoderDecoderModel` and I have a doubt in implementing it for TAPAS - 

Unlike normal BERT models, TAPAS tokenizer takes `table`, `queries` and `answers` for fine-tuning. So if I want to generate questions, should I skip questions for TAPAS (currently using `google/tapas-large`) encoder and give them to decoder (currently using `GPT-2-medium`) instead?

Yes, if you want to generate questions given a table, then you should only encode the table (you can set `queries=None` when providing a table to `TapasTokenizer`). 

Thanks @NielsRogge ,

I'll implement and let you know

Great, I'm curious to see the results. Another use case could be to generate answers given a question + table with an EncoderDecoder set-up. 

Hi @NielsRogge ,

I tried with the above approach by passing `table` to encoder and `queries` to decoder.  But while encoding, it's giving warning as - **TAPAS is a question answering model but you have not passed a query. Please be aware that the model will probably not behave correctly**  which is because of [this](https://github.com/huggingface/transformers/blob/master/src/transformers/models/tapas/tokenization_tapas.py#L1014). I thought passing 'queries' to TAPAS is mandatory.

Anyhow I trained the model but it's not performing as expected. While inferencing, it is giving same question (not fully formed) for any input that I pass. Below is the sample snippet
![enc_dec](https://user-images.githubusercontent.com/41769919/111947822-27469800-8b04-11eb-8802-e2a1f7f53231.PNG)



Yeah that warning is shown because TAPAS has been pre-trained on text-table pairs. You can ignore that warning, because we can still encode just the tables.

What kind of generation method are you using? Greedy decoding, beam search? (See [this post](https://huggingface.co/blog/how-to-generate) for the different arguments you can pass to `.generate()`). 

@NielsRogge ,

I'm not specifying any generation method so it should be Greedy itself.

Can you provide a notebook?

Hi @NielsRogge ,

[Here](https://colab.research.google.com/drive/1d8m_hmipL-1ZzU15LfA2XHmypKipvnJR?usp=sharing) is the colab link for the replica of my work. As I cannot share or upload any files from my Office' VPN, I created this notebook which is same as the one I'm working with in our VM's. Only change is I used `google/tapas-base` and `gpt2` in colab whereas I'm using `google/tapas-large` and `gpt2-medium` for my official work.

Hi @NielsRogge 

I'm able to generate decent questions but only one generic question per table. How can I extend this to generate a question based on a particular cell value? 

Because currently it's giving very basic ones like - 
`what are all of the countries?`
`what are the names of all the drivers?`

Ok great :) sorry I didn't have the time yet to look at your notebook. Looking at it now, it looks really clean!

I think the questions that it will generate highly depend on the training data you provide. I see you're currently training on SQA questions, and only those for which `position==0`. These questions are almost always very generic, because SQA is a dataset involving conversational questions, which means that the first question (with position 0) is most of the time a very generic question regarding a table, and the ones that come after (with position 1, 2, 3) are then more specific follow-up questions (regarding particular cell values).

So either you can also add those follow-up questions to your training dataset, or consider train on questions of the [WTQ](https://nlp.stanford.edu/blog/wikitablequestions-a-complex-real-world-question-understanding-dataset/) dataset? (Note that there is an overlap between WTQ and SQA questions - SQA was created based on WTQ). Or maybe questions from the WikiSQL dataset (which is available in HuggingFace datasets)?

Very nice use case!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Hi @NielsRogge, 
Thanks for the TAPAS implementation!

I'm trying to follow this use-case in order train the model to perform conditional generation from tables.
Since TAPAS can encode the semi-structured meaning in tables, I guessed it was a good choice to use it as an encoder and say GPT2 as decoder.

I however encountered a problem when trying to generate from that EncoderDecoder model:
Here is the relevant pieces of code, this: 
![image](https://user-images.githubusercontent.com/4630195/149192558-a2e6b2b7-4eed-4792-9540-8d76b4fb7b9c.png)
results in this error:
![image](https://user-images.githubusercontent.com/4630195/149193090-a88deb99-724f-4fdc-8010-615a1776e1b9.png)

I guess this is since model.generate() for EncoderDecoder does not expect to have the extra `token_type_ids` that TAPAS has. Can you think of a way I can make this work?

Thanks!