A couple of noob-to-transformers questions
Author: GrahamboJangles
Pull request: no
Labels: name: wontfix

## ❓ Questions & Help
#### If you don't want to or don't know the answer to all of these, just answer some that you know!
1. How is it that you can provide context to these models? Say, if you want to summarize or pull data from a text. Do you have to train it on that text or just put it somehow in the prompt?
2. Can you train each model? Like, if you wanted to, completely unfreeze and retrain each model?
4. How can I train? I ran into a bunch of issues including [this one](https://github.com/huggingface/transformers/issues/1517) when just running the official sample scripts in Colab.
5. Are certain models better at certain tasks? Which ones are good for what?

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


@GrahamboJangles

1) If you want to provide _context_ into any model offered by Transformers, you can **fine-tune** the model you've chosen with your custom data in such a way that the model can learn the context. 
I suggest you to look the _CTRL_ model developed by SalesForce. By using this model, you can pass a parameter called _control_code_ which specify the domain of the generated text by the model itself, for example Fitness, Funny, Diet, etc. [Here](https://github.com/huggingface/transformers/blob/master/transformers/modeling_ctrl.py) you can find the source code of the model implemented in Transformers, and [here](https://arxiv.org/pdf/1909.05858.pdf) you can find the scientific paper about it.

2) **Yes**, you can train the model you've chosen from scratch. You can import it with the pre-trained weights, unfreeze all layers and set random weights and starting to train the model.

3) There are many Python scripts written in TensorFlow 2.0 or PyTorch for fine-tuning the model. The from-scratch training scripts are missing, at the moment (in my best knowledge). You can find maybe some advide on the _Issues_ page.

4) **Yes**, and I think that in the Python scripts in this library they suggest to you to use a particular set of models for a certain task. In more details, they have developed the base model architecture, and they have added some layers on-top for addressing a particular task, e.g. you can use _BertForTokenClassification_, _RobertaForTokenClassification_, _DistilBertForTokenClassification_, _CamembertForTokenClassification_ for token classification. More details [here](https://github.com/huggingface/transformers/blob/master/examples/run_ner.py).

> ##  Questions & Help
> #### If you don't want to or don't know the answer to all of these, just answer some that you know!
> 1. How is it that you can provide context to these models? Say, if you want to summarize or pull data from a text. Do you have to train it on that text or just put it somehow in the prompt?
> 2. Can you train each model? Like, if you wanted to, completely unfreeze and retrain each model?
> 3. How can I train? I ran into a bunch of issues including [this one](https://github.com/huggingface/transformers/issues/1517) when just running the official sample scripts in Colab.
> 4. Are certain models better at certain tasks? Which ones are good for what?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.