Make Conv1D.bias optional
Author: comaniac
Pull request: yes

# What does this PR do?

A simple change to make Conv1D.bias optional.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

Conv1D is used in GPT models: @patrickvonplaten, @LysandreJik

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._

@sgugger thanks for the comment. Then in the case that if I'd like to disable bias in GPT-2 models, should we directly change the GPT-2 model implementation to avoid custom Conv1D, and deprecate it?

You can change the modeling code as you'd like to suit your need. We won't upstream it since it's not in the official GPT-2 code though.

So the code change guides are summarized as follows:
1. Official model code (e.g., GPT-2) cannot be changed to deprecate the use of Conv1D.
2. Conv1D in transformers is legacy and only used in some existing models (e.g., GPT-2).

IIUC, then I still feel this PR is required, as Conv1D cannot be deprecated and is still a "building block" of some models. Is there any strong restriction (e.g., code freeze) that these legacy code cannot be changed?

Thanks.

You can open a PR to remove the Conv1D use in GPT-2 and, but we won't add a new argument to control the bias like this. You can copy the modeling code and adapt it to your needs if this is something you want yourself.

I see. Yeah that makes sense and is actually my intention. I don't plan to add an argument to existing GPT-2. I just want to make sure I don't need to change other places in transformers to disable the bias in GPT-2. Thanks.