[WIP]Add TF BEiT Implementation
Author: MadElf1337
Pull request: yes

Porting BEiT model from PyTorch to TensorFlow backend

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #18085 


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@amyeroberts @gante @LysandreJik @NielsRogge 

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
@gante @amyeroberts Here's the WIP draft of BEiT!

Please tell me if I have done anything wrong, I'll make the changes right away!

Thanks!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_18559). All of your documentation changes will be reflected on that endpoint.

Hi @MadElf1337 - thanks for opening a PR and for adding this model! Outline looks good. 

As a quick overview, I see two main things that you'll want to add (alongside docs and tests): 
* `# Copied from` in the TF data2vec model definition
* `TFBeitForXxx` classes

Looking forward to seeing the full PR and having this model available for our TF users :) 

@amyeroberts Sure! I'll make the changes!

@amyeroberts @gante So I think I'm done with the model, can you just look over it once while I'll finish writing the tests?

@MadElf1337 From a quick glance, the model code looks fine 👍 As always, the devil is in the details, so you likely come across issues in the tests. Lets us know if you get stuck in a particular test (tip: `breakpoint()` + comparing to PT are your friends)

Will do an in-depth review when the tests are added.

@MadElf1337 As discussed on the issue #18085 [here](https://github.com/huggingface/transformers/issues/18085#issuecomment-1210544100) for this model, we want to copy the relevant code in data2vec to `modeling_tf_beit.py`, then add the necessary `#Copied from` statements in `modeling_tf_data2vec.py` i.e. `modeling_tf_beit.py` and modeling_tf_data2vec.py` should have the same structure and equivalent `#Copied from` statements as in `modeling_beit.py` and `modeling_data2vec.py`. Let me know if any of this isn't clear or you need any help.  

Yeah yeah it was clear, just wanted to see if the broad architecture was written correctly or not, once I complete the tests(I’m a bit stuck on the attention output test for tf), I’ll do the formatting, add the comments and then ask for a complete review

If you follow the same structure as the pytorch data2vec vision and beit, including the copied from statements, then almost all of the architecture considerations will be taken care of for you, and it will be easier for us as reviewers. 

If you need any help with the tests, let us know and we can try and lend a hand. 

Yeah so as I said, I just am stuck on the seq_len part in the attention output for TF, since that is one thing which is present in data2vec but not in BEIT, So just need to figure out that test

Hey @MadElf1337 -- we've just released a guide for TF conversions, might come handy to you :D 

https://huggingface.co/docs/transformers/main/en/add_tensorflow_model

Yep thanks!

Mostly done with the tests as well, just a little hiccup that will be solved soon, else I’ll make sure to ask for help!

@gante @amyeroberts Terribly sorry for the delay, had to deal with some personal stuff that could not be avoided.

I think I'm done writing the tests and the model, can I get a review to see if I've missed anything/done anything wrong?

Thanks!

(Also I'll add the comments of #Copied from TFData2vec in the final commit)

@amyeroberts @gante 

Can I get a review please?

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_18559). All of your documentation changes will be reflected on that endpoint.

@amyeroberts Thanks for the review!

1) As suggested I've added the comments of #Copied from...(Sorry that you had to ask twice, I thought they were just comments and did not know that it was a part of the review process)

2) I've also added the missing code and the torch references have been changed!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_18559). All of your documentation changes will be reflected on that endpoint.

Hi @MadElf1337  - thanks for the updates and iterating so quickly after review. 

There's still a few files that need to be added for the model to be importable and fully integrated into the library. The guidelines in the document @gante details these. Here's a [recent model PR for reference](https://github.com/huggingface/transformers/pull/17826). As the overall architecture looks good, this is the next step for this PR. 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_18559). All of your documentation changes will be reflected on that endpoint.

@amyeroberts @gante So I've done everything as specified in the docs(I think), can I get a review to see if I've missed anything?

Hey @amyeroberts @gante Can I get a review please?

@MadElf1337 Thanks for the update! 

The next stage for this PR is getting all of the tests running - the fun part! The tests aren't running at the moment as the models can't be imported: 

```
E   ImportError: cannot import name 'TFBeitForImageClassification' from 'transformers' (/home/circleci/transformers/src/transformers/__init__.py)
```

One thing I can see that needs to be added is included the beit models in `import_structure` in the `__init__.py` e.g. [here](https://github.com/huggingface/transformers/blob/7319850902ba9b2a44c36ccddd044f98abd9b597/src/transformers/__init__.py#L205). 

Some of the tests that are failing e.g. `check_code_quality` you can fix and/or find the issues by running `make fixup` locally. 

Finally, the ` # Copied from` statements should be added to the data2vec vision model in `modeling_tf_data2vec_vision.py` 
and the ones in `modeling_tf_beit.py` removed. 
`# Copied from transformers.models.beit.modeling_beit.TFBeitModelOutputWithPooling with Beit->Data2VecVision`


@amyeroberts Thanks for the review!

I can see that the original repo does not have the import structures in __init__.py, however I have added those to the init file in my dev branch, which is why it is showing a conflict for the same file

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hey, can I know what to do next to solve the merge conflict?

Hey @MadElf1337 -- You will have to rebase your PR with main :)

1. Get the latest main
```
git checkout main
git pull
```

2. Rebase
```
git checkout your_branch
git rebase origin/main
```

3. Handle conflicts manually (i.e. keep the desired changes and remove the unwanted ones in the conflicting files, and follow the instructions that git gives you)

4. Force-push your changes (force to avoid GitHub showing a diff of 666 files)
```
git push -u origin your_branch -f
```

There, I think I've solved the conflict but the test errors are occurring due to errors in data2vecvision

@MadElf1337 [Some of the failures](https://app.circleci.com/pipelines/github/huggingface/transformers/55390/workflows/2a766a55-113a-4c4e-8a4f-604926bcf9c4/jobs/668146) are because the `# Copied from` statements point to a path that doesn't exist e.g. 
`# Copied from transformers.models.data2vec.modeling_data2vec_vision.TFData2VecVisionEmbeddings with Data2VecVision->Beit` is copying the object `TFData2VecVisionEmbeddings` but is referring to the pytorch modeling file `transformers.models.data2vec.modeling_data2vec_vision`. 

Note: The copied from statement should be in the `modeling_tf_data2vec_vision.py` file and should copy from the beit model e.g. `# Copied from transformers.models.beit.modeling_tf_beit.TFBeitEmbeddings with Beit->Data2VecVision`. There shouldn't be any `# Copied from` comments in the BEiT modeling file `modeling_tf_beit.py`.

If you run `make fixup` locally in the repo, you'll be able to reproduce the `check_copies` and it will make 
 the `check_code_quality` checks pass. 

Ah I see, I will fix that right away

So the tests run locally, but whenever I run the `make fix-copies` command it changes the docstring in data2vec from data2vec to beit, thus throwing the style change errors.

How do I go about fixing this?

To do that, you'll need to add in the `# Copied from` all the patterns you want to change in the copied code e.g.:
`# Copied from xxx with Beit->Data2VecVision,beit->data2vec_vision`

What might help as well:
* You can add `# Copied from` statements at the method / function level, rather at the top of the class
* For some high-level classes, if it's essentially a wrapper around layers, it might be OK to not include the `# Copied from`

For example, for `TFData2VecVisionForSemanticSegmentation`, I would add copied from headers on top of class methods `__init__` and `compute_loss` and not add it to `call`. 

So now the only errors are in the actual tests for Imagenet and Masked Image Modeling, I think I'll need to add breakpoints and check the tensors manually?

Fixed the deprecated feature_extractor and fixed the comments, however tests still fail

@MadElf1337 - Thanks for updating the feature extractor tests. Regarding the currently failing tests, are you able to click on the `Details` link from the CI runs and see the CircleCI output? 

For tests like `check_code_quality` - you need to run `make fixup` in the top level of the repo. This is detailed in the doc @gante linked to previously: https://huggingface.co/docs/transformers/main/en/add_tensorflow_model#debugging-mismatches-across-ml-frameworks. Make sure to look through this and go through all the steps to get the model ready. 

Do you have an estimation on when you will be able to address these failing tests and have the PR merge ready? BEiT is a very impactful model to add to our TensorFlow library, we'd be excited to include it and we're a lot closer with the work you've done :) This PR has been open for over 6 months now, so would be good to push to get it over the finish line. 

Hey Amy, thanks for the review!

Sorry for the delay, I was out of town for a bit and could not work on the model, but I will try my best to complete the PR within a week's time, since now only the tests part needs to be fixed



Also regarding the tests, whenever I run them in the CLI, I get the Failed message for the following tests:

1) test_attention_outputs
2) test_determinism
3) test_for_masked_lm
4) test_for_semantic_segmentation
5) test_hidden_states_output
6) test_keyword_and_dict_args
7) test_model
8) test_model_common_attributes
9) test_model_from_pretrained
10) test_model_outputs_equivalence
11) test_numpy_arrays_inputs
12) test_prepare_serving_output
13) test_pt_tf_model_equivalence
14) test_save_load
15) test_save_load_config
16) test_saved_model_creation
17) test_inference_image_classification_head_imagenet_1k

and the Imagenet22k test is killed

> Also regarding the tests, whenever I run them in the CLI, I get the Failed message for the following tests:

Are these running tests for just this model? i.e. `pytest tests/models/beit/test_modeling_tf_beit.py`? 

Yes, I used `NVIDIA_TF32_OVERRIDE=0 RUN_SLOW=1 RUN_PT_TF_CROSS_TESTS=1 \
py.test -vv tests/models/beit/test_modeling_tf_beit.py`

OK, that's expected - even once the initial architecture's done there's always failures on the first go :) 

You'll need to look at the errors being thrown on the tests and debug from there - let us know if you need any help!

@amyeroberts So the main error seems to be in the `call` function for every failing test, i.e.

`AttributeError: Exception encountered when calling layer 'tf_beit_model' (type TFBeitModel).
'TFBeitModel' object has no attribute 'beit'`

When I tried running the tests for TFData2vec_vision, I came across the same errors

The error causing snippet of code is this - 
`outputs = self.beit(
            pixel_values=pixel_values,
            bool_masked_pos=bool_masked_pos,
            head_mask=head_mask,
            output_attentions=output_attentions,
            output_hidden_states=output_hidden_states,
            return_dict=return_dict,
            training=training,
        )`

@MadElf1337 This is because the `# Copied from ` statements in modeling_tf_data2vec.py need to be updated. Please refer to my comments in modeling file e.g. [this one](https://github.com/huggingface/transformers/pull/18559/files#r1081377920).

But then it should not throw that error for beit right?

Because it is not copying from anything

And even after putting the `#copied` statements as suggested, it still throws the same error

Also `make fixup` now throws `Error 127`

`make: /bin/sh: Argument list too long`
`make: *** [Makefile:10: modified_only_fixup] Error 127`

> But then it should not throw that error for beit right?

Yes - the updates should fix the issues for data2vec once `make fix-copies` has been run. 

The error is telling you that `self.beit` isn't defined for `TFBeitModel`. I can see, looking at the code `data2vec_vision` is defined instead in `modeling_tf_beit.py`. Any data2vec references will need to be updated in the beit modeling file. 

> Also make fixup now throws Error 127

I'm not sure what the issue is here. If I fetch your fork I'm able to run `make fixup` with no issues on this branch. Are you able to run if locally, with no problems on `main`? This will help diagnose whether there's an issue running `make` or the issue is with code on this branch. 

Okay so now it is down to just 2 failed tests, one is 
`tests/models/beit/test_modeling_tf_beit.py:160: in create_and_check_for_masked_lm
 self.parent.assertEqual(result.logits.shape, (self.batch_size, num_patches, self.vocab_size))`
 `AssertionError: TensorShape([13, 225, 2]) != (13, 225, 100)`

and second one is:
`mask_tokens = tf.broadcast_to(self.mask_token, (batch_size, seq_len, projection_dim))`
`ValueError: Exception encountered when calling layer 'embeddings' (type TFBeitEmbeddings).`
           
`Attempt to convert a value (None) with an unsupported type (<class 'NoneType'>) to a Tensor.`
           
`Call arguments received by layer 'embeddings' (type TFBeitEmbeddings):
• pixel_values=tf.Tensor(shape=(13, 3, 30, 30), dtype=float32)
• bool_masked_pos=tf.Tensor(shape=(13, 225), dtype=int32)`

`src/transformers/models/beit/modeling_tf_beit.py:173: ValueError`



I think for the first one I might've messed something up in the masked lm function but can't figure out what

@MadElf1337 You will also need to run the tests for the data2vec vision model to ensure it still works as expected. I can see at the moment some of these tests are currently failing.

For the first test:`tests/models/beit/test_modeling_tf_beit.py::TFBeitModelTest::test_for_masked_lm`
the output dimensions of the logits are different. You can compare the dimensions of the tensors in each step of `TFBeitForMaskedImageModeling.call` to `BeitForMaskedImageModeling.forward` to find where the difference is arising and from there figure out if it's coming from e.g. a specific layer or logic such as slicing. I would first check the creation of the layers to make sure they have the same settings as in `modeling_beit.py`

For the second test: `mask_tokens = tf.broadcast_to(self.mask_token, (batch_size, seq_len, projection_dim))`, the error is telling you `self.mask_token` is `None`.  I can see this line happens if `bool_masked_pos is not None`. You'll need to see what is being passed in at test time and if this is expected e.g. is self.config.use_mask_token set correctly? 

Yeah I'll debug those 2 errors by today, and will also look at the data2vec model and tests afterwards

Okay I fixed the first one, I had mistakenly put `num_labels` instead of `vocab_size` in the Classifier head

@amyeroberts Okay so all tests are running locally now

@amyeroberts So if I run `pytest tests/models/beit/test_modeling_tf_beit.py`, it shows that all the tests are passing, however on running `NVIDIA_TF32_OVERRIDE=0 RUN_SLOW=1 RUN_PT_TF_CROSS_TESTS=1 \ py.test -vv tests/models/beit/test_modeling_tf_beit.py` the imagenet1k test fails and the imagenet22k test is killed.

@MadElf1337 OK, thanks for the update! For the slow tests that are failing do you need any help in resolving these? Is imagenet22k killed because of OOM? 

As mentioned before, you'll also need to run these tests for data2vec vision. I can see on CI these are currently failing.  

Yes I think Imagenet22k is being killed due to OOM, but I don't know why the imagenet1k test is failing, as I can't see the error message in the slow tests.

And yes, once I finish with BEIT tests I'll take care of the data2vec vision tests as well

> as I can't see the error message in the slow tests.

What do you mean exactly- is there no output at all?  

If you run: 
```
NVIDIA_TF32_OVERRIDE=0 RUN_SLOW=1 RUN_PT_TF_CROSS_TESTS=1 py.test -vv tests/models/beit/test_modeling_tf_beit.py::TFBeitModelIntegrationTest::test_inference_image_classification_head_imagenet_1k
```

pytest should show you the exception or assert that was raised e.g. 

```
FAILED tests/models/beit/test_modeling_tf_beit.py::TFBeitModelIntegrationTest::test_inference_image_classification_head_imagenet_1k - {Reason the test failed}
```

and above the line in the code where the failure occurs. 

Is this an issue only for the slow tests? 

Ah I did not run the test standalone, I ran the entire suite every time, which is why it just showed me the `FAILED` message.

Imagenet gives the following error - `AssertionError: TensorShape([1, 2]) != <tf.Tensor: shape=(2,), dtype=int32, numpy=array([   1, 1000], dtype=int32)>`

Masked Image Modeling head gives - `tensorflow.python.framework.errors_impl.InvalidArgumentError: {{function_node __wrapped__Sub_device_/job:localhost/replica:0/task:0/device:CPU:0}} Incompatible shapes: [1,3,8192] vs. [3,3] [Op:Sub]`

I think the assertion errors are due to some error in the model architecture?


Any pointers on how I can fix the imagenet1k fail?, I tried going through the entire model but still can't understand why that test is failing

@MadElf1337 I'm not sure why it's failing and I don't see it as a failed test on the CI runs. If it's the same imagenet error [posted here](https://github.com/huggingface/transformers/pull/18559#issuecomment-1422953538): 
`AssertionError: TensorShape([1, 2]) != <tf.Tensor: shape=(2,), dtype=int32, numpy=array([   1, 1000], dtype=int32)>`

Then it seems it's a case of output shape. The shapes aren't wildly different, as one is a flattened version of the other. I would compare the shape of `pooled_outputs` before they're fed to the classifier between the pytorch beit and tensorflow beit models and work from there. If they're the same, then the issue is in the classifier. If they're different, keep going up the layers until you hit where the difference is arising. 

I got the reason for the error, it is a problem in the classifier itself, the shape returned by the beit classifier head is (3,2) where it should be (3,1000), but I do not understand why it throws that error since the architecture is exactly the same as of data2vec and the test passes for data2vec

Everything else is done, just the imagenet test is failing due to some problems in the classifier, don't know why it is erroring out since it is the same as of data2vec.

@amyeroberts 

@MadElf1337  - I can see there are other tests failing for beit, outside of the imagenet test e.g. `tests/models/beit/test_modeling_tf_beit.py::TFBeitModelTest::test_attention_outputs`. Would you be open to me fetching this branch and pushing some commits to see if I can resolve the failing tests? 

Oh that test is failing due to the breakpoint probably, wait I'll remove those completely.

And sure, please feel free to push commits :)

@amyeroberts the attention tests are passing for me on local, not sure why they are failing here

The only tests failing for me are the pt_tf_model_equivalence and imagenet1k, imagenet 22k goes OOM

Excuse my lack of knowledge on this point, but when this feature is ready, can I use DiT with TFBEiT, since it is said that DiT uses the same Beit architecture?

@FelipeErmeson  Yes! Just like the pytorch model, once this model is added you will be able to load in DiT weights into TFBeit like so:

```
from transformers import TFAutoModelForImageClassification

model = TFAutoModelForImageClassification.from_pretrained("microsoft/dit-base-finetuned-rvlcdip")
```

Can I get some help on this?

I can't seem to understand why the classifier throws an error for Imagenet, went through the entire architecture multiple times

Hi @MadElf1337 - yes, I'm looking into this now :) 

@MadElf1337 - I've managed to track down a few of the issues. As this is from a fork on your repo - I couldn't push the diffs, but I'll comment on the relevant bits of code that need to be updated. 

@amyeroberts Hey!
Still getting the same errors for imagenet and pt_tf_model_equivalence, those are the only 2 tests failing

@MadElf1337 For the pt_tf_equivalence failures, you'll need to go through the layer outputs and compare with the pytorch model to figure out where the differences are coming from. I recommend starting with `TFBeitModel` first, and progressively checking the inputs and outputs of larger blocks before digging into the lower layers. The way I typically tests is passing the same input to the PyTorch and TensorFlow models and saving out activations at different steps to numpy arrays. 

Common causes of differences are layer parameters like `eps` for layer or batch norms. 

@amyeroberts So all tests except imagenet are passing now :-(

I think the problem is in the num_channels in the classifier, it is not being initiated properly or something as the output is squashed

@MadElf1337 - OK. I'll dig into the imagenet issues a bit later. 

Whilst that is happening, the next step for this PR would be rebasing on main, and re-running the tests to make sure everything is still compatible. There's been some recent changes to the formatting libraries used, so it's important to make sure these are up-to-date by running `pip install -e .[quality]` and then linting the code for the update with `make fixup`

@amyeroberts So I ran `make fixup` and I am getting the following error:
`Exception: There were 1 failures:
TFBeitForMaskedImageModeling is defined in transformers.models.beit.modeling_tf_beit but is not 
present in any of the auto mapping. If that is intended behavior, add its name to 
'IGNORE_NON_AUTO_CONFIGURED' in the file 'utils/check_repo.py'.`

I think this is because we removed MaskedImageModeling from the auto class?

@MadElf1337  - `BeitForMaskedImageModeling` isn't compatible with `AutoModelForMaskingImageModeling` and so isn't part of the auto mapping. As the error indicates, `TFBeitForMaskedImageModeling` should be added to `IGNORE_NON_AUTO_CONFIGURED` e.g. like here for the [pytorch model](https://github.com/huggingface/transformers/blob/89087597ba8c565e7b854ae719b0230de8435a10/utils/check_repo.py#L252). 

I've ran `make fixup` according to the new formatting conditions, what to do next?

@MadElf1337 Have you rebased on main? I can't see a force push in the recent history cf [prev instructions here](https://github.com/huggingface/transformers/pull/18559#issuecomment-1383991956). If not, then after the rebase, you'll need to update the formatting settings with `pip install -e .["quality"]` and run `make fixup` again. 

The next step after that is to get all of the tests passing. 

Completed the rebase, it shows a conflict here but says `Current branch MadElf1337-dev is up to date.` in my terminal

@MadElf1337 Are you fetching and rebasing on the most recent version of main? i.e. 

```
git fetch upstream main
git rebase upstream/main
```

There, Fixed it

@MadElf1337 - great :) The next steps are resolving the failing tests. 

For the quality checks, you'll need to make sure that you have the most recent formatting configuration locally by running `pip install -e .["quality"]` in the repo. Running `make fix-copies` and `make style` should resolve these. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hey @amyeroberts 

Sorry for the delay, I was extremely sick and hence could not contribute, but I think the bulk of the suggestions are done, just the comments and the test part is remaining, but the PR was closed

@amyeroberts I think I've covered everything that was suggested, still Imagenet throws errors

@MadElf1337 

For the repo consistency tests, could you run `make fix-copies` and push those changes 

For the quality checks, could you run `make style` at the top level of the repo and push any changes? These should be run after running `make fix-copies` 



@amyeroberts Done!

@MadElf1337 The repo consistency checks are still failing. Could you try running `make fix-copies` again and push those changes?

@amyeroberts I think the consistency check is fixed now

@MadElf1337 Consistency checks have been fixed, but there are still a few tests (outside of doctests and imagenet) which are failing, including the quality/style checks I mentioned fixing with `make style` above. These should be resolved before a final review. 

For the doc tests, you should add `from_pt=True` for the moment, to make sure that they run, and then we remove that before we merge in and once the checkpoints have been uploaded

@amyeroberts  I've already added the `from_pt=True` flags to `test_modeling_tf_beit` file, still the doctest throws an error saying that the tf weights cannot be loaded

@MadElf1337 Could you please look at the tracebacks on the failing CI runs first? `from_pt=True` hasn't been added to the doc examples for `modeling_tf_beit.py`, this is [clear from the error message](https://app.circleci.com/pipelines/github/huggingface/transformers/67868/workflows/52e9162b-86bf-4d97-853e-ae75d410976a/jobs/850220).

We're happy to help clarifying bugs if they're not clear, or help bug something esoteric. However, in this case, what I would tell you to do is exactly what the error message indicates. It's simply not scalable for us to help on every individual failing test.

Yes Yes I understand, however the tests for pr documentation were passing on earlier runs, which is why I was a bit confused on what suddenly happened so as they were failing now, turns out that `make fix-copies` might've overwrote over the stuff I wrote myself

@amyeroberts Do I have to manually write the examples in the docstrings of `TFBeitForImageClassification` & `TFBeitForMaskedImageModeling` as done with `TFBeitForSemanticSegmentation`?

Because the example docstrings are nowhere to be found on my local repo

@MadElf1337 You don't need to have examples added to the docstring if the method has a `@add_code_sample_docstring` decorator. It might be the case that the standard docstring example isn't compatible with the model or doesn't exist for that task. In this case, a custom example should be added to the docstring, and no decorator used. 

Ah okay!

I''ll add the custom docstrings

@amyeroberts why does the code-quality check throw an error for `Dict` only in data2vec? The same name works in BEiT but not in the model file of data2vec

@amyeroberts The code quality checks are failing due to a lowercase-uppercase issue that gets reverted whenever I run `make style`, any suggestions on tackling this?

@MadElf1337 Do you have the versions if black and ruff installed that correspond to the current versions expected by the library? 

The quality checks only fail if there are changes that would be made by running `make fix-copies` or `make style`. So by running the make commands, changes that would resolve the problem are made, rather than created. 

As I have stated many times before in this PR, please look at the errors on the CI runs. These will tell you what the issues are and how to resolve them. [I can see](https://app.circleci.com/pipelines/github/huggingface/transformers/68484/workflows/9d7a4249-3a4a-4fb5-a5c1-aeba5cf99fa8/jobs/857594) that the issue is arising in data2vec because `Dict` isn't defined.