Introduce Textnet backbone (needed for Fast model)
Author: raghavanone
Pull request: yes

This is needed for merging fast model #26501

Comments:
@amyeroberts  I have pulled out all the changes for textnet backbone needed for fast model (https://github.com/huggingface/transformers/pull/26657) into separate PR. Requesting for a first round of review.

> Thanks for adding!
> 
> I've done an initial pass - it'll likely be at least one more review round before ready for approval. Main comment is about the deletion and addition of model parameters. The priority when adding models to transformers is to make sure that they're easy to read and understand - the current fused kernel logic should be simplified or completely removed. Have you run tests to see compare running times with and without the fused kernels?

Fair point, I just ran the tests. The fused kernel just saves some 5 % time during eval. I am going to remove the complex logic.

@amyeroberts  I have addressed almost all of the feedbacks, I have some questions of few of them. Help me with some more details for them.

@amyeroberts The feedbacks on configuration refactoring has been addressed.

@amyeroberts  With respect to textnet's image processor being present, I had removed it in favour of clip image processor, But then later realised that in the original preprocessing code, they do take the shortest_side as parameter, scale the short the side by that factor , then scale the long side by same factor. (like being in done our image processors). But in addition to this, they also align both the width and height of the final image to closest factor of 32.  ([Ref](https://github.com/czczup/FAST/blob/7ab105474ab226ab74a346b0648d2e2baab67d79/dataset/utils.py#L205)) . Hence I had to introduce image_processing_textnet. Let me know if I am missing something else.

@amyeroberts  Gentle reminder.

Hey, @amyeroberts is off, if this is urgent ping me or let' s wait until next week! 

I think this can wait .

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

@amyeroberts  All Feedbacks has been incorporated. Please have a look

@raghavanone Please make sure to look at the diff before asking for review, to catch any obvious things that should be updated. Opening the files tab, the first thing one sees are README's that need to be filled in. 

> @raghavanone Please make sure to look at the diff before asking for review, to catch any obvious things that should be updated. Opening the files tab, the first thing one sees are README's that need to be filled in. 

Oh , the rebase to main should have undone some of these changes . Let me fix and review them again .

@amyeroberts  I have fixed the issues coming from rebase to main and also validated that all the feedbacks are incorporated. Please have a look .

@amyeroberts  Let me know how to update the checkpoint to organisation  . 

Hi @Raghavan, nice work on getting TextNet across the line! 

For adding the checkpoints to the hub, I’d suggest reaching out to the lead paper author and asking them if they would be happy to host the model weights on the hub under their profile. As the model appears to have been a collab between universities, there isn’t an obvious org for it to go under. 

@NielsRogge has more experience here, so will have recommendations for everything that needs to be done. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.