Adding SegGPT
Author: EduardoPach
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

This PR adds `SegGPT` to the transformers library

- [Paper](https://arxiv.org/pdf/2304.03284.pdf)
- [Code](https://github.com/baaivision/Painter/tree/main/SegGPT/SegGPT_inference)
- [Checkpoint](https://huggingface.co/BAAI/SegGPT/blob/main/seggpt_vit_large.pth)

Fixes https://github.com/huggingface/transformers/issues/27514


## TO-DOs

- [x] Finish Implementation
- [x] Write conversion script
- [x] Finish `ImageProcessor`
- [x] Make all tests green 

Comments:
@NielsRogge just pinging you here

Hey @NielsRogge, could you take a look at the PR?  Code quality is failing, but is unrelated and the documentation tests I don't know what is going on because it's failing on an example that it's not even in the `modeling_seggpt.py`

Looks like we're nearly there. Just one comment, could we make the current `post_process_masks` method consistent with the existing `post_process_semantic_segmentation` and `post_process_instance_segmentation` methods in the library (perhaps also leveraging the same name)? These return a set of binary masks.

I have this notebook to showcase inference: https://colab.research.google.com/drive/1MZ0quroT0E2c5mnJjmjdDo_FrDXTUCQD. 

> Looks like we're nearly there. Just one comment, could we make the current `post_process_masks` method consistent with the existing `post_process_semantic_segmentation` and `post_process_instance_segmentation` methods in the library (perhaps also leveraging the same name)? These return a set of binary masks.
> 
> I have this notebook to showcase inference: https://colab.research.google.com/drive/1MZ0quroT0E2c5mnJjmjdDo_FrDXTUCQD.

Some of the modifications I did might break your inference example. Instead of using `post_process_masks` you should now use `post_process_semantic_segmentation` here is an example of my own https://colab.research.google.com/drive/1UfgLOOVHZJhdxIzfIc6Z1njdOkGTFTJT?usp=sharing

> Awesome work! Thanks for all the work adding and iterating on this
> 
> Just some tiny nits. Please do not just mark comments as resolved when they are not - if you don't think the suggestion should be applied then comment with why on the comment before marking.
> 
> Main thing left to do is update all the checkpoints and make sure all the model tests (incl slow) pass with these checkpoints. Once that's all done it'll be ready to merge :)

@amyeroberts what do you mean by updating the checkpoints? Move to the appropriate org or re-upload them (it's just one) with the conversion script. 

@EduardoPach The checkpoints should be uploaded under the correct org - @NielsRogge can help you with that. Then all the checkpoint references in this PR need to be updated to point to the right location 

> @EduardoPach The checkpoints should be uploaded under the correct org - @NielsRogge can help you with that. Then all the checkpoint references in this PR need to be updated to point to the right location

Hey @amyeroberts checkpoints were moved and tests are green

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_27735). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.