fix for custom pipeline configuration 
Author: not-lain
Pull request: yes

# What does this PR do?
 
fixes configuration file not pointing at a remote repo with custom pipeline architecture

Fixes #28907 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@Rocketknight1

Comments:
@Rocketknight1 could you review this one too ?
fixed the tests (i forgor to update my branch :D ) 

Sure, I'll try to take a look at this one and the pipeline upload one!

cc @Rocketknight1 any review on this one

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29004). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

cc @ArthurZucker for core maintainer review

cc @Rocketknight1 @ArthurZucker 
I made this video to explain why this is important https://youtu.be/2LPJ_1QuK90

it also explains the fundamentals of the transformers library on how to load the custom architecture from a repo that has the architecture defined on another one, which is a workaround i used before on [this](https://huggingface.co/briaai/RMBG-1.4/discussions/9) .

it's pretty much always depends on the config.json which is why it's essential to fix it

@ArthurZucker in this pr the configuration tag is added **when we are calling the model** not when we are saving it which is why it is funadamentally different from the other pr.

check more about the config.json in this [repo](https://huggingface.co/not-lain/29004) and then try loading the model and check the config after calling it for more details
```python
# Load model directly
from transformers import AutoModelForImageClassification
model = AutoModelForImageClassification.from_pretrained("not-lain/29004", trust_remote_code=True)
model.config
```

i know it's pretty weird but this is how the transformers library is working rn.

as for the tests
* could you create a repo in https://huggingface.co/hf-internal-testing and i'll add a pr in there with a custom pipeline
* or you can copy my repo under https://huggingface.co/not-lain/29004 and we can use it as a reference

@ArthurZucker 
your confusion is on me, i have wrongly mentioned the issue, now after checking the code and experimenting more i now know the configuration is being changed when we are calling the model and not when we are saving/pushing 

again apologies for the confusion

**EDIT**
the issue still stands for cases such as finetuning and others since we will push wrongly annotated config to a new repo meaning the new repo will have a broken pipeline.
I will also try to add dummy tests soon.

cc @ArthurZucker
this should be ready for review

@ArthurZucker @Rocketknight1 
could you review https://github.com/huggingface/transformers/pull/29172 first, since i need the push_to_hub feature from there added to the main branch to add more coverage to the tests just to be safe.

i'll update the tests on this pull request right after, sending lots of hugs 🤗🤗

Sorry on our late answers here! I was off for a while ! 

@ArthurZucker hope you had fun ✨✨, anyways I have updated the tests to be more straight forward and to the point, do let me know if you have any reviews on this

@ArthurZucker there's no need to wait for `29172` anymore since in 323b50d666a1fcb5ca09db7c6c4152b9bd5834a2 I switched to using the model to push to another repo (case finetuned model with custom pipeline), also friendly pinging for a review

Hi @ArthurZucker 
Any reviews on this PR? 

@ArthurZucker friendly tagging you here

hey @ArthurZucker , hope you're doing well.
can I get a review on this PR ?

Wow I am sorry @not-lain really my bad here

@ArthurZucker it's ok, you have been busy with lots of stuff including llama-3 and I know things are hectic on your side, just remember to take a break once in a while.
Also thanks a lot for the review, and I wish you a wonderful day ✨