fix auxiliary loss training in DetrSegmentation
Author: SangbumChoi
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [v] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @muellerzr and @pacman100

Integrations:

- deepspeed: HF Trainer/Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc and @younesbelkada

Documentation: @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->
Hi, @amyeroberts. I fixed the training error when setting auxiliary loss to set 'True'
`outputs_class = self.detr.class_embed(hs)`
Since original DETR does not have their own class of self.class_labels_classifier, we have to import from self.detr

see https://github.com/facebookresearch/detr/blob/3af9fa878e73b6894ce3596450a8d9b89d918ca9/models/segmentation.py#L49

Comments:
@amyeroberts Hi back again with the test!
Mainly I added function called test_forward_auxiliary_loss. This will check if the two major components when the config.auxiliary_loss is set to True
1. whether output of auxiliary loss is not None
2. check the output len is equivalent to the num_hidden_layers - 1 (-1 is due to last layers is not 'aux')

BTW, this test will also check in not only segmentation but also object_detection_model!

following result is the pytest of detr
```
root@0a2b4fe54761:/mnt/nas2/users/sbchoi/transformers# RUN_SLOW=1 pytest tests/models/detr
================================================================================ test session starts =================================================================================
platform linux -- Python 3.10.13, pytest-7.4.4, pluggy-1.0.0
rootdir: /mnt/nas2/users/sbchoi/transformers
configfile: pyproject.toml
plugins: hypothesis-6.92.0, hydra-core-1.3.2
collected 154 items

tests/models/detr/test_image_processing_detr.py ................                                                                                                               [ 10%]
tests/models/detr/test_modeling_detr.py .......................ssssss..sssssssss......s...............s......s........s....ssssssss.ss.sssssssssssssss.s..s.......s........... [ 97%]
....                                                                                                                                                                           [100%]

================================================================================== warnings summary ==================================================================================
../../../../../opt/conda/lib/python3.10/site-packages/_pytest/config/__init__.py:1373
  /opt/conda/lib/python3.10/site-packages/_pytest/config/__init__.py:1373: PytestConfigWarning: Unknown config option: doctest_glob

    self._warn_or_fail_if_strict(f"Unknown config option: {key}\n")

src/transformers/deepspeed.py:23
  /mnt/nas2/users/sbchoi/transformers/src/transformers/deepspeed.py:23: FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations
    warnings.warn(

../../../../../opt/conda/lib/python3.10/site-packages/torch/utils/cpp_extension.py:28
  /opt/conda/lib/python3.10/site-packages/torch/utils/cpp_extension.py:28: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    from pkg_resources import packaging  # type: ignore[attr-defined]

../../../../../opt/conda/lib/python3.10/site-packages/pkg_resources/__init__.py:2871
../../../../../opt/conda/lib/python3.10/site-packages/pkg_resources/__init__.py:2871
  /opt/conda/lib/python3.10/site-packages/pkg_resources/__init__.py:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('ruamel')`.
  Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
    declare_namespace(pkg)

tests/models/detr/test_modeling_detr.py::DetrModelTest::test_disk_offload_bin
  /opt/conda/lib/python3.10/site-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly.  To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
    return self.fget.__get__(instance, owner)()

tests/models/detr/test_modeling_detr.py::DetrModelTest::test_fast_init_context_manager
  /mnt/nas2/users/sbchoi/transformers/tests/test_modeling_common.py:460: FutureWarning: `torch.testing.assert_allclose()` is deprecated since 1.12 and will be removed in a future release. Please use `torch.testing.assert_close()` instead. You can find detailed upgrade instructions in https://github.com/pytorch/pytorch/issues/61844.
    torch.testing.assert_allclose(init_instance.linear.bias, expected_bias, rtol=1e-3, atol=1e-4)

tests/models/detr/test_modeling_detr.py::DetrModelTest::test_fast_init_context_manager
  /mnt/nas2/users/sbchoi/transformers/tests/test_modeling_common.py:463: FutureWarning: `torch.testing.assert_allclose()` is deprecated since 1.12 and will be removed in a future release. Please use `torch.testing.assert_close()` instead. You can find detailed upgrade instructions in https://github.com/pytorch/pytorch/issues/61844.
    torch.testing.assert_allclose(

tests/models/detr/test_modeling_detr.py::DetrModelTest::test_pipeline_image_segmentation
tests/models/detr/test_modeling_detr.py::DetrModelTest::test_pipeline_object_detection
  /opt/conda/lib/python3.10/site-packages/huggingface_hub/repocard.py:105: UserWarning: Repo card metadata block was not found. Setting CardData to empty.
    warnings.warn("Repo card metadata block was not found. Setting CardData to empty.")

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
============================================================== 107 passed, 47 skipped, 10 warnings in 110.95s (0:01:50) ==============================================================
root@0a2b4fe54761:/mnt/nas2/users/sbchoi/transformers#
```

@amyeroberts No problem! However, should I make another PR or just modify in this PR?

@SangbumChoi Up to you. I'd suggest making a follow-up PR to prevent holding this one up, but happy to go for whatever you'd prefer

@amyeroberts Yeah, I will make follow-up PR and link this PR also. If there's nothing else to add for this topic, let's MERGE this first :)

@SangbumChoi Merged! Thanks again for this contribution 💪