fix(granitemoe*): Only create block_sparse_moe if num_local_experts > 0
Author: gabe-l-hart
Pull request: yes

Branch: GraniteMoeAsDenseFix

# What does this PR do?

With the introduction of `modular_granitemoe.py` in https://github.com/huggingface/transformers/pull/40132, [the conditional](https://github.com/huggingface/transformers/blob/e6a8e7debed69d4cf5d493158f3f347931c24c9a/src/transformers/models/granitemoe/modeling_granitemoe.py#L511) that allowed `GraniteMoe` to also encapsulate dense models as a degenerate case was accidentally removed. This is never actually needed for the `GraniteMoe` architecture directly, but `GraniteMoe` is reused in `GraniteMoeShared` and then `GraniteMoeHybrid` which do need this ability to also encapsulate dense FFN blocks in place of the MoE block.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@ArthurZucker I believe this came in with your PR for MoE in vLLM, so I'd love your sanity check on this fix.

Comments:
Looks like I need to regenerate the `modeling_` code everywhere

Interestingly, when I run `make fix-copies`, the `python utils/check_modular_conversion.py --fix_and_overwrite` script makes far more changes than I would expect from adding this conditional. In particular, for each architecture in the `GraniteMoe*` chain, it adds `GraniteMoe<qualifier>SparseMoeBlock`, then in the `__init__` for `GraniteMoe<qualifier>DecoderLayer`, it adds `self.block_sparse_moe = GraniteMoe<qualifier>SparseMoeBlock(config)` (without any conditional) AND the guarded conditional block `self.block_sparse_moe = GraniteMoeHybridMoE(config)`.

Looking a little deeper, this seems to be caused by the inheritance from `modular_mixtral.py`. I've added a clause that explicitly uses `delattr` to remove `self.block_sparse_moe` when not used, but that seems a bit backwards. Alternatively, we could not inherit from `MixtralDecoderLayer` or we could move the conditional up to `MixtralDecoderLayer.__init__`.

The part that I'm still confused about is why regenerating the `modeling_*` files is adding these `SparseMoeBlock` implementations at all. The inheritance from `Mixtral` was already there prior to my change, so I would expect that those should have already been added unless somehow making creation of `self.block_sparse_moe` conditional triggered logic in the generation to require that those be added?

It appears that putting creation of `self.block_sparse_moe` behind a conditional does, in fact, trigger the inclusion of those `SparseMoeBlock` pieces in the generation. I've used an inline conditional now which seems to prevent this.

🤦 Ok, all that was because of a bad copy-paste somewhere that had me creating a completely incorrect block for `self.block_sparse_moe`. Fixed and cleaned up history.

One more redo: Based on advice from @ArthurZucker, since there are no models using either `GraniteMoe` or `GraniteMoeShared` with the degenerate dense configuration, it's preferable to _only_ have this conditional override in `GraniteMoeHybrid` where it is needed for various flavors of `granite-4.0-*` models.

Hi @gabe-l-hart, thanks for the PR! You can get the code style tests to pass with `pip install -e .[quality]` followed by `make fixup` or `make style`.

Overall it looks good to me, and it does seem like the zero-experts case was accidentally deleted. Will wait for @arthurzucker to confirm before merging!

@Rocketknight1 Thanks! I'll get it cleaned up and hopefully green today

Yep, looks good now! If you don't get core maintainer approval within a few days, ping me and I'll see what I can do

@Rocketknight1 @ArthurZucker I lost track of this one. I just resolved the conflict with the GH web ui, but I didn't look to see if there are other changes that would conflict (ie this being solved at a higher level in the class hierarchy). Do we still need this PR to support non-hybrid/non-moe with `GraniteMoeHybrid`? If so, can we get this looked over?

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: granitemoehybrid

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42036). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.