[WIP]Add openpangu_dense model
Author: yzy1996
Pull request: yes
Labels: name: New model

# What does this PR do?

This PR introduces inference support for the openPangu-Embedded model. By integrating the inference implementation directly into the Transformers library, we aim to ensure seamless compatibility and enable continuous updates alongside future open-source releases.

This implementation requires torch_npu as a dependency. I'd like to confirm whether introducing this NPU-specific dependency aligns with the community's guidelines and acceptance criteria for the Transformers library. Any guidance on this matter would be greatly appreciated.

I welcome any feedback on potential improvements or additional changes that would enhance this implementation. Please feel free to share your suggestions!

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR. @ArthurZucker


Comments:
Hmm, it seems like this model has the same architecture as Llama, but just adds an NPU operation in attention. I'm not sure we need a whole architecture for that! cc @cyrilvallez for attention, @ivarflakstad for NPU

Thanks for the reply! Due to precision and performance gaps on NPU, we'll gradually add more NPU operators (beyond just attention ops). Although we have the same architecture as Llama, we'd like to add a separate model to keep it updated. 

Hey @yzy1996! Indeed, as @Rocketknight1 mentioned, modeling code should be agnostic to the device used, so we don't really want special conditional branching based on accelerator type. You can however register custom attention function and masking to use with Llama if you follow what is shown here https://huggingface.co/docs/transformers/en/attention_interface, which will allow you to perform npu-specific optimizations!

Related to #39238 for NPU we'd love to just have proper `kernels` support!

> Related to #39238 for NPU we'd love to just have proper `kernels` support!


We are excited to announce NPU support for kernels! 
- https://github.com/huggingface/kernels/pull/155
- https://github.com/huggingface/kernels/pull/146

Example of Usage: 
- https://github.com/huggingface/transformers/pull/41098




**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, openpangu_dense