Adding support for Nandi Models
Author: HemanthSai7
Pull request: yes

Co-authored-by: Vishesht27

This PR adds support for codes for the upcoming Nandi series models. We also appreciate the valuable feedback and thorough review provided by @vasqu and @ArthurZucker 🤗🙏

Comments:
We are the team at **RTA AI Labs**, a tiny but passionate startup dedicated to making high-performance language models more accessible through efficient architecture. Today, we are excited (and a little nervous!) to submit this PR to add support for **Nandi**, our custom "smol" model series. As a very small team, we have poured our hearts, late nights, and limited resources into building Nandi. We believe that the future of AI belongs to efficient, edge-compatible models, and we’ve designed Nandi to punch significantly above its weight class in terms of reasoning and throughput. Bringing Nandi to the Hugging Face ecosystem is a massive milestone for us. It is the "make or break" step for our upcoming release, as it will allow the community to easily fine-tune, deploy, and experiment with what we've built.

**Why this PR matters**
- **Architecture**: Combines **Factorized embeddings**, Grouped Query Attention (GQA), RoPE, layer sharing for high efficiency.
- **Community Impact**: Enables developers to build and deploy capable models without large-scale compute.

**A small note to the maintainers**
We know how incredibly busy the `transformers` maintainers are, and we have the utmost respect for the work you do in keeping this ecosystem thriving. For a small lab like ours, getting this PR reviewed and merged is more than just a technical update, it is the foundation of our startup’s mission. We have done our best to follow the contribution guidelines strictly to make the review process as smooth as possible for you. We are standing by to make any requested changes immediately.

@Rocketknight1 @vasqu @ArthurZucker @xenova @zucchini-nlp 🤗

Heya super excited to see this :wave: 

Just as a heads up, we have holidays + torch conference so reviews will be delayed for at least a week ish. Not sure if I will be the one reviewing or someone else, but as a first step it would be best to fully utilize modular https://huggingface.co/docs/transformers/v5.5.0/en/modular_transformers#implementing-a-modular-file

Appreciate the work, and dont hesitate to ping us :smile: 

Hi @vasqu, thanks for the heads up! I’ve updated the PR to fully utilise the modular transformer format as suggested in the documentation. Looking forward to your feedback whenever you're back. 

🎉 https://huggingface.co/collections/Rta-AILabs/nandi-mini

Hi @xenova ,  I’ve inherited the Llama modules wherever necessary and followed the modular structure closely. I also removed the deprecate_kwargs decorator. Please let me know if there are any other changes you’d like me to make.

hi @xenova , we have made all changes from our side, can you review and please give us some feedback

Sorry this PR got a bit lost in my notifications, will try to take a look next week. Could you sync with main in the mean time and fix whatever broke :hugs: 

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: auto, nandi

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=45101&sha=00c441