feat: Add ConvaiCausalLM model for Hindi Causal Language Modeling
Author: NandhaKishorM
Pull request: yes
Labels: name: New model

# What does this PR do?

This PR introduces the `ConvaiCausalLM` model, a causal language model specifically designed and trained for Hindi text generation by Convai Innovations. The reference checkpoint is available on the Hub at [convaiinnovations/hindi-causal-lm](https://huggingface.co/convaiinnovations/hindi-causal-lm).

**Model Architecture:**

The `ConvaiCausalLM` is a decoder-only Transformer model with the following key characteristics:

*   **Architecture:** Pre-LayerNorm Transformer Decoder
*   **Attention:** Grouped Query Attention (GQA) - Specifically, `num_attention_heads=16`, `num_key_value_heads=4`.
*   **Positional Embeddings:** Uses standard learned absolute position embeddings (unlike RoPE used in models like Llama/Mistral).
*   **Normalization:** Standard `torch.nn.LayerNorm`.
*   **Activation:** SiLU (SwiGLU) in the MLP layers.
*   **Vocabulary Size:** 16,000 (trained using SentencePiece)
*   **Hidden Size:** 768
*   **Intermediate Size:** 3072
*   **Number of Layers:** 12
*   **Context Length:** 512

**Implementation Details:**

This implementation leverages the **Modular Transformers** framework (`modular_convaicausallm.py`) to minimize code duplication and inherit components where possible:

*   **Configuration (`ConvaiCausalLMConfig`):** Defined specifically for this model due to unique parameters (like `num_key_value_heads` and lack of RoPE config).
*   **Attention (`ConvaiCausalLMAttention`):** Implemented from scratch within the modular file as it uses GQA *without* Rotary Positional Embeddings, differing significantly from common base models like Llama/Mistral. It includes logic for KV caching and GQA KV repetition.
*   **MLP (`ConvaiCausalLMMLP`):** Inherited directly from `LlamaMLP` as the structure (SiLU activation, up/down projection) is identical.
*   **Normalization:** Uses `torch.nn.LayerNorm` directly where needed, replacing the `RMSNorm` inherited from Llama structures.
*   **Decoder Layer (`ConvaiCausalLMDecoderLayer`):** Defined in the modular file, inheriting structure but overriding components to use `ConvaiCausalLMAttention`, the inherited `LlamaMLP`, and `nn.LayerNorm`. The forward pass logic is adapted for the Pre-LN structure.
*   **Model (`ConvaiCausalLMModel`):** Defined in the modular file, inheriting from `ConvaiCausalLMPreTrainedModel`. The `__init__` sets up the specific embeddings, `ConvaiCausalLMDecoderLayer` stack, and final `LayerNorm`. The `forward` pass is adapted from standard decoder models but omits RoPE calculations.
*   **Causal LM Head (`ConvaiCausalLMForCausalLM`):** Defined in the modular file, inheriting from `ConvaiCausalLMPreTrainedModel`. It wraps `ConvaiCausalLMModel` and adds the standard language modeling head (`lm_head`). Includes standard `forward` and `prepare_inputs_for_generation` logic.
*   **Tokenizer (`ConvaiCausalLMTokenizer`):** A standard `PreTrainedTokenizer` wrapper around the SentencePiece model (`tokenizer.model`) used for training.

The `modeling_convaicausallm.py` file is automatically generated from `modular_convaicausallm.py` using the `modular_model_converter.py` script.

**Files Added:**

*   `src/transformers/models/convaicausallm/__init__.py`
*   `src/transformers/models/convaicausallm/configuration_convaicausallm.py`
*   `src/transformers/models/convaicausallm/modeling_convaicausallm.py` (Generated)
*   `src/transformers/models/convaicausallm/modular_convaicausallm.py`
*   `src/transformers/models/convaicausallm/tokenization_convaicausallm.py`
*   `tests/models/convaicausallm/test_modeling_convaicausallm.py` (Assuming tests are added)
*   `tests/models/convaicausallm/test_tokenization_convaicausallm.py` (Assuming tests are added)
*   `docs/source/en/model_doc/convaicausallm.md` (Assuming docs are added)

<!-- Remove if not applicable -->

Fixes # (issue) <!-- If this PR fixes a specific issue, link it here. Otherwise, remove this line. -->

## Before submitting

-   [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case). <!-- Mark 'x' if only docs/typo -->
-   [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request), Pull Request section?
-   [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Link at https://discuss.huggingface.co/t/announcing-convaicausallm-a-foundational-hindi-causal-language-model-102m-yahh-small/152704/1
-   [x] Did you make sure to update the documentation with your changes? Here are the [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation). <!-- Mark 'x' once docs are added/updated -->
-   [x] Did you write any new necessary tests? <!-- Mark 'x' once tests are added -->

## Who can review?

Anyone in the community is free to review the PR once the tests have passed.

Tagging potential reviewers based on contribution guide:

*   Models (text models): @ArthurZucker
*   Library (core modeling, potentially generation): @gante



Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

Looking for an update from @ArthurZucker @Rocketknight1 

Looking for an update 

@ArthurZucker can you look into it

Super sorry for the delay I was swampt!

> Super sorry for the delay I was swampt!

what should be the next step?