Add HindiCausalLM: A specialized Hindi language model with grouped query attention (~102M parameters)
Author: NandhaKishorM
Pull request: yes

# What does this PR do?

This PR adds complete support for the HindiCausalLM model, a causal language model specifically designed for Hindi text generation. The model was developed by ConvaiInnovations and the original implementation is available on the [Hugging Face Hub](https://huggingface.co/convaiinnovations/hindi-causal-lm).

HindiCausalLM addresses the need for specialized language models for the Hindi language, which is one of the most widely spoken languages in the world with over 600 million speakers. The model provides improved performance on Hindi text generation tasks compared to multilingual models of similar size.

## Model Architecture and Technical Specifications

HindiCausalLM is based on a decoder-only transformer architecture with the following specifications:

- **Base Architecture**: Transformer decoder-only, similar to LLaMA but with architecture modifications
- **Size**: 12 layers, 768 hidden dimensions, ~102M parameters
- **Attention Mechanism**: Implements Grouped Query Attention (GQA) with 16 attention heads and 4 key-value heads for efficient inference
- **Feed-Forward Network**: Uses intermediate size of 3072 with SiLU activation functions
- **Normalization**: Layer normalization applied before attention and feed-forward blocks (pre-normalization)
- **Vocabulary Size**: 16,000 tokens using a SentencePiece tokenizer optimized for Hindi text
- **Context Window**: Supports sequences up to 512 tokens in length

## Implementation Details

This PR includes:

1. **Model Implementation**:
   - PyTorch implementation with three main model classes:
     - `HindiCausalLMModel`: Base model outputting hidden states
     - `HindiCausalLMForCausalLM`: Model with a language modeling head for text generation
     - `HindiCausalLMForSequenceClassification`: Model with a classification head for sequence classification tasks
   - TensorFlow implementation with equivalent classes for cross-framework compatibility
   - Full support for features like model parallelism, gradient checkpointing, and mixed precision

2. **Tokenizer Implementation**:
   - `HindiCausalLMTokenizer`: SentencePiece-based slow tokenizer
   - `HindiCausalLMTokenizerFast`: Fast tokenizer implementation using 🤗 Tokenizers
   - Special token handling optimized for Hindi text

3. **Generation Support**:
   - Custom generation configuration optimized for Hindi text generation
   - Support for standard generation parameters (temperature, top-k, top-p, etc.)
   - Efficient beam search and sampling implementations

4. **Conversion Utilities**:
   - `convert_hindicausallm_original_pytorch_to_hf.py`: Script for converting original checkpoints to the Hugging Face format
   - Comprehensive weight mapping to ensure compatibility

5. **Auto Class Integration**:
   - Added to all auto classes: `AutoModel`, `AutoModelForCausalLM`, `AutoModelForSequenceClassification`, `AutoTokenizer`, etc.
   - Properly registered with model types and mappings

## Files Added or Modified

New files:
```
src/transformers/models/hindicausallm/
├── __init__.py
├── configuration_hindicausallm.py
├── convert_hindicausallm_original_pytorch_to_hf.py
├── generation_config_hindicausallm.py
├── modeling_hindicausallm.py
├── modeling_tf_hindicausallm.py
├── README.md
├── tokenization_hindicausallm.py
└── tokenization_hindicausallm_fast.py

tests/models/hindicausallm/
├── __init__.py
├── test_modeling_hindicausallm.py
├── test_tokenization_hindicausallm.py
└── test_docstring_hindicausallm.py

docs/source/en/model_doc/hindicausallm.md
```

Modified files:
```
src/transformers/models/__init__.py
src/transformers/models/auto/configuration_auto.py
src/transformers/models/auto/modeling_auto.py
src/transformers/models/auto/tokenization_auto.py
src/transformers/models/auto/tokenization_auto_fast.py
docs/source/en/model_summary.md
```

## Testing

The implementation includes comprehensive tests:

1. **Model Tests**:
   - Basic functionality tests for all model variants
   - Forward pass testing with various input configurations
   - Integration with generation pipelines
   - Weight loading and saving tests
   - Gradient flow verification

2. **Tokenizer Tests**:
   - Tokenization and detokenization verification with Hindi text
   - Special token handling tests
   - Fast and slow tokenizer compatibility tests
   - Serialization and deserialization tests

3. **Documentation Tests**:
   - Docstring example verification
   - Usage pattern validation

4. **Integration Tests**:
   - Model loading from Hub
   - Text generation verification with real Hindi text
   - Model/tokenizer integration tests

All tests pass on both CPU and GPU, with and without mixed precision.

## Documentation

This PR includes comprehensive documentation:

1. **Model Card**:
   - Detailed model card in `docs/source/en/model_doc/hindicausallm.md`
   - Architecture description
   - Usage examples
   - Performance characteristics

2. **API Documentation**:
   - Well-documented API for all classes
   - Parameter descriptions and type annotations
   - Example usage in docstrings

3. **README**:
   - Internal README.md with implementation details
   - Links to original model and paper

## Usage Examples

### Basic Usage for Text Generation

```python
from transformers import HindiCausalLMForCausalLM, HindiCausalLMTokenizer

# Load model and tokenizer
model = HindiCausalLMForCausalLM.from_pretrained("convaiinnovations/hindi-causal-lm")
tokenizer = HindiCausalLMTokenizer.from_pretrained("convaiinnovations/hindi-causal-lm")

# Generate text
input_text = "भारत एक विशाल देश है"  # "India is a vast country"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids

# Generate
outputs = model.generate(
    input_ids, 
    max_length=50, 
    num_return_sequences=1, 
    temperature=0.7, 
    top_p=0.9,
    do_sample=True
)

# Decode the generated text
generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(generated_text)
```

### Classification Example

```python
from transformers import HindiCausalLMForSequenceClassification, HindiCausalLMTokenizer
import torch

# Load model and tokenizer
model = HindiCausalLMForSequenceClassification.from_pretrained(
    "convaiinnovations/hindi-causal-lm", num_labels=3
)
tokenizer = HindiCausalLMTokenizer.from_pretrained("convaiinnovations/hindi-causal-lm")

# Example texts
texts = ["यह एक सकारात्मक समीक्षा है", "यह एक नकारात्मक समीक्षा है", "यह एक तटस्थ समीक्षा है"]
inputs = tokenizer(texts, padding=True, return_tensors="pt")

# Forward pass
with torch.no_grad():
    outputs = model(**inputs)
    predictions = torch.argmax(outputs.logits, dim=1)

print(f"Predictions: {predictions}")
```

### TensorFlow Example

```python
from transformers import TFHindiCausalLMForCausalLM, HindiCausalLMTokenizer
import tensorflow as tf

# Load model and tokenizer
model = TFHindiCausalLMForCausalLM.from_pretrained("convaiinnovations/hindi-causal-lm")
tokenizer = HindiCausalLMTokenizer.from_pretrained("convaiinnovations/hindi-causal-lm")

# Process input
input_text = "हिंदी भाषा बहुत समृद्ध है"  # "Hindi language is very rich"
inputs = tokenizer(input_text, return_tensors="tf")

# Generate
generated_ids = model.generate(inputs.input_ids, max_length=30)
generated_text = tokenizer.decode(generated_ids[0], skip_special_tokens=True)
print(generated_text)
```

## Model Performance

The HindiCausalLM model demonstrates strong performance on Hindi text generation tasks. Based on evaluations, it shows significant improvements over generic multilingual models of similar size on Hindi-specific tasks:

- **Text Generation**: Produces fluent and coherent Hindi text
- **Hindi Understanding**: Demonstrates good understanding of Hindi grammar and syntax
- **Context Handling**: Maintains context effectively through longer generated sequences

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request), Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? This model is publicly available on the Hugging Face Hub and meets the criteria for inclusion in the Transformers library.
- [x] Did you make sure to update the documentation with your changes? Here are the [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests? Yes, comprehensive tests have been added for model and tokenizer functionality.

## Who can review?
@ArthurZucker - For text model implementation review
@Rocketknight1 - For TensorFlow implementation review

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

removed .history folder and revert the styles