`latent_recurrent_depth` Model Type Not Recognized in `transformers`
Author: codewithdark-git
Pull request: no
Labels: name: New model

### Model description

#### **Describe the Issue**  
I have created a custom model (`latent_recurrent_depth`) and uploaded it to Hugging Face (`codewithdark/latent-recurrent-depth-lm`). The model trains successfully using `AutoModel`, `AutoModelForCausalLM`, and its actual class. However, when trying to load it with `from_pretrained()`, I encounter the following error:  

```
ValueError: The checkpoint you are trying to load has model type `latent_recurrent_depth` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.
```

#### **Steps to Reproduce**  
```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "codewithdark/latent-recurrent-depth-lm"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
```

#### **Expected Behavior**  
Since this is my own model, I want `transformers` to recognize `latent_recurrent_depth` and load the model properly.

---

### 🔍 What I Tried  

#### ✅ **1. Confirmed That Training Works**  
During training, the model works fine when instantiated with `AutoModel`, `AutoModelForCausalLM`, and the actual model class. Example:  

```python
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_config("codewithdark/latent-recurrent-depth-lm")  # Works during training
```
- The training pipeline runs without issues.
- The error only occurs when trying to load the model from a checkpoint.

#### ✅ **2. Updating `transformers`**  
```bash
pip install --upgrade transformers
```
- Still getting the same error.

#### ✅ **3. Checking the Model Config**  
```python
from transformers import AutoConfig

config = AutoConfig.from_pretrained("codewithdark/latent-recurrent-depth-lm")
print(config)
```
- The `model_type` is `latent_recurrent_depth`, which is not registered in `transformers`.

#### ✅ **4. Implementing a Custom Model Class**  
Since `transformers` does not recognize my custom architecture, I attempted to manually register it:

```python
from transformers import AutoModel, AutoConfig, PreTrainedModel
import torch.nn as nn

class LatentRecurrentDepthModel(PreTrainedModel):
    config_class = AutoConfig  # Define config class

    def __init__(self, config):
        super().__init__(config)
        self.layer = nn.Linear(config.hidden_size, config.hidden_size)  # Example layer

    def forward(self, input_ids, attention_mask=None):
        return self.layer(input_ids)

# Register model
AutoConfig.register("latent_recurrent_depth", AutoConfig)
AutoModel.register(AutoConfig, LatentRecurrentDepthModel)
```
- However, I’m not sure if this is the correct approach.

---

### ❓ Questions  
1. If the model works during training with `AutoModelForCausalLM`, why does `from_pretrained()` fail?  
2. Is there an official way to add custom architectures to `transformers`?  
3. Do I need to provide a `config.json` file in my Hugging Face model repository for better compatibility?  
4. If I register my model this way, can I still load it from Hugging Face (`from_pretrained()`), or do I need to modify `transformers` source code?  

### 🛠️ Environment  
- `transformers` version: (output of `pip show transformers`)  
- Python version: (e.g., 3.10)  
- OS: (Windows/Linux/Mac)  

Any guidance on how to properly integrate a custom model type into `transformers` would be greatly appreciated! 🚀  

### Open source status

- [x] The model implementation is available
- [x] The model weights are available

### Provide useful links for the implementation

Github --> Model Implementaion code --> https://github.com/codewithdark-git/LatentRecurrentDepthLM.git

HuggingFace --> codewithdark/latent-recurrent-depth-lm

Comments:
Hi @codewithdark-git, when you want to upload a model that isn't contained in the `Transformers` codebase, that's a "custom code" model, and you'll need to do two things:

1) Upload the modeling code in a file like `modeling_latent_recurrent_depth.py`, and also the code for any other custom classes like the config file or tokenizer
2) Set the `auto_map` variable in `config.json`

The error you're getting is because the repo does not have any custom code, and so `Transformers` tries to load the model as a library class, but since it does not exist in the library, it fails with an error.

Please see the full guide here: https://huggingface.co/docs/transformers/en/custom_models

Thanks! Now It's Working.